Multivariate Statistical Process Control Using Enhanced Bottleneck Neural Network

Abstract

Monitoring process upsets and malfunctions as early as possible and then finding and removing the factors causing the respective events is of great importance for safe operation and improved productivity. Conventional process monitoring using principal component analysis (PCA) often supposes that process data follow a Gaussian distribution. However, this kind of constraint cannot be satisfied in practice because many industrial processes frequently span multiple operating states. To overcome this difficulty, PCA can be combined with nonparametric control charts for which there is no assumption need on the distribution. However, this approach still uses a constant confidence limit where a relatively high rate of false alarms are generated. Although nonlinear PCA (NLPCA) using autoassociative bottle-neck neural networks plays an important role in the monitoring of industrial processes, it is difficult to design correct monitoring statistics and confidence limits that check new performance. In this work, a new monitoring strategy using an enhanced bottleneck neural network (EBNN) with an adaptive confidence limit for non Gaussian data is proposed. The basic idea behind it is to extract internally homogeneous segments from the historical normal data sets by filling a Gaussian mixture model (GMM). Based on the assumption that process data follow a Gaussian distribution within an operating mode, a local confidence limit can be established. The EBNN is used to reconstruct input data and estimate probabilities of belonging to the various local operating regimes, as modelled by GMM. An abnormal event for an input measurement vector is detected if the squared prediction error (SPE) is too large, or above a certain threshold which is made adaptive. Moreover, the sensor validity index (SVI) is employed successfully to identify the detected faulty variable. The results demonstrate that, compared with NLPCA, the proposed approach can effectively reduce the number of false alarms, and is hence expected to better monitor many practical processes.

1. Introduction

Recently, increasing sensor availability in process monitoring has led to higher demands on the ability to early detect and identify any sensor faults, especially when the monitoring procedure is based on the information obtained from many sensors. Thus, monitoring or control using accurate measurements is very useful to enhance process performance and improve production quality. In the literature, multivariate statistical tools have been widely used to enhance process monitoring [1,2,3,4,5,6,7,8,9,10,11]. Among the popular multivariate statistical process control, the traditional principal component analysis (PCA) and the independent component analysis (ICA) [10,12,13] are the most early and major methods often used in monitoring, which serve as reference of the desired process behaviour and against which new data can be compared (Rosen and Olsson, 2007). However, recent modification of them, such as multiscale PCA, robust NLPCA, kernel PCA, and ICA–PCA combination are used in the literature (Bakshi, 1998, Xiong Li et al., 2007, Zhao and Xu, 2005). Generally, traditional industrial process monitoring relies on the Gaussian assumption of mean $\mu$ and variance $\sigma$ which leads to obtaining a confidence limit as constant threshold. However, in practice, the process variables follow approximately mixture Gaussian distributions $({\mu }_j,{\sigma }_j)$ due to process nonlinearity, which gives a multimodal behaviour; therefore, an adaptive confidence limit (ACL) is expected to improve the process performance. In this context, we propose a robust process monitoring strategy based on Gaussian mixture model (GMM) to extract multiple normal operating modes characterized by m Gaussian components $({\mu }_j,{\sigma }_j)$ during normal conditions [14,15].

In this work, an enhanced bottleneck neural network (EBNN) is used to estimate the monitored measurements and to classify them according to their probability rates of belonging to various operating modes. Compared to the classical auto-associative neural networks (AANNs) that can be used only for mapping inputs to outputs, the proposed EBNN is not limited to mapping the outputs from input dataset, but also provides a supervised classification of the monitored variables. Then, the kernel density estimation has been adopted to calculate the local confidence limits of the various operating modes, [14,16,17,18].

Subsequently, an ACL is obtained through a weighted sum of the probability rates of each normal mode estimated by the m neurons of the EBNN output layer dedicated to classification and the m Local Confidence Limits $LC{L}_j^{(KDE)}$ of each local $SPEj$.

The obtained ACL is associated with the global $SPE(k)$, which is able to significantly improve the monitoring performance, reduce the false alarms, and detect the onset of process deviations earlier compared to other traditional methods. Traditional contribution plots to the squared prediction error (SPE) are used to isolate the defective sensor. In order to overcome the drawback of the contribution plots, we have successfully proposed a variant of the sensor validity index (SVI) method, which is based on the elimination of the fault on the SPE by reconstructing the faulty variable, the Figure 1 shows a monitoring cycle of the proposed strategy.

The paper is organised as follows: Section 2 introduces the multivariate monitoring tools, including the Gaussian mixture model for calculating the probability rates of normal operating modes, which are used as targets to train the EBNN. Section 3 is dedicated to the case study, in which the effectiveness of the proposed process monitoring strategy is investigated, where we discuss and assess the different results obtained from the used process monitoring tools which are tested on benchmark simulation model no. 1 (BSM1), which is introduced by the working group of the International Water Association (IWA) task group on benchmarking of control strategies for wastewater treatment plants and on actual process. Finally, conclusions are described in Section 4.

2. Materials and Methods

2.1. Multivariate Statistical Process Control

2.1.1. Gaussian Mixture Model

The use of Gaussian mixture models for modelling data distribution is motivated by the interpretation that the Gaussian components represent some general underlying physical phenomena and the capability of Gaussian mixtures to arbitrary shaped densities model densities. This subsection describes the form of the Gaussian mixture model (GMM) and motivates its use as a representation of historical data for advanced statistical process monitoring (SPM). The use of Gaussian mixture density for process monitoring is motivated by two interpretations. First, the individual component Gaussians are interpreted to represent some operating modes or classes. These classes reflect some underlying phenomena having generated data; for example, hydrological or biological in water treatment plants. Second, a Gaussian mixture density is shown to provide a smooth approximation of the underlying long-term sample distribution of observations collected from industrial plants. The GMM parametrization and the maximum-likelihood parameter estimation are described. Multiple operating modes in normal process conditions can be characterized by different Gaussian components in GMM, and the prior probability of each Gaussian component represents the percentage of total operation when the process runs at the particular mode [19,20]. The probability density function of Gaussian mixture model is equivalent to the weighted sum of the density functions of all Gaussian components as given below:

$$p(x|\wedge )=\sum _{j=1}^K{\beta }_{j}g({\mathrm{x}}_{\mathrm{i}}|{\lambda }_j)$$

(1)

where $\mathrm{x}$ $\in R^l$, ${\beta }_j$ denotes the prior probability of the $jth$ Gaussian component and $g(x|{\lambda }_j)$ is the multivariate Gaussian density function of the $jth$ component. For each component, the model parameters to be estimated are ${\beta }_j$ and ${\lambda }_j=\left\{{\mu }_j,{\sigma }_j\right\}$, the latter of which include the mean vector ${\mu }_j$ and the covariance matrix ${\sigma }_j$ (Duda et al., 2001). During the model learning, the following log-likelihood function is used as objective function to estimate the parameter values

$$logL(x|\wedge )=\sum _{i=1}^{N}log\left(\sum _{j=1}^K{\beta }_{j}g({\mathrm{x}}_{\mathrm{i}}|{\lambda }_j)\right)$$

(2)

where ${\mathrm{x}}_{\mathrm{i}}$ is the $ith$ training sample among the total N measurements.

The Gaussian mixture model can be estimated by Expectation-Maximization algorithm through the following iterative procedure:

• E-Step: compute the posterior probability of the $ith$ training sample ${\mathrm{x}}_{\mathrm{i}}$ at the $sth$ iteration

  $$P^{(s)}(m_j|{\mathrm{x}}_{\mathrm{i}})=\frac{{\alpha }_j^{(s)}g({\mathrm{x}}_{\mathrm{i}}|{\lambda }_j^{(s)})}{{\sum }_{l=1}^K{\beta }_l^{(s)}g({\mathrm{x}}_{\mathrm{i}}|{\lambda }_l^{(s)})}$$

  (3)

  where $m_j$ denotes the $jth$ Gaussian component.
• M-Step: update the model parameters at the $(s+1)th$ iteration

  $${\mu }_j^{(s+1)}=\frac{{\sum }_{i=1}^N{P}^{(s)}(m_j|{\mathrm{x}}_{\mathrm{i}}){\mathrm{x}}_{\mathrm{i}}}{{\sum }_i^N{P}^{(s)}(m_j|{\mathrm{x}}_{\mathrm{i}})}$$

  (4)

  $${\sigma }_j^{(s+1)}=\frac{{\sum }_{i=1}^N{P}^{(s)}(m_j|{\mathrm{x}}_{\mathrm{i}})({\mathrm{x}}_{\mathrm{i}}-{\mu }_j^{(s+1)}){({\mathrm{x}}_{\mathrm{i}}-{\mu }_j^{(s+1)})}^T}{{\sum }_i^N{P}^{(s)}(m_j|{\mathrm{x}}_{\mathrm{i}})}$$

  (5)

  $${\beta }_j^{(s+1)}=\frac{{\sum }_{i=1}^N{P}^{(s)}(m_j|{\mathrm{x}}_{\mathrm{i}})}{N}$$

  (6)

It should be noted that the number of Gaussian components corresponds to the number of operating modes in normal conditions [21]. The next phase after extracting the mixture operating modes consists to design the EBNN model.

2.1.2. Enhanced Bottleneck Neural Network

Auto-associative neural networks are powerful tools for mapping inputs to outputs, especially when this mapping is nonlinear. In this work, an enhanced bottleneck neural network (EBNN) topology is used that includes five layers with three hidden layers: mapping layer, bottleneck layer, and de-mapping layer, the input and output variables [13,22,23]. Once the general structure is defined, it remains to determine the necessary number of neurons in each hidden layer. This number is generally determined by cross-validation. In addition, the classification using feedforward artificial neural networks is an effective tool to produce a specific output pattern for a specific input pattern. For this purpose, a small artificial neural network classifier (ANNC) is included in the output layer of EBNN (part dedicated to estimating the probability rates of normal operating modes) as shown in Figure 2. ANNC is used because of its ability to estimate the probability rates of the monitored variables, leading to their classification according to their operating modes. In the training phase, the number of normal operating modes to be extracted is assumed to be known.

The training of the whole network includes two steps:

• Encoding process (compression): For an input vector $x_i$, the encoding process (compress inputs) can be described as:

  $$t_k=\sum _{j=1}^h{w}_{jk}^{(2)}\sigma \left(\sum _{i=1}^n{w}_{ij}^{(1)}{x}_i+{\theta }_i\right),k=1,2,\cdots ,q$$

  (7)

  When the bottleneck layer forces the EBNN to compress the inputs (variables to be monitored).
• Decoding process (decompression): The bottleneck layer produces the network outputs (estimation and classification) by decompression, which is given as:

  $${\widehat{x}}_k=\sum _{j=1}^h{w}_{jk}^{(4)}\sigma \left(\sum _{i=1}^q{w}_{ij}^{(3)}{t}_i+{\theta }_i\right),k=1,2,\cdots ,n$$

  (8)

  $${\widehat{\rho }}_m=\sum _{j=1}^h{w}_{jk}^{(4)}\sigma \left(\sum _{i=1}^q{w}_{ij}^{(3)}{t}_i+{\theta }_i\right),k=1,2,\cdots ,m$$

  (9)

  with

  -

  n is the number of neurons in the input layer.

  -

  h is the number of neurons in the mapping layer.

  -

  q is the number of neurons in the bottleneck layer.

  -

  w is the weight values of the network.

  -

  ${\theta }_i$ is the threshold value for the $ith$ node of the mapping layer.

  -

  m is the number of neurons in the output layer of the operating modes classification part.

  -

  $\sigma$ is the sigmoid transfer function,

  where the transfer function in the mapping and de-mapping layers is sigmoid, and in bottleneck and output layers is linear. Exceptionally, in the output classification part is log-sigmoid in order to generate a posterior probability rates vary between 0 and 1.

The gradient descent algorithm is adopted to train the BNN and to find the optimal values [24,25]. This is done iteratively by changing the weights w and threshold $\theta$ (randomly initialized before the training) according to the gradient descent so that the difference between the ideal output and the real output is the smallest. The training process is finished when the cost function E for all samples is minimized, which is calculated as:

$$E=\sum _{n=1}^N\sum _{i=1}^n{\left(x_i^n-{\widehat{x}}_i^n\right)}^2+\sum _{n=1}^N\sum _{j=1}^m{\left({\rho }_j^n-{\widehat{\rho }}_j^n\right)}^2$$

(10)

where

-

N is the number of training samples.

-

n is the number of neurons in the output layer of the estimation part.

-

m is the number of neurons in the output layer of the modes classification part.

the variables estimation part:

-

$x_i^n$ is the n desired values of the $ith$ output neuron.

-

${\widehat{x}}_i^n$ is the n actual outputs of that neuron.

the modes classification part:

-

${\rho }_j^n$ is the probability rate already obtained with GMM of the $jth$ mode.

-

${\widehat{\rho }}_j^n$ is the actual output of $jth$ neuron which corresponds to jth mode.

It should be noted that the number of neurons in the output layer of the modes classification part corresponds to the extracted normal operating modes.

2.2. Process Monitoring and Diagnosis

2.2.1. Squared Prediction Error

The difference between the enhanced BNN inputs and their estimated values can be used as an indicator to judge the occurrence of the abnormal events and their severity. Several multivariate extensions of fault detection approaches have been proposed in literature; the univariate statistic SPE, which is established using the error $e(k)$ that follows a central Chi-squared distribution [9,26] has a fundamental importance for process monitoring because it indicates the changes in the correlation structure of the measured variables. Its expression at the instant k is given by:

$$SPE(k)=e(k)e{(k)}^T$$

(11)

where;

$$e(k)=x(k)-\widehat{x}(k)$$

(12)

In offline stage, once the EBNN model is designed, it should be noted that m local $SP{E}_j^{(\ell )}$ and $UC{L}_j^{(KDE)}$ must be calculated in residual subspace for the various operating modes.

In online phase, the global $SPE(k)$ (which is multimodal) provides fault detection in residual space. The process is considered in abnormal operating state at the $kth$ observation if the global $SPE(k)$:

$$SPE(k)>AUC{L}^{(KDE)}$$

(13)

with $AUC{L}^{(KDE)}$ is the adaptive upper control limit of the global $SPE(k)$.

2.2.2. Adaptive Upper Control Limits

Once the constants $UC{L}^{(KDE)}$ of each $SP{E}_j^{(\ell )}$ and the probability rates of each operating mode are calculated, an adaptive upper control limit can be obtained according to the following formula:

$$AUCL=\sum _{j=1}^m\left(UC{L}_j^{(KDE)}{\rho }_j(k)\right),k=1,2,...,n$$

(14)

where

-

m is number of Gaussian components corresponding to the normal operating modes.

-

${\rho }_j$ is the probability rate of $jth$ mode during the normal operating regime.

-

$UC{L}_j^{(KDE)}$ is the upper control limit using the kernel density estimation (KDE) of each mode during the normal operating regime.

-

n is the number of samples.

2.2.3. Upper Control Limit by KDE

$KDE$ is a powerful data-driven technique for the nonparametric estimation of density functions [27,28,29]. Given a sample matrix with n variables and l samples ${\left\{x_1,x_2,\cdots ,x_l\right\}}^T$ $\in R^{l\times n}$, a general estimator of the density function $f(x)$ can be defined as:

$$\widehat{f}(x,H)=\frac{1}{l}\frac{1}{{|H|}^{(n+1)/2}}\sum _{i=1}^{n}K[H^{-1/2}(x-x_i)H^{-1/2}]$$

(15)

where $H\in R^{n\times n}$ is the bandwidth matrix, $\left|H\right|$ indicates the determinant of H, and $K(.)$ is the kernel function, where the kernel estimator is the sum of the bumps placed at the sample points. The $K(.)$ defines the shape of the bumps, while the bandwidth defines the width. A popular choice for $K(.)$ is the Gaussian kernel function:

$$k(x)={(2\pi )}^{-n/2}exp\left(-\frac{1}{2}{x}^{T}x\right)$$

(16)

The critical issue for determining $\widehat{f}(x,H)$ is the selection of bandwidth matrix H. In order to determine H, we must first determine its matrix structure. Generally, there are three possible matrix structure cases:

(1)

a full symmetrical positive definite matrix with $n(n+1)/2$ parameters $h_{rj}^2$—for example, in which $h_{rj}^2$ = $h_{jr}^2$;

(2)

a diagonal matrix with only l parameters, $H=diag(h_1^2,h_2^2,\cdots ,h_n^2)$;

(3)

a diagonal matrix with one parameter, $H=h^{2}I$, where, I is unit matrix.

According to [30], the first case is the most accurate, but is unrealistic in terms of computational load and time, while the third case is the simplest but can cause problems in certain situations due to the loss of accuracy by forcing the bandwidths in all dimensions to be the same. In this study, the second matrix structure bandwidth will be studied as a compromise of the first case and the third case, and an optimal selection of bandwidth matrix algorithm will be used [31].

The above $KDE$ approach is able to estimate the underlying probability density function (PDF) of the $SP{E}_j^{(\ell )}$ matrix. Thus, the corresponding upper control limit $UC{L}_j^{(KDE)}$ can be obtained from the $PDF$ of the $SP{E}_j^{(\ell )}$ matrix with a given confidence bound $\alpha$ by solving the following equation:

$${\int }_{\alpha }^{UC{L}_j^{(KDE)}}P(SP{E}_j^{(\ell )})dSP{E}_j^{(\ell )}=\alpha ,j=1,2,\cdots ,m$$

(17)

where $P(SP{E}_j^{(\ell )})$ is the probability density function of $SP{E}_j^{(\ell )}$ for normal operating data, $\alpha$ is the confidence level, and $SP{E}_j^{(\ell )}$ are the local $SP{E}_j$.

Therefore, the KDE is a well-established approach to estimating the PDF, particularly for random variables like $SPE$ which are univariate although the underlying processes are multivariate.

2.3. Fault Isolation

After detecting a fault, it is necessary to identify which sensor becomes faulty. In this section, we re-examine the concept of contribution plots and SVI based on nonlinear reconstruction.

2.3.1. Isolation by Contribution Plots

The contribution plot is the most conventional method that has been widely used in fault isolation approaches. This method is generally based on the contribution rate of each variable to indicate which variable has an extreme contribution to the $SP{E}_j$. These contributions can be calculated as:

$$Con{t}_j^{SPE}(k)={(e_j(k))}^2={({\mathrm{x}}_{\mathrm{j}}(k)-\widehat{{\mathrm{x}}_{\mathrm{j}}}(k))}^2$$

(18)

2.3.2. Sensor Validity Index

This approach is applied when a fault sensor is detected. Thus, it consists of reconstructing its measurement using the EBNN model which was already trained in normal operating conditions. Therefore the isolation is performed by calculating the $SP{E}_j$ indicators of each measurement ($j=1,2,\cdots ,N_1$ is the number of sensors) after the reconstruction of all sensors. It is noted that the reconstruction of the faulty sensors eliminates the abnormal condition; we must also remember that the $jth$ indicator $SP{E}_j$ of the faulty sensor calculated after reconstruction is below its adaptive upper control limit $SP{E}_j(k)<AUCL$ compared to the $SP{E}_j(k)$ of other variables, which are all above their respective $AUC{L}_j$ [32].

The reconstructed measurement $x_j(k)$ can be obtained iteratively, estimated, and re-estimated (re-injected) by EBNN model until convergence as indicated in Figure 3.

The SVI is based on the reconstruction principle, which consists to suspect a faulty sensor and reconstruct (or correct) the value of the faulty sensor based on the EBNN model already performed and the other measurements of all sensors; the procedure is repeated for each sensor. The identification is performed by comparing the SVIs before and after reconstruction. The sensor validity index is a measure of sensor performance; it should have a standardized range regardless of the number of principal components in the bottleneck layer, noise, measurement variances, or fault type. The ratio of $SP{E}_j(k)$ and the global SPE can provide these desired properties for the identification of a sensor fault [4]). The SVI can be calculated as:

$${\eta }_j^2=\frac{SPE}{SP{E}_j}$$

(19)

where the SPE is the global squared prediction error calculated before reconstruction and SPE${}_j$ is the $jth$ sensor calculated after reconstruction, [33,34,35].

3. Case Study : Wastewater Treatment Plant (WWTP) Monitoring

The main purpose in biological wastewater treatment is to transform nitrogen compounds in order to remove them from wastewater, where admissible concentrations of these harmful pollutants are defined by European standards. In order to generate a limit for the toxic organic chemicals which have recently caused a lot of harm both to human beings and the environment, stringent laws have been increasingly imposed on the quality of wastewater discharged in industrial and municipal effluent. This rising concern about the environmental protection leads the wastewater treatment plants (WWTPs) to the necessity of running under normal operating conditions. To improve the operating performance of wastewater treatment plants like all industrial processes, an on-line monitoring system equipped with electronic sensors is necessary for early detection and identification of any abnormal event affecting the process [36,37]. The proposed strategy is applied to the simulated wastewater treatment process BSM1 and to the real-process, where many sensor are prone to malfunctions [38,39,40].

3.1. Simulated Process Case

This benchmark plant is composed of a five compartment activated sludge reactor consisting of two anoxic tanks followed by three aerobic tanks (see Figure 4). The plant thus combines nitrification with pre-denitrification in a configuration that is commonly used for achieving biological nitrogen removal in full-scale plants. The activated sludge reactor is followed by a secondary settler. The simulated wastewater treatment process BSM1 (Figure 5) is used under dry weather during 7 days under normal operating conditions. The sampling period is 15 min, where the training dataset is composed of 673 observations and 672 observations are used for the test phase. The plant is designed for an average influent dry-weather flow rate of 18,446 m${}^3·$d${}^{-1}$ and an average biodegradable chemical oxygen demand (COD) in the influent of 300 g·m${}^{-3}$. For more details, the simulated wastewater treatment process BSM1 is available on [41].

The monitored sensors of the BSM1 used in this work are given in

Table 1. The monitored sensors for BSM1 plant.

N	Monitored Sensors	Notation
1	Dissolved oxygen in $Tan{k}_1$	$S_O$
2	Nitrate and nitrite nitrogen in $Tan{k}_1$	$S_{NO}$
3	$N{H}_{4^{+}}+N{H}_3$ nitrogen in $Tan{k}_2$	$S_{NH}$
4	Soluble biodegradable organic nitrogen in $Tan{k}_2$	$S_{ND}$
5	Particulate biodegradable organic nitrogen in $Tan{k}_2$	$X_{ND}$
6	Dissolved oxygen in $Tan{k}_3$	$S_O$
7	Nitrate and nitrite nitrogen in $Tan{k}_3$	$S_{NO}$
8	$N{H}_{4^{+}}+N{H}_3$ nitrogen in $Tan{k}_5$	$S_{NH}$
9	Soluble biodegradable organic nitrogen in $Tan{k}_5$	$S_{ND}$
10	Particulate biodegradable organic nitrogen in $Tan{k}_5$	$X_{ND}$

.

3.2. Real Process Case

In order to illustrate the effectiveness of the proposed strategy, the monitoring approaches have been tested on real data collected from an actual wastewater treatment process of Annaba, situated in the North-East Algeria (450 km of Algiers and 8 km east of Annaba city), where the real data contains 1300 observations of some sensors (see

Table 2. The monitored sensors for the real process.

N	Monitored Sensors	Notation
1	Dissolved oxygen in influent	$S_O$
2	Nitrites in influent	$S_{NO2}$
3	Nitrates in influent	$S_{NO3}$
4	Ammoniacal nitrogen in influent	$N{H}_4$
5	Chemical oxygen demand in influent	$COD$
6	Dissolved oxygen in effluent	$S_O$
7	Nitrites in effluent	$S_{NO2}$
8	Nitrates in effluent	$S_{NO3}$
9	Ammoniacal nitrogen in effluent	$N{H}_4$
10	Chemical oxygen demand in effluent	$COD$

) in normal operating conditions, with a medium flow of 83,620 m${}^3$/day and peak flow during dry weather 5923 m${}^3$/h.

The purification process used is activated sludge, and has two principal treatment sectors; the primary includes bar racks, grit chamber, de-oiling, and sand filters, whose objective is the removal of solids. The secondary sector is made for removing the organic matter present in the influent wastewater, where the organic matter pollutants serve as food for microorganisms. This is done by using either aerobic or anaerobic treatment processes. The schematic of the used actual plant is shown in Figure 6. The monitored sensors in this work are given in Table 2.

3.3. Results and Discussion

This section presents a simulation study with some comments about the obtained results to examine the effectiveness of the proposed strategy for on-line process monitoring applied to simulated wastewater treatment process BSM1 and to real data.

In order to illustrate the advantages of the AUCL based on EBNN compared to two base-line multivariate statistical monitoring methods (namely the nonlinear PCA (NLPCA) and ICA), simulation tests were carried out on BSM1 and on actual data. Figure 7 and Figure 8 show the SPE during normal conditions using ICA of BSM1 data and real data, respectively. The upper control limit (UCL) was made constant.

Figure 9 and Figure 10 present the SPE using the traditional NLPCA based on AANN with a constant UCL of the BSM1 data and real data, respectively. The high number of false alarms can be seen. In the following part of this section, the results of the proposed method using UACL based on EBNN show the effectiveness in overcoming this drawback.

Before the construction of EBNN, a normalization of the monitored variables is necessary. Each variable of the training and testing data matrices are centred and scaled by subtracting and dividing, respectively, the mean/standard-deviation from/by each column to render the results independent of the used units.

In the testing phase under normal operating conditions, as illustrative examples, three of the estimated variables and the output classified modes (for q = 5 the number of components in bottleneck layer and m = 5 the number of normal operating modes) are plotted in Figure 11 and Figure 12 for the BSM1 case and Figure 13 and Figure 14 for real data, respectively.

It can be clearly seen that the monitored variables have been reconstructed and the probability rates are successfully estimated.

Figure 15 for BSM1 data and Figure 16 for real data represent the evolutions of the filtered SPE for BSM1 and real data, respectively, with an AUCL using KDE at a confidence level of $95\%$ under normal operating conditions. To reduce the false alarms, an exponentially weighted moving average (EWMA) is used to filter the effect of outliers and noise, as shown in figures of the filtered SPE.

The formulation of the process monitoring strategy of this work focuses on the problem of faulty sensors precisely in WWTP, which may exhibit partial failures such as precision degradation due to significant noise, bias, or drift due to sludge clogging [42,43], which can be defined below:

• a- Precision degradation: The precision degradation model is defined as a Gaussian random process with zero mean and unknown covariance matrix.
• b- Bias: The bias error evolution can be characterized by positive or negative value.
• c- Drift: This error follows an increasing deviation, such as polynomial change.

Figure 17 and Figure 18 show the results from a precision degradation fault tested on both cases of data, where the filtered SPE moves up and down many times above its AUCL.

Bias fault can be detected by filtered SPE, as shown in Figure 19 for BSM1 data and Figure 20 for real data.

The last simulation test concerns the drift fault, which can also be detected by filtered SPE, as indicated in Figure 21 and Figure 22 for BSM1 data and real data, respectively.

It is worth noting that the selected abnormal events correspond, respectively, to precision degradation, bias, and drift malfunctions related to the sensors of dissolved oxygen, nitrate, and nitrite nitrogen, which are the most important sensors in WWTP. However, $S_O$ represents a high correlation with the other variables. It can be seen that the SPE indicator exceeds its adaptive upper control limit in the case of bias and drift faults; however, in the precision degradation case, it can be seen that this malfunction type is translated by discontinuous alarms despite the use of AUCL. It is clear that the different faults are detected at various samples according to the type of fault, as indicated in

Table 3. Summary of fault scenarios.

Fault type	Precision degradation	Bias	Drift
Fault expression	${\zeta }_i=w(\mu ,{\sigma }^2)$	${\zeta }_i=const$	${\zeta }_i=k1*{ϵ}_{i-1}^n+k2*{ϵ}_{i-1}^{n-1}+...k_0$.
Faulty sensor expression	$Sensor(t)+(0,{\sigma }^2)$	$Sensor(t)+1.2$	$Sensor(t)+0.5t$
Fault time	336	366	366
2*Detection time	349.4 for BSM1 data and	336.7 for BSM1 data and	342.3 for BSM1 data and
	346.2 for real data	337.2 for real data	392.1 for real data

. The results show the proposed robust monitoring performance in the presence of faults and produces few alarms. For identification, we cannot observe anything in the previous figures.

Sensor Fault Identification and Reconstruction

Once a fault is detected, it is necessary to identify the faulty sensor according to the contribution plots approach, as given in Figure 23 and Figure 24 for BSM1 data and real data, respectively (drift fault case), where in the detection moment the $S_O$ sensor having the largest contribution to the SPE is the faulty sensor. To overcome the drawbacks of the traditional contribution plots, identification via reconstruction is proposed, as illustrated in the figures below.

Figure 25 and Figure 26 illustrate the sensor validity indexes of BSM1 data and real data of all sensors after reconstruction for bias fault, and Figure 27 and Figure 28 for drift fault, in which the faults are eliminated by the correction of erroneous measure using the reconstruction principle. In this way, it is clear that the corrected sensor is faulty.

The SVIs can be immediately displayed on-line as soon as the abnormal event is detected. According to these SVIs which are not limited to isolating the faulty sensor, but can also show the correlation of the faulty sensor with other variables, in this way it can be seen that some sensors have been influenced by the reconstruction (as the sensor having the second largest contribution; see Figure 23 and Figure 24) due to high correlation with the faulty sensor. This can help the operator to provide a feasible interpretation. It should be noted that we can easily test any type of fault on any sensor, except the precision degradation fault is not localizable by the contribution plots or SVIs.

Finally, the comparative study shows that the proposed strategy has superior performance and is more effective for fault monitoring and diagnosis compared to the traditional methods like NLPCA or ICA, which are sensitive with the false alarms because of their constant confidence limit, and also are considered very late in fault detection.

4. Conclusions

The study described in this work concerns an enhanced control chart to monitor nonlinear processes and detect abnormal behaviours. The proposed strategy combines a Gaussian mixture model (GMM) with enhanced bottleneck neural networks (EBNN) to improve multivariate statistical process control (MSPC) using an adaptive upper control limit which is able to reduce the false alarms usually generated when the traditional upper control limit is used, since the proposed EBNN not only provides an estimated value of the input vector, but also probability rates of belonging to different operating modes. In addition, the filter applied to the SVI and SPE adds an important feature to minimize the number of false alarms.

To identify the faulty sensor, SVI based on the reconstruction error of the erroneous measure is successfully employed which gives a better diagnosis rate compared to the traditional contribution plots. However, the reconstruction allows the faulty sensor to be identified by comparing the sensor validity indexes before and after reconstructions of all measurements for each sensor—this can lead to the isolation of the faulty sensor.