A Weighted EFOR Algorithm for Dynamic Parametrical Model Identification of the Nonlinear System

Abstract

In this paper, the Nonlinear Auto-Regressive with exogenous inputs (NARX) model with parameters of interest for design (NARX-M-for-D), where the design parameter of the system is connected to the coefficients of the NARX model by a predefined polynomial function is studied. For the NARX-M-for-D of nonlinear systems, in practice, to predict the output by design parameter values are often difficult due to the uncertain relationship between the design parameter and the coefficients of the NARX model. To solve this issue and conduct the analysis and design, an improved algorithm, defined as the Weighted Extended Forward Orthogonal Regression (WEFOR), is proposed. Firstly, the initial NARX-M-for-D is obtained through the traditional Extended Forward Orthogonal Regression (EFOR) algorithm. Then a weight matrix is introduced to modify the polynomial functions with respect to the design parameter, and then an improved model, which is referred to as the final NARX-M-for-D is established. The genetic algorithm (GA) is used for deriving the weight matrix by minimizing the normalized mean square error (NMSE) over the data sets corresponding to the design parameter values used for modeling and first prediction. Finally, both the numerical and experimental studies are conducted to demonstrate the application of the WEFOR algorithm. The results indicate that the final NARX-M-for-D can accurately predict the system output of a nonlinear system. The new algorithm is expected to provide a reliable model for dynamic analysis and design of the nonlinear system.

1. Introduction

For the nonlinear systems, in general, the conventional dynamic modeling methods are often difficult or even impossible to obtain the motion equations describing the dynamical characteristics due to the uncertainty, time variability, and complexity [1,2]. An alternative approach to this issue is the data-driven modeling method known as system identification, which can be used to establish the numerical model of a nonlinear system by the input and output data sets. The Volterra model [3,4], some modern Volterra series approaches [5,6], and Hammerstein–Wiener model [7,8,9] have been widely used for representing the numerical representation of the nonlinear system. In the past years, the Nonlinear Auto-Regressive with exogenous inputs (NARX) model, which was introduced in 1985 [10,11], was widely used for establishing the nonlinear system models for prediction and analysis due to the convenience (see [12,13,14,15]). Among the numerous related model classes and neural network models, the NARX model is commonly adopted in many real-world cases, and this model is an expansion of past exogenous inputs and outputs. The identification algorithms developed based on NARX models have generally proven efficient for a wide range of nonlinear systems. Compared with neural network models, which have strong fitting ability, the NARX model is more convenient and can provide an explicit model structure to facilitate analysis and design [16]. However, the NARX model cannot represent the Common Model Structure (CMS) of the system and provide insight into the effect of design parameters on system response [17].

In recent years, researchers have looked for a modeling framework that involves design parameters to characterize better the dynamic behavior of the underlying system with different parameter values. The NARX model with parameters of interest for design (NARX-M-for-D) provides an effective solution to this issue, which based on the NARX model and connects the design parameter to the coefficients of the NARX model by a polynomial function [18]. The critical tasks in identification the NARX-M-for-D include detecting the CMS, estimating the associated coefficients, and establishing the function of the coefficients of the NARX model with respect to the design parameter [19]. To obtain the CMS and build a parsimonious model, Wei et al. [17] proposed the Extended Forward Orthogonal Regression (EFOR) algorithm, moreover, the Average Approximate Minimum Description Length (AAMDL) criterion is introduced to determine the total number of model terms. Incorrect model structure [20,21] can be obtained due to the overlapped information by using the EFOR algorithm, the Iterative EFOR (IEFOR) algorithm proposed by Liu et al. [19] can be used to overcome this issue. The IEFOR is developed based on the idea of the iterative Orthogonal Forward Regression (iOFR) algorithm, where the optimal model is obtained by iteration [22]. However, the relationship between the design parameter and the coefficients of the NARX model are predefined as a power-form polynomial function in the above research. For the nonlinear systems, in practice, to predict the coefficients of the NARX model by design parameter values and a predefined function may not reveal the system output accurately. Although the previous studies provide the methods to obtain the CMS, no reports have been found to consider the nonlinear relation between the design parameter values and system output in the process of predicting the system output.

To overcome the issue above, in this paper, an algorithm defined as the Weighted EFOR (WEFOR) is proposed to establish the NARX-M-for-D of a nonlinear system. The weight matrix is introduced to modify the coefficients polynomial functions. The first step of the identification process is similar to the idea of other improved methods for the identification of NARX-M-for-D, such as the IEFOR algorithm [19], the initial NARX-M-for-D is obtained based on the EFOR algorithm. Consequently, the polynomial functions of the coefficients of NARX models with respect to the design parameters are modified by introducing a weight matrix, and then an improved model referred to as the final NARX-M-for-D is obtained. The genetic algorithm (GA) [23] is used for deriving the weight matrix by minimizing the normalized mean square error (NMSE) over the training data sets corresponding to the design parameter values used for modeling and first prediction. The proposed algorithm can establish a reliable model for dynamic analysis and design of the nonlinear system. A key novelty of the WEFOR algorithm is that for the first time, the modeling process considers the uncertain relationship between the coefficients of the NARX model and design parameter.

The organization of the paper is as follows: Section 2 describes the NARX-M-for-D. In Section 3, the traditional identification algorithm known as the EFOR is introduced, and then the WEFOR algorithm is proposed to identify the final NARX-M-for-D. In Section 4, a NARX system is proposed as an illustrative case to show the effectiveness of the new algorithm, and then an experimental case is demonstrated by using a cylinder with a bolted joint. Finally, Section 5 draws the conclusions.

2. Modeling Framework—The NARX-M-for-D

It has been reported that the CMS with the design parameter can be represented by the following polynomial function [19,24]:

$$\begin{array}{ll}y(t)& =F(y_k(t-1),\dots y_k(t-n_y),u_k(t-1),\dots u_k(t-n_u),\mathsf{\theta }(\mathsf{\xi }))\\ & ={\theta }_{k,0}(\xi (k))+{\displaystyle \sum _{i_1}^n{\theta }_{k,i}(\xi (k))}{x}_{k,i_1}(t)+\cdots \\ & +{\displaystyle \sum _{i_1}^n\cdots {\displaystyle \sum _{i_l=i_l-1}^n{\theta }_{k,i_1\cdots i_l}(\xi (k))x_{k,i_1}(t){\displaystyle \prod _{k=1}^l{x}_{k,i_k}(t)}}}+e(t)\end{array}$$

(1)

where $F(\cdot )$ is an unknown nonlinear function.$u_k(\cdot )$ and $y_k(\cdot )$ are system input and output variables of the system under the kth design parameter value, respectively. n_u and n_y are the maximum lags of the system input and output variables, respectively, n = n_u + n_y. l is the nonlinear degree order of function $F(\cdot )$. ${\theta }_{k,i_1\cdots i_l}(\xi (k))$ are the coefficients of NARX model under the kth design parameter value. $\mathsf{\theta }(\mathsf{\xi })$ is the vector formed by the design parameter, $\xi =[{\xi }_1,{\xi }_2,\dots ,{\xi }_K]$. K is the number of design parameter values. e(t) represents the noise variable.$x_{k,m}(t)$ (k = 1,2,…,K, m = 1,2,…,n) represent the delayed system input and output terms, which can be expressed as [19]:

$$x_m=\{\begin{array}{l}u(t-m),1\le m\le n_u\hfill \\ y(t-(m-n_u)),n_u+1\le m\le n_u+n_y\hfill \end{array}$$

(2)

Assume that the input and output variables of a Single-Input and Single-Output (SISO) system under K different design parameter values have been obtained, the nonlinear dynamic parametrical model with K different cases of design parameter properties can be obtained, which can be written as [25]:

$$y_k(t)=F^k(y_k(t-1),\dots y_k(t-n_y),u_k(t-1),\dots u_k(t-n_u),\mathsf{\theta }(\mathsf{\xi }(k)))+e(t),k=1,\dots ,K$$

(3)

where $F^k(y_k(\cdot ),u_k(\cdot ),\theta (\cdot )),k=1,\dots ,K$ are the CMS with K sets of coefficients, which need to be identified from model dictionaries. It is worth mentioning that when K = 1, Models (1) and (3) are traditional NARX models [19].

To illustrate the model explicitly, the NARX-M-for-D can often be expressed in a linear regression function as [17]:

$$y_k(t)={\displaystyle \sum _{m=1}^M{\theta }_{k,m}{p}_{k,m}(t)}+e(t),k=1,\dots ,K$$

(4)

where $p_{k,m}(t)$ are model terms made of delayed system input and output variables; ${\theta }_{k,m}$ represent the kth set of coefficients corresponding to the kth design parameter value; M is the number of candidate model terms in the predictor vectors, which can be derived by:

$$M=\frac{(n+l)!}{n!l!}$$

(5)

Remark 1: Considering the input variables $u(\cdot )$ are impossible to keep the same in practice, the framework of the NARX-M-for-D described in this section denotes the system input under different design parameter values by subscript k, and make the difference with Multi-Input and Multi-Output (MIMO) system. It is worth mentioning that, to emphasize the feasibility of modeling method, the noise is assumed to be white noise and not shown in the following.

3. The Weighted EFOR (WEFOR) Algorithm

Due to the uncertain and nonlinear relation between the design parameter and output of the nonlinear system, to predict the coefficients of the NARX model by design parameter values and a predefined polynomial function cannot reveal the behaviors of a nonlinear system accurately. To overcome this issue, in this section, the WEFOR algorithm is proposed, where the initial NARX-M-for-D is established based on the traditional EFOR algorithm, and then a weight matrix is introduced to modify the predefined polynomial function. The weight matrix is derived using the GA by minimizing the NMSE of all the data sets corresponding to the design parameter values used for modeling and first prediction. And then the improved model, which referred to as the final NARX-M-for-D is obtained by pre-multiplying the right side of the predefined function by the matrix. The details are as follows.

3.1. Traditional EFOR Algorithm

The EFOR algorithm is applied to obtain the CMS of a nonlinear system, determine the functional relationship between design parameter and coefficient of NARX models by using the Least Squares (LS) algorithm, and then the dynamic parametrical model can be obtained, which referred as the initial NARX-M-for-D. The EFOR algorithm is introduced as follows [19,24]:

Step 1: Orthogonalization of the models

Assuming that the NARX models under K parameter values have been obtained by using the FROLS algorithm [2,16], Model (4) can be rewritten as a matrix form:

$${\mathit{y}}_{\mathit{k}}={\mathit{P}}_{\mathit{k}}{\mathsf{\theta }}_{\mathit{k}}$$

(6)

where ${\mathit{y}}_{\mathit{k}}$ is system output under the kth design parameter value; ${\mathit{P}}_{\mathit{k}}$ is regression matrix made of the candidate model terms, ${\mathit{P}}_{\mathit{k}}=[p_{k,1}(t),p_{k,2}(t),\dots p_{k,M}(t)]$; and ${\mathsf{\theta }}_{\mathit{k}}$ is the vector formed by coefficients of the kth NARX model, ${\mathsf{\theta }}_{\mathit{k}}=[{\theta }_{k,1},{\theta }_{k,2},\dots ,{\theta }_{k,M}{]}^{\mathrm{T}}$.

To minimize the predicting error, the regression matrix ${\mathit{P}}_{\mathit{k}}$ needs to be orthogonalized [25]. The orthogonalization can be done by using the Gram–Schmidt algorithm [22], and then Model (6) can be represented as:

$${\mathit{y}}_{\mathit{k}}={\mathit{W}}_{\mathit{k}}{\mathit{G}}_{\mathit{k}}$$

(7)

where ${\mathit{W}}_{\mathit{k}}$ is the orthogonal matrix formed by the orthogonal vectors ${\mathit{w}}_{\mathit{k},1},{\mathit{w}}_{\mathit{k},2},\dots {\mathit{w}}_{\mathit{k},\mathit{M}}$; ${\mathit{G}}_{\mathit{k}}$ is the matrix of coefficients corresponding to ${\mathit{W}}_{\mathit{k}}$, ${\mathit{G}}_{\mathit{k}}=[g_{k,1},g_{k,2},\dots ,g_{k,M}{]}^{\mathrm{T}}$. ${\mathit{w}}_{\mathit{k},\mathit{m}}$ and $g_{k,m}$ can be derived by Equations (8) and (9), respectively.

$${\mathit{w}}_{\mathit{k},\mathit{m}}={\mathit{p}}_{\mathit{k},\mathit{m}}-{\displaystyle \sum _{i=1}^{m-1}\frac{\langle {\mathit{p}}_{\mathit{k},\mathit{m}},{\mathit{w}}_{\mathit{k},\mathit{i}}\rangle }{\langle {\mathit{w}}_{\mathit{k},\mathit{i}},{\mathit{w}}_{\mathit{k},\mathit{i}}\rangle }}{\mathit{w}}_{\mathit{k},\mathit{i}},m=1,2,\dots ,M$$

(8)

$$g_{k,m}=\frac{\langle y_k,w_{k,m}\left(t\right)\rangle }{\langle w_{k,m}\left(t\right),w_{k,m}\left(t\right)\rangle },m=1,2,\dots ,M$$

(9)

where <·,·> represents the inner product.

Step 2: The common model structure detection

The EFOR algorithm selects the common model terms in a stepwise manner based on the significance of all the model terms, which is measured by the index named Average Error Reduction Ratio (AERR). The formula to calculate the AERR can be written as follows [17]:

$${\mathrm{AERR}}_m=\frac{1}{K}{\displaystyle \sum _{k=1}^K\frac{{\left(g_{k,m}\right)}^2\langle w_{k,m}\left(t\right),w_{k,m}\left(t\right)\rangle }{\langle w_{k,m}\left(t\right),w_{k,m}\left(t\right)\rangle }}\times 100\%=\frac{1}{K}{\displaystyle \sum _{k=1}^K\frac{{\langle {\mathit{y}}_k,w_{k,m}\rangle }^2}{\langle y_k,y_k\rangle \langle w_{k,m},w_{k,m}\rangle }}\times 100\%$$

(10)

where the subscript m denotes the mth regressor in the candidate pool of orthogonal model terms.

The ith AERR in the searching process is defined as:

$${\mathrm{AERR}}_i=\max \left\{{\mathrm{AERR}}_m\right\}$$

(11)

Specify a threshold $\sigma$, the searching process is terminated when the Error to Signal Ratio (ESR) is less than $\sigma$. The ESR can be obtained by:

$$\mathrm{ESR}=100-{\displaystyle \sum _{i=1}^{M_0}{\mathrm{AERR}}_i}\le \sigma$$

(12)

where M₀ denotes the M₀th step of the searching process.

Finally, the orthogonalized parametrical dynamical model can be expressed as:

$${\mathit{y}}_{\mathit{k}}={\displaystyle \sum _{m_0=1}^{M_0}{\overline{g}}_{k,m_0}{\overline{\mathit{w}}}_{m_0}},m_0=1,2,\dots ,M_0$$

(13)

Where ${\overline{\mathit{w}}}_{m_0}$ is the selected orthogonalized regressor and ${\overline{g}}_{k,m_0}$ are the coefficients corresponding with ${\overline{\mathit{w}}}_{m_0}$.

Step 3: Unifying the CMS

By using the Inverse-Gram–Schmidt algorithm and based on the Equation (13), the CMS can be represented by:

$${\mathit{y}}_{\mathit{k}}={\displaystyle \sum _{m_0=1}^{M_0}{\overline{\theta }}_{k,m_0}{\overline{\mathit{p}}}_{m_0}},m_0=1,2,\dots ,M_0$$

(14)

where ${\overline{\mathit{p}}}_{m_0}$ is the selected regressor from the full candidate term dictionary and ${\overline{\theta }}_{k,m_0}$ is the associated parameters.

The Inverse-Gram–Schmidt algorithm is introduced in details as follows [16,25]:

$$\{\begin{array}{l}{\overline{\theta }}_{k,M_0}={\overline{g}}_{k,M_0}\hfill \\ {\overline{\theta }}_{k,M_0-1}={\overline{g}}_{k,M_0-1}-a_{M_0-1,M_0}{\overline{\theta }}_{k,M_0}\hfill \\ {\overline{\theta }}_{k,M_0-2}={\overline{g}}_{k,M_0-2}-a_{M_0-2,M_0-1}{\overline{\theta }}_{k,M_0-1}-a_{M_0-2,M_0}{\overline{\theta }}_{k,M_0}\hfill \\ ⋮\hfill \\ {\overline{\theta }}_{k,m}={\overline{g}}_{k,m}-{\displaystyle \sum _{i=m+1}^{M_0}{a}_{m,i}{\overline{\theta }}_{k,i}}\hfill \end{array}$$

(15)

where:

$$a_{m,i}=\frac{\langle {\overline{p}}_{k,i},{\overline{w}}_{k,m}\rangle }{\langle {\overline{w}}_{k,m},{\overline{w}}_{k,m}\rangle },1\le m\le M_0-1$$

(16)

where M₀ is the total number of terms in the CMS (14).

Step 4: Determinate the function of the coefficients of the NARX model by design parameter values.

The relationship between the design parameter $\xi$($\xi =[{\xi }_1,{\xi }_2,\dots ,{\xi }_K]$) and the coefficients of the NARX model can be revealed by a polynomial function, which can be expressed as:

$${\overline{\theta }}_{k,m_0}\left(\xi \right)={\displaystyle \sum _{j_1=0}^J\cdots {\displaystyle \sum _{j_K=0}^J{\beta }_{j_1,\dots ,j_S}{\xi }_1^{j_1}\cdots {\xi }_K^{j_K}}}$$

(17)

where J is the degree of the polynomial function and ${\beta }_{j_1,\dots ,j_S}$ is the corresponding coefficients, which can be estimated using the LS algorithm [26].

The implementation of NARX model identification is done based on the EFOR algorithm and Average Error Reduction Ratio (AERR) criterion. This algorithm determines which potential model terms should be in the model by computing the contribution that each potential model term makes to the system output. This allows the model to be built up one term at a time in a manner that exposes the significance of each new term that is added, and provides a simple but effective way of determining a subset of significant regressors.

It should be noted that the traditional EFOR algorithm builds the relationship between the coefficients of the NARX model and design parameter by a predefined polynomial function. The predicting process starts from deriving the coefficients by design parameter values, and then obtains the system output by the NARX model. In general, however, the relation between the design parameter and the output of a nonlinear system is uncertain and nonlinear. Therefore, the EFOR output may not reveal the system response accurately for a nonlinear system.

To demonstrate this issue explicitly, consider a NARX model expressed as:

$$\begin{array}{ll}y(t)=& {\theta }_1(\phi )u_k{(t-2)}^2+{\theta }_2(\phi )y_k(t-1)+{\theta }_3(\phi )y_k(t-2)\\ & +{\theta }_4(\phi )u_k(t-1)y_k(t-3)+{\theta }_5(\phi )u_k(t-1)\end{array}$$

(18)

where u_k(t)~N(0, 1) is the input signal. y_k(t) is the output signal corresponding to the design parameter. The subscript k denotes the signals corresponding to different design parameter values. $\phi$ is the design parameter. ${\theta }_i(\phi )$ is a third order polynomial function with respect to the design parameter, which is used to derive the coefficients of the NARX model. The third order polynomial function was predefined as:

$$\{\begin{array}{l}{\theta }_1(\phi )=0.0436+0.0083\phi -9.3730\times {10}^{-4}{\phi }^2+0.001{\phi }^3\\ {\theta }_2(\phi )=0.0043+0.0032\phi -2.0697\times {10}^{-4}{\phi }^2+1.8542\times {10}^{-4}{\phi }^3\\ {\theta }_3(\phi )=0.0307-0.0072\phi +0.005{\phi }^2+1.7933\times {10}^{-5}{\phi }^3\\ {\theta }_4(\phi )=0.0461+0.0032\phi +0.0096{\phi }^2+1.42\times {10}^{-4}{\phi }^3\\ {\theta }_5(\phi )=0.0037+0.0049\phi +0.0084{\phi }^2+7.7501\times {10}^{-4}{\phi }^3\end{array}$$

(19)

Note that the coefficients of these polynomial functions are randomly generated. Three data sets of 5121 sample points were generated as the input signals, and then the output signals corresponding to the design parameter φ = [2, 4, 6] are obtained by Equations (18) and (19), which are used for modeling. The data sets are shown in Figure 1.

In this case, the maximum time lags for both the input and output signals were chosen to be 3, respectively. The degree of the nonlinearity was 3. The results of the identification by using the traditional EFOR algorithm are shown in

Table 1. The CMS identified by using the traditional EFOR algorithm.

Step	Term	Coefficients for Different Design Parameter Values			AERR (%)
		φ = 2	φ = 4	φ = 6
1	u(t − 1)	0.05337	0.20775	0.50488	69.05
2	u(t − 2)2	0.06466	0.12820	0.27330	26.56
3	u(t − 1)y(t − 3)	0.09095	0.22194	0.44975	2.98
4	y(t − 2)	0.03640	0.08289	0.17207	1.08
5	y(t − 1)	0.01205	0.02580	0.05619	0.12
Total					99.80

. It can be seen in Table 1, the CMS identified by using the traditional EFOR algorithm has all correct terms, and can be written as:

$$\begin{array}{ll}y(t)=& {\gamma }_1(\phi )u_k{(t-2)}^2+{\gamma }_2(\phi )y_k(t-1)+{\gamma }_3(\phi )y_k(t-2)\\ & +{\gamma }_4(\phi )u_k(t-1)y_k(t-3)+{\gamma }_5(\phi )u_k(t-1)\end{array}$$

(20)

where the coefficient ${\gamma }_m(\phi )(m=1,\dots ,5)$ can be expressed as five third-order polynomial functions. The coefficients are obtained by using the LS algorithm, the functions can be written as:

$$\{\begin{array}{l}{\gamma }_1(\phi )=0.0031+0.0056\phi +0.0082{\phi }^2+8.0556\times {10}^{-4}{\phi }^3\\ {\gamma }_2(\phi )=0.0237+0.0246\phi -0.0045{\phi }^2+0.0012{\phi }^3\\ {\gamma }_3(\phi )=0.0219+0.0248\phi +0.0034{\phi }^2+7.2579\times {10}^{-4}{\phi }^3\\ {\gamma }_4(\phi )=0.0105+0.0114\phi -1.7638\times {10}^{-4}{\phi }^2+4.5937\times {10}^{-4}{\phi }^3\\ {\gamma }_5(\phi )=0.0041+0.0043\phi -6.2907\times {10}^{-4}{\phi }^2+2.2591\times {10}^{-4}{\phi }^3\end{array}$$

(21)

It should be mentioned that the polynomial functions (21) are different with (19) due to the only parts of the data sets are used for modeling. This also shows that the polynomial function obtained by fitting the coefficients of the NARX model and physical parameter values cannot reflect the system characteristics accurately, although the model terms in the CMS are all correct. The EFOR output and simulation result of φ = 10 are compared as shown in Figure 2.

It can be seen in Figure 2, the EFOR output has a significant error for design parameter value not used for the modeling. It also illustrates that the initial NARX-M-for-D cannot reveal the system output accurately by predicting the coefficients through the design parameter value and the predefined functions.

3.2. Evaluation of the Final NARX-M-for-D

In order to facilitate the analysis and design of the nonlinear system by the method of system identification, the WEFOR algorithm is proposed to fix the issue demonstrated in Section 3.1. Firstly, select an arbitrary design parameter value in the interval of interest values to predict the system output, and then compare the results between the predict output and the true value. Secondly, introduce a weight matrix to modify the initial NARX-M-for-D while the error cannot meet the modeling requirement. Finally, derive the weight matrix by using the GA to obtain a more accurate final NARX-M-for-D.

The details of the WEFOR algorithm are summarized as follows:

Step 1: Rewrite Equation (17) into a matrix form as:

$$\overline{\mathsf{\theta }}=\mathsf{\beta }\mathsf{\xi }$$

(22)

where $\overline{\mathsf{\theta }}$ is the coefficient matrix of the NARX models. $\mathsf{\beta }$ is the coefficient matrix associated with the design parameters. $\xi$ is the matrix formed by the design parameter as well as the exponents of the design parameter.

Step 2: According to the initial NARX-M-for-D, select a design parameter value ${\xi }^{\prime }$ which is not used for modeling and then obtain the predicting output by Equations (14) and (17). Introduce the Normalized Mean Square Error (NMSE) [27] to assess the error of the prediction, which can be defined as:

$$\mathrm{NMSE}=\frac{{\sum }_{t=1}^M{[y_p(t)-y(t)]}^2}{{\sum }_{t=1}^M{[y_p(t)]}^2}$$

(23)

where $y_p(t)$ is the prediction output at time instant t. $y(t)$ is the actual output at time instant t. M is the total number of sampling points.

Step 3: Pre-multiplying the right side of Equation (22) by the weight matrix $\rho$, where $\rho$ is a J × J diagonal matrix, which can be written as:

$$\rho =\left[\begin{array}{cccc}{\rho }_{11}& & & \\ & {\rho }_{22}& & \\ & & \ddots & \\ & & & {\rho }_{JJ}\end{array}\right]$$

(24)

To find the optimal global value of the weight matrix $\rho$, in this paper, the GA is applied to minimize the NMSE of all the data sets corresponding to the design parameter $\overline{\xi }$($\overline{\xi }=[{\xi }_1,{\xi }_2,\dots ,{\xi }_K,{\xi }^{\prime }]$), where ${\xi }^{\prime }$ is the design parameter value used for the first prediction. The coefficients matrix of the NARX model can then be calculated by:

$$\overline{\mathsf{\theta }}={\mathit{M}}^{\prime }\overline{\mathsf{\xi }}$$

(25)

where ${\mathsf{M}}^{\prime }$ is the modified coefficients matrix associated with the design parameters, ${\mathsf{M}}^{\prime }=\mathsf{\rho }\mathsf{\beta }$.

Models (14) and (25) are referred to as the final NARX-M-for-D.

Step 4: Select another design parameter value ${\xi }^{″}$ and then obtain the coefficients of the NARX model by Equation (25). Compare the system output from predicting and simulation, as well as the NMSE calculated by the initial NARX-M-for-D and final NARX-M-for-D to assess the predictive ability of the modified model.

The NMSE of ${\xi }^{″}$ can be derived by:

$$\mathrm{NMSE}({\xi }^{″})=\frac{{\sum }_{t=1}^M{[y_p(t,{\xi }^{″})-y(t,{\xi }^{″})]}^2}{{\sum }_{t=1}^M{[y_p(t,{\xi }^{″})]}^2}$$

(26)

where $y_p(t,{\xi }^{″})$ is the prediction output at time instant t. $y(t,{\xi }^{″})$ is the actual output at time instant t.

Step 5: If the comparison results in Step 4 cannot meet the requirements, repeat Steps 2 and 3. By repeating, the weight matrix is modified until the output reaches an acceptable level. In the repeating process, the values of design parameter should be chosen in the interval of interest values to modify the relationship function, and then obtain a reliable model. It should be stressed that the parameter ${\xi }^{\prime }$ and ${\xi }^{″}$ can be selected as any parameter value that close to the parameter values used for initial model identification. The flowchart of Weighted EFOR (WEFOR) algorithm is shown in Figure 3.

The GA is widely used to obtain the optimum values by minimizing the criterion for the complex optimization problems [28]. It is worth mentioning that the GA has also been widely applied in establishing the NARX models. i.e., Ghosh et al. [29] used the GA to obtain the NARX model with predefined input and output lags and then derived the index of insulin sensitivity. Chen et al. [30] proposed a new fitness function and then identified the number of input–output lags and the coefficients of the NARX model by using the GA. Benabdelwahed et al. [28] proposed a new model structure by expanding the NARX model on Laguerre bases, and the GA was applied to get the optimal Laguerre poles. In these works, the GA was used for model structure detection and parameter estimation.

Compared with these studies, the proposed WEFOR algorithm introduced a weight matrix to modify the relationship between the design parameter and the coefficients of the NARX model by using the GA to minimize the NMSE of all the data sets, this can obtain the optimal global function. The problem that the initial NARX-M-for-D cannot predict the output accurately by an any given design parameter value can be solved by the improved model, which referred to as the final NARX-M-for-D. Therefore, the WEFOR-NARX-M-for-D framework can be applied to reveal the dynamic behaviors of a nonlinear system under certain parameter values better. It should be noted that the proposed WEFOR algorithm introduced a weight matrix to modify the initial identified model, and the values of weight matrix are obtained by GA algorithm, which can be extended to a wide class of nonlinear models. Moreover, comparing with a single LS stage by introducing one more design parameter, the proposed method can obtain a more reliable model because a global optimal model can be obtained by using a genetic algorithm. In addition, the prediction results of the model identified by using the traditional method with more parameters are easier to diverge in some cases.

Remark 2: The function (17) can take any form to connect the design parameter and the coefficients of the NARX model [19], in the absence of prior knowledge, the GA can obtain the global optimum coefficients of the function effectively. Furthermore, it does not matter what type of the actual relationship function is by using the WEFOR algorithm for identification of the NARX-M-for-D.

4. Validation of the WEFOR Algorithm

In order to validate the proposed WEFOR algorithm, the NARX model proposed in Section 3.1 is demonstrated as a numerical study case as well as a cylinder with the bolted joint is illustrated as an experimental case. The predicting results are all obtained by using the MPO method [31], and the calculation of the weight matrix is done by using the GA Toolbox of MATLAB. The results show that the final NARX-M-for-D obtained by using the WEFOR algorithm can predict the output of a nonlinear system accurately in a specific range of parameter values.

4.1. The Numerical Illustrative Example

Consider the NARX model expressed in Equation (18) and the results from Figure 2, the output predicted by the initial NARX-M-for-D have significant errors for the design parameter values not used for the modeling. Therefore, the weight matrix is introduced to modify the initial NARX-M-for-D. In order to obtain the matrix, the GA is used to minimize the NMSE of the data sets of φ = [2, 4, 6, 10]. Figure 4 is the fitness function performance trends in the process of the evolution of the NMSE.

From Figure 3, the weight matrix can be obtained with twenty generations, which can be written as:

$$\rho =\left[\begin{array}{lllll}0.9993\hfill & \hfill & \hfill & \hfill & \hfill \\ \hfill & 0.9712\hfill & \hfill & \hfill & \hfill \\ \hfill & \hfill & 0.9001\hfill & \hfill & \hfill \\ \hfill & \hfill & \hfill & 0.9003\hfill & \hfill \\ \hfill & \hfill & \hfill & \hfill & 0.0084\hfill \end{array}\right]$$

(27)

According to Steps 1 and 3 described in Section 3.2, rewrite the polynomial function (21) as a matrix form:

$$\overline{\mathsf{\theta }}={\mathsf{M}}^{\prime }\mathsf{\xi }$$

(28)

The Models (20) and (28) are the final NARX-M-for-D. And then the data sets of φ = 9 predicted by the final NARX-M-for-D are compared with the EFOR output and the simulation output, as shown in Figure 5.

It can be seen in Figure 5, the WEFOR algorithm has much better predictive ability than the EFOR does. In order to quantitative the predictive ability, the NMSE values calculate from the EFOR output and the WEFOR output was presented in

Table 2. Comparison of the NMSE values.

	φ = 9	φ = 10
EFOR	0.08996	0.39294
WEFOR	0.03613	0.02506

.

It can be observed from Figure 2 and Figure 5 and Table 2 that the output predictions of the initial NARX-M-for-D will produce significant errors for the other design parameter values due to the complex relationships between the coefficients of the NARX model and design parameter. It can be seen in Equation (23) that the NMSE value is used to quantify the error between the predicted result and the actual result. Therefore, for the identified model, when the value of NMSE is 0, the obtained model is optimal. However, the error between the predicted result of the identified model and the real result cannot be 0, so the smaller the value is, the more reliable the model is. It should be stressed that the final model was obtained by introducing the corresponding data of another parameter for correction. Therefore, the comparison between EFOR and WEFOR was not made on the same basis of adjustable parameters. The comparison here is to show that the optimized model obtained by WEFOR algorithm has better predictive ability compared with the initial model obtained by EFOR algorithm.

In this numerical illustrative example, the NMSE values provided by the final NARX-M-for-D are less than the initial NARX-M-for-D, it shows that the WEFOR algorithm can obtain the accurate predicting output of the underlying system, and then provide a reliable model for dynamic analysis and design for the nonlinear system. It should be mentioned that the present study is carried out on a workstation (Intel i5-3320M CPU and 16G RAM), the computational time is less than 6 s for the identification of NARX-M-for-D and 2 min for the optimized model in a numerical case study.

In order to illustrate the process of design by the final NARX-M-for-D, the Gaussian process with zero mean and a variance of ${\sigma }^2=1$ is generated as the input signal, as shown in Figure 6a. The predicting outputs corresponding to φ = [2:0.5:6] are obtained by using the MPO method, which is shown in Figure 6b. According to the predicted system output, the corresponding design parameter values can be selected to meet the requirement of design.

4.2. Experimental Verification

A cylinder with a bolted joint structure, which belongs to the strong nonlinear class [32] is taken as a test subject for the experimental validation. The acceleration transducer is located at the top of the cylinder wall, the force transducer is connected to the vibration exciter and opposite to the acceleration transducer. The experimental test subject and the test system are shown in Figure 7.

In this case, the Simcenter Testlab system is applied to obtain the input and output signals under different design parameter values of the bolted joint cylinder for verification of the WEFOR algorithm. Denote the input (force) and output (acceleration) variables using u(t) and y(t), respectively. The pre-tightening torque of the bolts is chosen to be the design parameter, which denotes as $\delta$ in this section. Five data sets of 40,000 sample points corresponding to the design parameter δ = [2, 4, 6, 7, 8] Nm were sampled as the input and the output signals. The signals of δ = [2, 4, 6] Nm are used for modeling, and the signals of δ = [7, 8] Nm are used for the modification and validation. The test data sets are shown in Figure 8.

In the experimental case, the maximum time lags for both the input and output signals were chosen to be n_u = 3 and n_y = 3, respectively. The order of the nonlinearity is selected as l = 3, and the CMS of the bolted joint system identified by using the EFOR algorithm is obtained, the results are shown in

Table 3. The CMS of the bolted joint cylinder.

Step	Term	Coefficients for Different Design Parameter Values			AERR (%)
		δ = 2 Nm	δ = 4 Nm	δ = 6 Nm
1	y(t − 1)	2.4571	2.3883	2.3563	91.98
2	y(t − 2)	−2.1348	−2.0344	−1.9764	7.91
3	y(t − 3)	0.6376	0.5913	0.5602	5.89 × 10−2
4	u(t − 3)	11.5313	10.8823	11.2157	5.67 × 10−3
5	1	0.1413	0.144	0.1627	1.39 × 10−3
6	u(t − 1)	12.2887	12.0533	12.6782	6.26 × 10−4
7	u(t − 2)	−23.6811	−22.839	−23.7511	4.58 × 10−3
8	u(t − 2)2y(t − 3)	−0.0023	−0.0014	−8.36 × 10−4	3.34 × 10−5
9	u(t − 1)y(t − 1)2	2.51 × 10−5	−5.24 × 10−5	−8.58 × 10−5	1.53 × 10−5
10	u(t − 1)2	0.0102	−0.0021	0.0355	7.61 × 10−6
Total					99.96

.

And then the initial NARX-M-for-D can be obtained as:

$$\begin{array}{ll}y(t)& ={\theta }_1(\delta )y(t-1)+{\theta }_2(\delta )y(t-2)+{\theta }_3(\delta )y(t-3)\\ & +{\theta }_4(\delta )u(t-3)+{\theta }_5(\delta )\cdot 1+{\theta }_6(\delta )u(t-1)+{\theta }_7(\delta )u(t-2)\\ & +{\theta }_8(\delta )u{(t-2)}^{2}y(t-3)+{\theta }_9(\delta )u(t-1)y{(t-1)}^2+{\theta }_{10}(\delta )u{(t-1)}^2\end{array}$$

(29)

where ${\theta }_i(\delta )\left(i=1,\dots ,10\right)$ is the coefficient of the initial NARX-M-for-D.

In this case, the coefficients ${\theta }_i(\delta )$ were fitted as third order power-form polynomial functions with respect to the design parameter δ, which can be written as:

$${\theta }_i(\delta )={\beta }_{i,j}+{\beta }_{i,j}\delta +{\beta }_{i,j}{\delta }^2+{\beta }_{i,j}{\delta }^3$$

(30)

The results of ${\beta }_{i,j}$ were obtained by using the LS algorithm, as shown in

Table 4. Results for the parameters ${\beta }_{m,i}$ in (30).

i	j
	0	1	2	3
1	2.5625	−61.89	4.587 × 103	55.045
2	−2.2778	82.125	−5.314 × 103	−63.779
3	0.6993	−34.671	1.912 × 103	22.992
4	13.1629	−1.06 × 103	1.228 × 105	1.47 × 103
5	0.1547	−10.7183	2.009 × 103	24.113
6	13.385	−762.99	1.075 × 105	1.29 × 103
7	−26.277	1.7367 × 103	−2.193 × 105	−2.63 × 103
8	−0.0037	0.8008	−54.306	−0.6517
9	1.47 × 10−4	−0.0717	5.5013	0.066
10	0.0725	−43.593	6.238 × 103	74.852

.

The EFOR output of δ = 10 Nm predicted by the Models (29) and (30) are compared with the simulation results, as shown in Figure 9.

It can be seen in Figure 8, the output predicted by the initial NARX-M-for-D have significant errors. Therefore, the GA is used to minimize the NMSE of the data sets of δ = [2, 4, 6, 8] Nm and obtain the weight matrix. The fitness function performance trends are shown in Figure 10.

From Figure 9, the values of the weight matrix can be obtained with twenty-five generations, and then the weight matrix is written as:

$$\rho =diag([\begin{array}{llllllllll}0.99\hfill & \mathrm{0..98}\hfill & 0.96\hfill & 0.94\hfill & 0.99\hfill & 0.98\hfill & 0.96\hfill & 0.97\hfill & 0.98\hfill & 0.91\hfill \end{array}])$$

(31)

And then the data sets of δ = [7, 8] Nm predicted by the final NARX-M-for-D are compared with the EFOR output and the simulation results, as shown in Figure 11. In order to quantitative the prediction results, the NMSE calculate from the EFOR output, and the WEFOR output was presented in

Table 5. Comparison of the NMSE.

	δ = 7	δ = 8
EFOR	0.0389	0.0975
WEFOR	0.028	0.0724

.

It can be observed from Figure 11 and Table 5 that the output predicted by the initial NARX-M-for-D will produce significant errors for the other design parameter in practice. In this experimental study case, the NMSE of the model identified by the WEFOR and EFOR algorithm of δ = 7 Nm are 0.028 and 0.0389, respectively, the NMSE provided by the WEFOR and EFOR algorithm of δ = 8 Nm are 0.0724 and 0.0975, respectively. It shows that the WEFOR algorithm has a better predictive capacity than the EFOR algorithm, and provide a reliable model to predict the output of a nonlinear system under different working conditions accurately. It can be concluded that the WEFOR can predict the system responses under different design parameter values better than the EFOR algorithm. It should be stated that in the experimental case study, the time used for NARX-M-for-D identification is 11 s, and the optimized model is obtained in 3 min.

Take the system input signal measured at δ = 2 Nm as the input for prediction, which is shown in Figure 12a. And then obtain the system output corresponding to δ = [2:0.2:8] Nm through the final NARX-M-for-D by using the MPO method, as illustrated in Figure 11b. The corresponding design parameter values can be chosen to complete the process of system design according to the results in Figure 12b. The results also show that the WEFOR algorithm can obtain a more accurate model to predict the output of the complex nonlinear system, and provide a reliable model for the analysis and design.

5. Conclusions

A nonlinear NARX model is demonstrated to show that it is often impossible to obtain the system output by using the traditional EFOR algorithm due to the uncertain and nonlinear relationship between the coefficients of the NARX model and design parameter. To solve this issue, an improved algorithm, referred to as the WEFOR is proposed. The identification process of the WEFOR is conducted by five steps, it can be summarized as follows: (i) establish the initial NARX-M-for-D by using the EFOR algorithm, which defined as the initial NARX-M-for-D; (ii) rewrite the polynomial functions into a matrix form; (iii) select a design parameter value to do the first predict and calculate the NMSE; (iv) introduce a weight matrix, and obtain the values of the matrix by using the GA; and (v) repeat, predict, and calculate the weight matrix until the output meets the requirement.

Both the numerical and experimental studies are conducted to validate the efficiency and accuracy of the new proposed WEFOR algorithm. The results indicate that the output predictions by the traditional EFOR algorithm will produce significant errors for the other design parameter values due to the complex relationships between the coefficients of the NARX models and design parameter. The WEFOR can predict the system responses under different design parameter values better than the EFOR algorithm. The results also show that the WEFOR algorithm can obtain a more accurate model to predict the output of the complex nonlinear system, and provide a reliable model for the analysis and design.