Maximum-Likelihood-Based Adaptive and Intelligent Computing for Nonlinear System Identification

Abstract

Most real-time systems are nonlinear in nature, and their optimization is very difficult due to inherit stiffness and complex system representation. The computational intelligent algorithms of evolutionary computing paradigm (ECP) effectively solve various complex, nonlinear optimization problems. The differential evolution algorithm (DEA) is one of the most important approaches in ECP, which outperforms other standard approaches in terms of accuracy and convergence performance. In this study, a novel application of a recently proposed variant of DEA, the so-called, maximum-likelihood-based, adaptive, differential evolution algorithm (ADEA), is investigated for the identification of nonlinear Hammerstein output error (HOE) systems that are widely used to model different nonlinear processes of engineering and applied sciences. The performance of the ADEA is evaluated by taking polynomial- and sigmoidal-type nonlinearities in two case studies of HOE systems. Moreover, the robustness of the proposed scheme is examined for different noise levels. Reliability and consistent accuracy are assessed through multiple independent trials of the scheme. The convergence, accuracy, robustness and reliability of the ADEA are carefully examined for HOE identification in comparison with the standard counterpart of the DEA. The ADEA achieves the fitness values of 1.43 × 10⁻⁸ and 3.46 × 10⁻⁹ for a population size of 80 and 100, respectively, in the HOE system identification problem of case study 1 for a 0.01 nose level, while the respective fitness values in the case of DEA are 1.43 × 10⁻⁶ and 3.46 × 10⁻⁷. The ADEA is more statistically consistent but less complex when compared to the DEA due to the extra operations involved in introducing the adaptiveness during the mutation and crossover. The current study may consider the approach of effective nonlinear system identification as a step further in developing ECP-based computational intelligence.

1. Introduction

1.1. Background and Motivation

System identification or parameter estimation involves the approximation of unknown variables of the system, and this concept provides the foundation for solving different engineering, science and technology problems [1]. Most real-time systems are nonlinear and complex in nature. There are many applications for nonlinear systems in science and engineering, such as the inverted pendulum system [2], motion control of a motor driven robot [3], average dwell-time switching [4], tail-control missile system [5], and weather station systems [6].

Nonlinear systems can be described through different nonlinear models, including the Volterra series [7], Wiener series [8], NARMAX model [9], Wiener model [10,11,12] and Hammerstein model [13], etc. Researchers revealed the strong relations between different nonlinear models and the Volterra series [14]. Sidorov et al. contributed significantly in the theory and applications of Volterra equations by proposing different methods [15,16,17] and exploring the applications in power system operations and energy storage systems [18]. The Volterra series can also represent the Hammerstein model, and Kibangou et al. [19] described Hammerstein model through Volterra series representation and identified the coefficients of the Hammerstein model using the Volterra series. The Hammerstein model has a simpler structure with easier identification than the Volterra series [20]. Therefore, the Hammerstein model is often used to represent a wide class of nonlinear systems [21,22,23,24].

The Hammerstein structure, presented in Figure 1, belongs to a class of input nonlinear systems (INL) where a nonlinear block is cascaded with a linear block. Different local and global search algorithms were proposed for the identification of INL models. Local search algorithms are easy to implement but prone to become stuck in local minima. Local search algorithms include the key term separation technique for the parameter estimation of the Hammerstein-controlled autoregressive system [25]; impulse response, constrained, least-square support vector machine modeling for multiple-input and multiple-output Hammerstein system identification [26]; fractional calculus-based adaptive techniques [27,28]; and the parameter estimation problems of input-nonlinear-output error autoregressive systems, based on the key variable separation technique and the auxiliary model-based identification [29], whereas global search techniques effectively handle the local minima issues. The global search methods based on evolutionary and swarm optimization heuristics are effectively applied for the parameter estimation of different-input nonlinear systems, such as genetic algorithms, which are used for the parameter estimation of nonlinear, Hammerstein-controlled autoregressive systems [30]. Meta-heuristic computing techniques are used for the parameter estimation of Hammerstein-controlled auto-regressive, moving-average systems using differential evolution, genetic algorithms, pattern searches and simulated annealing algorithms [31]. Evolutionary computational heuristics are presented for the parameter estimation problem of nonlinear Hammerstein-controlled, auto-regressive systems through a global search competency of the backtracking search algorithm, differential evolution, and genetic algorithms [32]. The neural networks and fuzzy-logic-based, computational, intelligent approaches are also used to solve complex system identification problems [33,34,35,36,37].

The DEA was also effectively applied to INL systems, and it showed better results than its standard counterparts [38]. Recently, a new variant of the DEA called the maximum-likelihood-based adaptive DEA (ADEA) was proposed [39] for linear systems. The ADEA showed an improved performance compared to the standard DAE in terms of convergence speed and accuracy. The increasing complexity of nonlinear systems requires a continuous search for more accurate and reliable computing algorithms. Thus, the enhanced performance of the DEA and ADEA inspired authors to investigate the behavior of these algorithms for effective INL system identification.

1.2. Objectives and Contribution

In this study, the performance of the DEA and ADEA in terms of correctness, robustness, and convergence, is examined for different nonlinearities, as well as noise levels, in INL systems. The most important contributions of this study are as follows:

• A novel application of the evolutionary computing paradigm through maximum-likelihood-based adaptive, differential, evolution algorithm, ADEA, is explored for efficient optimization in nonlinear system identification.
• The ADEA is developed by introducing the concept of adaptiveness in the mutation and crossover operators of the standard DEA approach.
• The convergence, accuracy and robustness analyses of the ADEA are conducted for different types of nonlinearities and noise levels considered in two case studies of nonlinear systems.
• The reliability of the ADEA is tested in comparison with the standard counterpart of the DEA through executing multiple independent executions of both schemes.
• The ADEA is statistically more consistent than the DEA but less complex due to the extra operations involved in introducing the adaptiveness during the mutation and crossover.

1.3. Paper Outline

The rest of the paper is presented as follows: the INL-based system model of the Hammerstein output error (HOE) structure is given in Section 2. The differential evolution based proposed schemes are presented in Section 3. The simulation results for two case studies of HOE systems are provided in Section 4. The main conclusions and some future research directions are listed in Section 5.

2. Mathematical Model of HOE Systems

Figure 2 shows the block diagram of the HOE model [40].

The input–output relation of HOE system in Figure 2 is represented as:

$$y\left(t\right)=u\left(t\right)+v\left(t\right)$$

(1)

where $y\left(t\right)$ represents the systems’ output, $v\left(t\right)$ denotes the additive noise, and $u\left(t\right)$ denotes noise-free output, defined as:

$$u\left(t\right)=\frac{C\left(z\right)}{D\left(z\right)} \overline{s}\left(t\right)$$

(2)

$\overline{s}\left(t\right)$ shows the nonlinear block’s output and is defined as a nonlinear function of the system input $s\left(t\right)$ with a known basis: ${\gamma }_1, {\gamma }_2,\dots , {\gamma }_m$,

$$\overline{s}\left(t\right)=f\left(s\left(t\right)\right)=e_1{\gamma }_1\left(s\left(t\right)\right)+e_2{\gamma }_2\left(s\left(t\right)\right)+\dots +e_m{\gamma }_m\left(s\left(t\right)\right)$$

(3)

or:

$$\overline{s}\left(t\right)={\displaystyle \sum }_{j=1}^m{e}_j{\gamma }_j\left(s\left(t\right)\right)$$

(4)

Substituting (3) in (2) yields:

$$u\left(t\right)=\frac{C\left(z\right)}{D\left(z\right)}{e}_1{\gamma }_1\left(s\left(t\right)\right)+e_2{\gamma }_2\left(s\left(t\right)\right)+\cdots +e_m{\gamma }_m\left(s\left(t\right)\right)$$

(5)

where $D\left(z\right)$ and $C\left(z\right)$ represents the polynomials with shifting operator as: $z^{-1}$[$z^{-1} y\left(t\right)=y\left(t-1\right)$]

$$\begin{gathered} D\left(z\right)=1+d_1{z}^{-1}+d_2{z}^{-2}+\dots +d_n{z}^{-n} \\ C\left(z\right)=c_1{z}^{-1}+c_2{z}^{-2}+\cdots +c_n{z}^{-n} \end{gathered}$$

(6)

The output of the HOE can be expressed in terms of information and parameter vectors, where the information vector containing the input and output delay terms is denoted by w(t) and the corresponding parameter vector of the HOE is defined as [40]:

$$\mathsf{\theta }={\left[\mathit{d},\mathit{c},\mathit{e}\right]}^{\mathrm{T}}ϵ{ℛ}^{n_0}$$

where $n_0=2n+m$ and the variables in the parameter vector are:

$$\mathit{d}={\left[d_1, d_2,\dots \dots \dots \dots .d_n\right]}^{\mathrm{T}} ϵ {ℛ}^n$$

$$\mathit{c}={\left[c_1, c_2,\dots \dots \dots \dots .c_n\right]}^{\mathrm{T}} ϵ {ℛ}^n$$

$$\mathit{e}={\left[e_1, e_2,\dots \dots \dots \dots .e_m\right]}^{\mathrm{T}} ϵ {ℛ}^m$$

The block diagram of the identification of the nonlinear system modelled through the block-oriented HOE structure shown in Figure 2, by using the proposed evolutionary algorithms, is shown in Figure 3. The objective is to minimize the error z(t) between the desired response and the estimated response by exploiting the proposed evolutionary computing approach, such that $y\left(t\right)$ approaches $\widehat{y}\left(t\right)$.

3. Proposed Methodology

In this section, the proposed methodology based on the DEA and ADEA with maximum-likelihood criteria are presented for the identification of the HOE system given in Section 2.

3.1. Differential Evolution Algorithm (DEA)

The DEA is one of the most broadly exploited algorithms in ECP, developed by Rainer Stron and Kenneth Price in 1995 [41]. This is a population-based algorithm which has the ability to solve global optimum problems. Due to its usefulness and efficiency, this algorithm is applied to various problems, such as the parameter estimation of Hammerstein control autoregressive systems [38], deep belief network [42], effective long short-term memory for electricity price prediction [43], parameter estimation of solar cells [44], effective electricity energy consumption forecasting using an echo state network [45]. In this study, a recently introduced maximum-likelihood-based adaptive DEA is exploited for HOE identification and the maximum-likelihood-based DEA is used for the purpose of comparison [39]. The flowchart describing the main steps of the DEA is presented in Figure 4.

3.2. Adaptive Differential Evolution Algorithm (ADEA)

Different variants of the DEA are proposed through introducing the adaptivity in the process of mutation and crossover. These adaptive DEA variants are effectively exploited to solve many nonlinear problems, such as those involving photovoltaic models and other optimization problems [46,47,48,49,50,51,52,53,54]. The main steps of the adaptive DEA are similar to the simple DEA that starts from population initialization, mutation, crossover, selection and then termination. The only aspect which differs is the adaptiveness factor of both the mutation and crossover processes. Recently, a maximum-likelihood-criterion-based adaptive DEA, i.e., ADEA, was proposed, where the fitness value is calculated by the maximum-likelihood-criterion function [39]. In ADEA, the values of mutation and crossover process change automatically according to the generation (T). The pseudocode of the ADEA is presented in Algorithm 1, whereas the stepwise mechanism involved in the learning of the ADEA is as follows:

Step 1. Initialization:

Set the generation t = 1 and set the initial population ${\mathit{P}}_{i,0}$.

Given the population size Np, the mutation Factor F and the maximum generation T.

Step 2. Data Collection:

Collect the calculated data {${\mathit{w}}_{\mathit{i}}$ (1), ${\mathit{w}}_{\mathit{i}}$ (2), …, ${\mathit{w}}_{\mathit{i}}$ (N)} and {${\mathit{y}}_{\mathit{i}}$ (1), ${\mathit{y}}_{\mathit{i}}$ (2), …, ${\mathit{y}}_{\mathit{i}}$ (N)}.

Step 3. Adaptive Mutation Operation:

Calculate the mutation vector $V_{i.a,t}$ using L = exp(1 − $\frac{T}{T+1-t}$); AMF = F ⋅ $2^L$;

$V_{i.a,t}$ = $p_{i.X1,t-1}$ + AMF ⋅ ($p_{i.X2,t-1}$ − $p_{i.X3,t-1}$);

Step 4. Adaptive Crossover Operation:

Read $V_{i.a,j,t}$ from mutation vector ${\mathit{V}}_{i.a,t}$ = ${\left[V_{i.a,1,t}, V_{i.a,2,t},\dots , V_{i.a,D,t}\right]}^T$; and read $p_{i.a,j,t-1}$ from target vector ${\mathit{p}}_{i.a,t-1}$ = $[p_{i.a,1,t-1}, p_{i.a,2,t-1},\dots , p_{i.a,D,t-1}{]}^T$; to create the crossover vector ${\mathit{U}}_{i.a,t}$.

For t = 1, the adaptive crossover probability Pc will be Pc = $\frac{1+\cos \left(t\right)}{2}$; and for t = 2l the adaptive crossover probability Pc will be Pc = $\frac{1+\cos \left(t-1\right)}{2}$;

Step 5. Selection Procedure:

Compute the maximum-likelihood criterion function of ${\mathit{U}}_{i.a,t}$ and ${\mathit{p}}_{i.a,t-1}$ using the equations:

J(${\mathit{U}}_{i.a,t}$) = $\frac{1}{N}{{\displaystyle \sum }}_{t=1}^N{[{\mathit{y}}_i\left(t\right)-{\mathit{w}}_i^T\left(t\right){\mathit{U}}_{i.a,t}]}^2;$ J(${\mathit{p}}_{i.a,t-1}$) = $\frac{1}{N}{{\displaystyle \sum }}_{t=1}^N{[{\mathit{y}}_i\left(t\right)-{\mathit{w}}_i^T\left(t\right){\mathit{p}}_{i.a,t-1}]}^2;$

• Develop the target vector ${\mathit{p}}_{i.a,t}$.

Step 6. Optimal Target Vector:

• Compute the optimal target vector ${\mathit{p}}_{i.best,t}$.
• Using equations J(${\mathit{p}}_{i.a,t}$) = $\frac{1}{N}{{\displaystyle \sum }}_{t=1}^N{[{\mathit{y}}_i\left(t\right)-{\mathit{w}}_i^T\left(t\right){\mathit{p}}_{i.a,t}]}^2$; ${\mathit{p}}_{i.best,t}$ = $\mathit{a}\mathit{r}\mathit{g} \mathit{m}\mathit{i}{\mathit{n}}_{{\mathit{p}}_{\mathit{i}\mathbf{.}\mathit{a}\mathbf{,}\mathit{t}}}$ J(${\mathit{p}}_{i.a,t}$);

Step 7. Iteration:

• If t > T, then let t := t + 1 and go back to Step 2; otherwise, obtain the optimal target vector ${\mathit{p}}_{i.best,t}$.

Algorithm 1 Pseudo-code of the ADEA

Input: Collect data {${\mathit{w}}_{\mathit{i}}$ (1), ${\mathit{w}}_{\mathit{i}}$ (2), …, ${\mathit{w}}_{\mathit{i}}$ (N)} and {${\mathit{y}}_{\mathit{i}}$ (1), ${\mathit{y}}_{\mathit{i}}$ (2), …, ${\mathit{y}}_{\mathit{i}}$ (N)}. Given the population size Np, the mutation factor F and maximum generation T. Let the generation t = 1. Output: ${\mathit{p}}_{i,best,t}$   (1) for a = 1 : Np do   (2)    for j = 1: D do   (3)      $p_{i.a,j,0}$ = rand(0, 1)   (4)    end   (5)    ${\mathit{p}}_{i.a,0}$ = $[p_{i.a,1,0}, p_{i.a,2,0},\dots , p_{i.a,D,0}{]}^T$   (6) end   (7) ${\mathit{P}}_{i,0}$ = $[{\mathit{p}}_{i.1,0}, {\mathit{p}}_{i.2,0},\dots , {\mathit{p}}_{i.Np,0}{]}^T$   (8) for t = 1 : T do   (9)    for i = 1 : Np do   (10)      X1 = randperm(Np, 1)   (11)      while X1 = p do   (12)        X1 = randperm(Np, 1)   (13)      end   (14)      X2 = randperm(Np, 1)   (15)      while X2 = p or X2 = X1 do   (16)        X2 = randperm(Np, 1)   (17)      end   (18)      X3 = randperm(Np, 1); while X3 = p or X3 = X1 or X3 = X2 do   (19)      X3 = randperm(Np, 1)   (20)    end   (21) L = exp (1 − $\frac{T}{T+1-t}$); AMF = F ⋅ $2^L$; $V_{i.a,t}$ = $p_{i.X1,t-1}$ + AMF ⋅ ($p_{i.X2,t-1}$ − $p_{i.X3,t-1}$)   (22) if t = 1 or t = 2l then   (23)    Pc = $\frac{1+\cos \left(t\right)}{2}$   (24) else   (25)    Pc = $\frac{1+\cos \left(t-1\right)}{2}$   (26) end   (27) ${\mathit{V}}_{i.a,t}$ = ${\left[V_{i.a,1,t}, V_{i.a,2,t},\dots , V_{i.a,D,t}\right]}^T$; ${\mathit{p}}_{i.a,t-1}$ = $[p_{i.a,1,t-1}, p_{i.a,2,t-1},\dots , p_{i.a,D,t-1}{]}^T$   (28) for j = 1 : D do   (29)    if rand(0, 1) ⩽ Pc or j = randperm(D, 1) then   (30)      $U_{i.a,j,t}$ = $V_{i.a,j,t}$   (31)    else   (32)      $U_{i.a,j,t}$ = $p_{i.a,j,t-1}$   (33)    end   (34) end   (35) ${\mathit{U}}_{i.a,t}$ = $[U_{i.a,1,t}, U_{i.a,2,t},\dots , U_{i.a,D,t}{]}^T$   (36) J(${\mathit{U}}_{i.a,t}$) = $\frac{1}{N}{{\displaystyle \sum }}_{t=1}^N{[{\mathit{y}}_i\left(t\right)-{\mathit{w}}_i^T\left(t\right){\mathit{U}}_{i.a,t}]}^2$   (37) J(${\mathit{p}}_{i.a,t-1}$) = $\frac{1}{N}{{\displaystyle \sum }}_{t=1}^N{[{\mathit{y}}_i\left(t\right)-{\mathit{w}}_i^T\left(t\right){\mathit{p}}_{i.a,t-1}]}^2$   (38) if J(${\mathit{U}}_{i.a,t}$) > J(${\mathit{p}}_{i.a,t-1}$) then   (39)    ${\mathit{p}}_{i.a,t}$ = ${\mathit{U}}_{i.a,t}$   (40) else   (41)    ${\mathit{p}}_{i.a,t}$ = ${\mathit{p}}_{i.a,t-1}$   (42) end   (43) J(${\mathit{p}}_{i.a,t}$) = $\frac{1}{N}{{\displaystyle \sum }}_{t=1}^N{[{\mathit{y}}_i\left(t\right)-{\mathit{w}}_i^T\left(t\right){\mathit{p}}_{i.a,t}]}^2$   (44) ${\mathit{p}}_{i.best,t}$ = $\mathit{a}\mathit{r}\mathit{g} \mathit{m}\mathit{i}{\mathit{n}}_{{\mathit{p}}_{i.a,t}}$ J(${\mathit{p}}_{i.a,t}$)   (45) End

4. Simulation and Performance Analyses

This section includes the simulation results of two case studies for HOE system identification using DEA and ADEA. The simulations for both algorithms are performed in MATLAB. The identification of HOE systems is performed by considering different noise levels, as well as various sets of generation size and diverse population size, while the results are presented in a variety of convergence graphs and multiple statistical analyses. The input s(t) for this system is taken as a zero mean and unit variance, while noise v(t) is an additive noise. The performance of algorithms in terms of convergence speed, accuracy, robustness and reliability is evaluated through fitness function formulation. The equation for the fitness function is given below:

$$\mathrm{Fitness}=\mathrm{mean}{\left(\mathbf{y}-\widehat{\mathbf{y}}\right)}^2$$

where y represents the desired response and $\widehat{\mathbf{y}}$ is the estimated response through proposed evolutionary algorithms. The optimal parameter settings for DAE and ADAE technique are presented in

Table 1. Parameter settings of DEA and ADEA.

Sr. No.	Type of Parameter	DEA	ADEA
1	Number of variables	8	8
2	Mutation factor	0.25	Adaptive
3	Crossover probability	0.8	Adaptive
4	Lower bound	−2	−2
5	Upper bound	2	2

.

4.1. Case Study 1

The desired response for case study 1 of the HOE system is obtained through a set of parameters taken in [40]. The performance of the ADEA is assessed by considering two type of nonlinearities and different noise levels in the HOE system. The performances of DEA and ADEA in terms of fitness are initially investigated for the variable size of generations (400, 600, 800) and populations (50, 100, 150). The detailed results with polynomial-type nonlinearity are presented in

Table 2. Comparison of DEA and ADEA with respect to generations and population size for polynomial-type nonlinearity in case study 1.

Generations (T)	Population Size (Np)	DEA Fitness	ADEA Fitness
400	50	1.32 × 10−4	2.33 × 10−6
	100	2.89 × 10−6	7.86 × 10−9
	150	3.41 × 10−10	6.63 × 10−9
600	50	1.09 × 10−4	3.44 × 10−7
	100	2.44 × 10−7	4.21 × 10−11
	150	4.53 × 10−9	3.33 × 10−12
800	50	5.16 × 10−5	6.54 × 10−9
	100	1.01 × 10−6	3.24 × 10−14
	150	2.26 × 10−9	7.08 × 10−15

. It is observed from the fitness values in Table 2 that, for the given generations size (400, 600, 800), the fitness of both algorithms decreases with the increase in population size. Furthermore, both methods achieved minimum fitness values for largest generation sizes. It is observed that the ADEA showed an improved performance compared to the DEA for almost all generations and population sizes. The best fitness achieved by ADEA for 800 generations with 150 population size is 7.08 × 10⁻¹⁵.

The fitness-based learning curves for different generations and population sizes with polynomial-type nonlinearity are shown in Figure 5. Figure 5a–c represent the learning curves for DEA with different generations and population size, whereas Figure 5d–f denote learning curves for ADEA with variations in generations and population size. Figure 5a–c show that the DEA aachieved a fast and accurate convergence for a large number of generations and population sizes, but a slight difference in convergence is observed for DEA until 100 generations with different populations are reached. Likewise, Figure 5d–f show that the ADEA also accomplished minimum fitness values for more generations and populations.

The performance of DAE and ADAE is further examined for three noise variances (0.09, 0.05, 0.01) with fixed population sizes, (50, 80, 100) and a changing size of generations. To analyze the methods in terms of optimal fitness, the results with polynomial-type nonlinearity are provided in

Table 3. Comparison of DEA and ADEA with respect to noise variance and fixed population size for polynomial-type nonlinearity in case study 1.

Noise Variance	Population Size (Np)	DEA Fitness	ADEA Fitness
0.09	50	2.84 × 10−3	5.02 × 10−3
	80	4.94 × 10−3	3.73 × 10−3
	100	3.71 × 10−3	4.21 × 10−3
0.05	50	2.01 × 10−4	1.13 × 10−5
	80	1.30 × 10−5	9.05 × 10−6
	100	1.44 × 10−5	8.68 × 10−6
0.01	50	1.11 × 10−4	3.03 × 10−8
	80	8.22 × 10−6	9.18 × 10−10
	100	2.12 × 10−6	8.69 × 10−10

for three noise levels, as well as three different populations. It is witnessed from the fitness values in Table 3 that both the DEA and ADEA accomplished a significant performance in terms of fitness for small values of noise variances along with a different number of populations. However, both methods did not perform significantly for higher noise variances with different population sizes. The optimal fitness achieved by both DEA and ADEA with a noise variance of 0.01 and population size of 100 is 2.12 × 10⁻⁶ and 8.69 × 10⁻¹⁰, respectively.

The learning curves for the fitness achieved with polynomial-type nonlinearity, three noise variances and three population variations are presented in Figure 6. Learning curves for DEA and ADEA are shown in Figure 6a–c and Figure 6d–f, respectively. Figure 6a–c show that the convergence and steady-state performance of the DEA increases with the reducing population size, noise variance and generations and vice versa. Similar behavior was shown by the ADEA for lower noise levels and smaller population sizes. Moreover, ADEA accomplishes optimal fitness for generations, twice that of the DEA.

The fitness with regard to MSE for both DEA and ADEA is also evaluated by introducing sigmoidal-type nonlinearity to the HOE system. The investigations are made for different populations (50, 100, 150) and generations (400, 600, 800). The comparison of fitness results between DEA and ADEA for the HOE system under consideration with sigmoidal-type nonlinearity are shown in

Table 4. Performance comparison of DEA and ADEA with regard to generations and population size for sigmoidal-type nonlinearity in case study 1.

Generations (T)	Population Size (Np)	DEA Fitness	ADEA Fitness
400	50	7.14 × 10−5	1.77 × 10−4
	100	9.44 × 10−6	1.37 × 10−5
	150	8.89 × 10−7	1.41 × 10−4
600	50	2.08 × 10−5	1.18 × 10−4
	100	3.82 × 10−7	3.40 × 10−8
	150	8.54 × 10−8	8.18 × 10−9
800	50	1.10 × 10−5	1.21 × 10−7
	100	5.17 × 10−7	4.63 × 10−8
	150	6.26 × 10−11	3.24 × 10−9

. It is seen from the MSE results shown in Table 4 that the performances of DEA and ADEA increase with the increase in population size for various generations (400, 600, 800). The optimal fitness of both algorithms is accomplished for the largest values of generations and populations. The relative performance, in terms of minimum value of fitness achieved by both methods with respect to particular generations and populations, is not consistent. The minimum fitness attained by DEA and ADEA with a maximum number of generations and populations is 6.26 × 10⁻¹¹ and 3.24 × 10⁻⁹, respectively.

The learning plots representing fitness for sigmoidal-type nonlinearity with variations of generation and population are shown in Figure 7. The fitness curves for DEA are shown in Figure 7a–c and the learning curves for ADEA are given in Figure 7d–f. A similar trend in performance of DEA and ADEA is noticed from Figure 7a–f regarding convergence speed and final estimated accuracy. Both methods exhibit fast convergence for smaller population and generation size. However, they have achieved optimal fitness for bigger values of population and generation.

The behavior of DEA and ADEA methods in terms of minimal fitness achieved with sigmoidal-type nonlinearity for the HOE system is also assessed by fixing the three population sizes, i.e., [50, 80, 100] against three noise variances [0.09, 0.05, 0.01] and different generations set. The optimal fitness attained for different noise levels and population sizes are presented in

Table 5. Performance comparison of DEA and ADEA with regard to noise variance and population size for sigmoidal-type nonlinearity in case study 1.

Noise Variance	Population Size (Np)	DEA Fitness	ADEA Fitness
0.09	50	5.60 × 10−3	6.79 × 10−3
	80	3.60 × 10−3	5.37 × 10−3
	100	3.70 × 10−3	3.82 × 10−3
0.05	50	5.50 × 10−5	1.91 × 10−5
	80	8.68 × 10−6	1.29 × 10−5
	100	9.04 × 10−6	8.27 × 10−5
0.01	50	1.67 × 10−7	6.63 × 10−7
	80	8.74 × 10−6	1.43 × 10−8
	100	5.49 × 10−7	3.46 × 10−9

. It is observed that both DEA and ADEA performed well for smallest value of noise by obtaining optimal fitness of 1.67 × 10⁻⁷ and 3.46 × 10⁻⁹, respectively.

Figure 8 shows the fitness-based learning curves with sigmoidal-type nonlinearity for various noise levels, population sizes, and different generations. The learning curves for DEA, shown in Figure 8a–c, demonstrate that DEA performs effectively in terms of convergence rate for low noise variances with the maximum number of populations. DEA achieves a fast convergence by increasing the generation size up to 200, whereas the graphs in Figure 8d–f show a fast convergence rate for ADEA up to 600 generations with low noise levels, e.g., 0.01, and a small population size, e.g., 50.

4.2. Case Study 2

The desired response for case study 2 of the HOE system is obtained through a set of parameters taken in [55]. The performance of the ADEA is assessed by considering two type of nonlinearities and different noise levels in the HOE system.

In case study 2, the methods DEA and ADEA are assessed for various populations, (50, 100, 150) and generations (400, 600, 800) using two types of nonlinearities: polynomial and sigmoidal. The performance outcomes of DEA and ADEA for polynomial- and sigmoidal-type nonlinearities are shown in

Table 6. Comparison of DEA and ADEA with respect to generations and population size for polynomial-type nonlinearity.

Generations (T)	Population Size (Np)	DEA Fitness	ADEA Fitness
400	50	6.60 × 10−5	5.04 × 10−4
	100	3.39 × 10−5	2.06 × 10−6
	150	2.12 × 10−11	9.71 × 10−7
600	50	8.87 × 10−4	1.54 × 10−3
	100	7.26 × 10−7	9.41 × 10−9
	150	7.01 × 10−16	1.69 × 10−9
800	50	1.65 × 10−4	4.00 × 10−4
	100	9.14 × 10−8	9.87 × 10−12
	150	6.32 × 10−19	6.84 × 10−12

and

Table 7. Comparison of DEA and ADEA with respect to generations and population size for sigmoidal-type nonlinearity in case study 2.

Generations (T)	Population Size (Np)	DEA Fitness	ADEA Fitness
400	50	1.46 × 10−3	2.51 × 10−5
	100	3.18 × 10−5	4.79 × 10−7
	150	1.04 × 10−5	1.90 × 10−7
600	50	2.34 × 10−4	3.30 × 10−5
	100	3.88 × 10−6	3.67 × 10−7
	150	4.66 × 10−10	4.37 × 10−9
800	50	5.55 × 10−4	5.84 × 10−10
	100	2.31 × 10−7	5.03 × 10−10
	150	7.68 × 10−12	3.78 × 10−12

respectively. Table 6 and Table 7 show that the performance of both DEA and ADEA for different numbers of generations increases for both types of nonlinearities with an increase in the population size. Moreover, the best performance of both methods is achieved for a larger generation size. For polynomial-type nonlinearity, the minimum fitness values achieved by DEA and ADEA are 6.32 × 10⁻¹⁹ and 6.84 × 10⁻¹², respectively, whereas the minimum fitness values accomplished by DEA and ADEA for sigmoidal-type nonlinearity are 7.68 × 10⁻¹² and 3.78 × 10⁻¹², respectively.

Figure 9 and Figure 10 show fitness-based learning curves with different populations and generations for polynomial-type and sigmoidal-type nonlinearities, respectively. Figure 9 shows that both DEA and ADEA show fast convergence for a smaller population and generation size, but both methods obtained better steady-state performance for larger population and generation sizes. A similar performance trend was shown by DEA and ADEA in Figure 10 for sigmoidal-type nonlinearity.

To prove the robustness of DEA and ADEA, the performance of both techniques was evaluated for different noise variances (0.09, 0.05, 0.01), variable generation sizes and three population sizes (50, 80, 100). The optimal results achieved by DEA and ADEA with polynomial- and sigmoidal-type nonlinearities for three noise variances and populations are presented in

Table 8. Comparison of DEA and ADEA with respect to noise variance and fixed population size for polynomial-type nonlinearity.

Noise Level	Population Size (Np)	DEA Fitness	ADEA Fitness
0.09	50	0.013	4.23 × 10−3
	80	0.008	2.91 × 10−3
	100	0.001	1.98 × 10−3
0.05	50	4.89 × 10−5	1.05 × 10−5
	80	8.97 × 10−4	7.46 × 10−6
	100	3.57 × 10−5	5.01 × 10−6
0.01	50	1.54 × 10−6	8.66 × 10−8
	80	1.03 × 10−6	8.66 × 10−10
	100	3.40 × 10−6	5.02 × 10−10

and

Table 9. Comparison of DEA and ADEA with respect to noise variance and fixed population size for sigmoidal-type nonlinearities in case study 2.

Noise Variance	Population Size (Np)	DEA Fitness	ADEA Fitness
0.09	50	1.36 × 10−3	4.60 × 10−3
	80	3.90 × 10−3	3.29 × 10−3
	100	2.70 × 10−3	1.59 × 10−3
0.05	50	5.28 × 10−5	1.14 × 10−5
	80	8.80 × 10−6	2.63 × 10−5
	100	4.61 × 10−6	3.97 × 10−6
0.01	50	2.58 × 10−5	1.54 × 10−7
	80	1.09 × 10−9	4.39 × 10−6
	100	4.20 × 10−7	1.39 × 10−7

, respectively. It is seen from the fitness values shown in Table 8 and Table 9 that both DEA and ADEA obtained optimal fitness values for the smallest value of noise level. Furthermore, the performance of both methods in terms of fitness is increased by increasing the population size for different noise levels. The optimum fitness values achieved by DEA and ADEA with polynomial-type nonlinearity and smallest value of noise (0.01) are 1.03 × 10⁻⁶ and 5.02 × 10⁻¹⁰, respectively. However, the minimum fitness values accomplished by DEA and ADEA with a sigmoidal-type nonlinearity are 1.09 × 10⁻⁹ and 1.39 × 10⁻⁷, respectively.

The fitness-based learning curves for polynomial- and sigmoidal-type nonlinearities with three noise variances, three populations and varying generation sizes are shown in Figure 10 and Figure 11, respectively. Figure 10a–c and Figure 11a–c represent the performance-based learning curves of DEA; Figure 10d–f and Figure 11d–f denote the plots for ADEA with different settings. Figure 9 shows that the convergence rate of both DEA and ADEA with polynomial-type nonlinearity increases by increasing the population size and decreasing the noise level, as well as generation size, while both methods accomplished an optimum steady-state performance for the smallest value of noise, a larger population, and larger generation size. A similar performance was demonstrated by both methods for sigmoidal-type nonlinearity, as shown in Figure 12.

4.3. Statistical Study of DEA and ADEA

The statistical investigations of DEA and ADEA for various numbers of runs, with different noise variances, fixed population sizes, and constant generation sizes are shown in Figure 13. It is witnessed from Figure 13a–c that, for all values of noise variances, ADEA is more convergent than DEA, and the optimal fitness achieved by ADEA is much better than that of DEA for all noise variances. It is also noticed that the performance of both DEA and ADEA only slightly degrades by increasing the noise level.

It is observed from the detail results presented for the two case studies that the proposed evolutionary algorithms can be effectively utilized for nonlinear systems identification with polynomial- and sigmoidal-type nonlinearities. The proposed evolutionary algorithms identify the unknown HOE system through optimizing the fitness function that makes the difference between the desired and the estimated response approach to zero. However, the optimal fitness value is not required to correspond to the same set of parameters taken to generate the desired response since, in practical applications, only the desired response is available, rather than the set of parameters.

5. Conclusions

The following are the conclusions drawn from the extensive simulation results presented in the last section:

The evolutionary, computing, paradigm-based DEA and ADEA are effectively used for the nonlinear system identification of Hammerstein output error structures. The DEA and ADEA are accurate and convergent for different nonlinearities, based on polynomial- and sigmoidal-type nonlinearities. The robustness of the DEA and ADEA is established for different levels of external disturbances. However, the accuracy of both algorithms decreases by increasing the noise level. The performance of both DEA and ADEA improves by increasing the population size and generation count, but at the cost of a higher computational budget. The reliable inferences regarding the performance of the DEA and ADEA are drawn through statistical analyses based on 20 independent executions of the algorithms. The convergence speed of the ADEA is slightly slower than the DEA due to the crossover and mutation adaptiveness factor. In comparison, the ADEA is more accurate and statistically consistent compared to the DEA, but at the cost of a little more complexity due to the extra operations involved in introducing the adaptiveness during the mutation and crossover steps. The presented study is a step further in the domain of nonlinear system identification through the use of intelligent computing based on evolutionary algorithms.

In future, the application of the proposed methodology can be investigated for solving nonlinear supply energy systems [56], industrial reactive distillation processes [57], power supply systems [58] and delivery systems [59]. Moreover, the other recently introduced evolutionary algorithms [60] and fuzzy predictive control [61,62,63,64] can be used for efficient nonlinear system identification.