Predictions Based on Evolutionary Algorithms Using Predefined Control Profiles

Abstract

The general motivation of our work is to meet the main time constraint when implementing a control loop: the Controller’s execution time is less than the sampling period. This paper proposes a practical method to diminish the computational complexity of the controllers using predictions based on the Evolutionary Algorithm (EA). It is the case of Model Predictive Control or, more generally, Receding Horizon Control structures. The main drawback of the metaheuristic algorithms (including EAs) working in control structures is their great complexity. Usually, the control variables take values between minimum and maximum technological limits. This work’s main idea is to consider the control variables’ domain inside a predefined control profile’s neighbourhood. The Controller takes into account a smaller domain of the control variables without tracking the predefined control profile or a reference trajectory. The convergence of the EA under consideration is not affected; hence, the same best predictions are found. The predefined control profile is already known or can be determined by solving the optimal control problem without time constraints in open-loop and offline. This work also presents a simulation study applying the proposed technique that involves two benchmark control problems. The results prove that the computational complexity decreases significantly.

1. Introduction

Process engineering often involves controlling a dynamic system subjected to a performance index. If the controlled process has certain mathematical properties, theoretical control laws can be successfully used. This approach produces controllers with moderate computational complexity, which, in addition, can be estimated. A realistic solution must be found when the process has profound nonlinearities or its model is incomplete, imprecise or uncertain. A possible solution is to implement adequate control structures using metaheuristic algorithms, such as the EAs.

Metaheuristic algorithms [1,2,3] are recommended to be largely used in control engineering [4,5,6,7,8] because they are robust and can deal with complex problems. However, there is a price to pay: the computational effort of the Controller could be very large and involve a lengthy Controller’s execution time, which has to be less than the sampling period. For the Controller’s implementation, the fulfilment of this time constraint is sine qua non. Therefore, the diminution of computational complexity constitutes a crucial subject for control applications.

In previous papers, the authors have used the Receding Horizon Control (RHC) [9,10,11,12] to solve Optimal Control Problems (OCPs). RHC is an appropriate control structure due to several properties:

• RHC is a closed-loop control structure (Appendix A) using a Process Model (PM).
• The Controller makes predictions concerning the optimal control sequence over the prediction horizon using the PM [12,13,14].
• The Controller can integrate Metaheuristics, such as the EA, which can be used to compute optimal predictions [12,15,16].

The Controller calls, in real-time, the EA at each sampling period and yields an optimal prediction. The EA’s population comprises individuals (chromozomes), which are candidate predictions, namely, sequences of control variables’ values. Hence, the EA search for optimal prediction inside the control variables’ space. The computational complexity is related to how extended this search space is. Roughly speaking, the larger the search space, the bigger the computational complexity.

When EAs are used as metaheuristics, the predicted sequences can be encoded using a technique that decreases the computational complexity [17]. In paper [18], the authors proposed a method to shrink the control variables’ space using an additional state estimator and a so-called “quality criterion”. The latter is specific to each Process and is expressed through a mathematical constraint characterizing the desired evolution. That is why this method is not always applicable.

In this work, which deals with the same topic as the paper [18], we propose a method to shrink the search space, which can be applied to any Process considered in OCPs. Obviously, before implementing the closed-loop control structure, the control engineer has tested that a certain version of the EA is appropriate for solving the OCP at hand. Hence, an “optimal” or quasi-optimal solution is available, which can give the best-known evolution of the process, optimizing the performance index [19]. The control inputs evolve inside the technological limits given in the OCP’s statement as bound constraints. A “reference” for the future closed-loop control structure can be the control profile (CP), which has generated the open-loop solution. This CP is the sequence of the best control values and is referred to as the predefined CP.

The predefined CP will be perceived and implemented specifically. The Controller’s objective is not to mechanically reproduce this CP, hoping that the optimal state trajectory will be reproduced. At least because of perturbations, this task would not be feasible. The Process and PM still have different behaviour, even if the model has very good accuracy. That is why closed-loop control is mandatory: at sampling moments, the Controller obtains the actual Process state, which is eventually affected by perturbations. The predefined CP is used initially to compute the control ranges and no longer matters in the control phase. The Controller only adapts the current control range at each sampling period.

Because the main objective is to solve OCPs, Section 2 briefly recalls the typical elements of an OCP, allowing the paper to be self-contained and notations to be introduced.

Section 3 describes how the predefined CP can be defined and used by the proposed Controller, such that the control variables’ ranges would be adapted at each sampling period. The structure and implementation of the proposed Controller are also presented. The Controller calls the Predictor, which adapts the control ranges before calling the EA. Each control variable will be constrained to belong to a narrower interval. The control ranges have a crucial influence on the initial population generated by the EA and even the search space. The Predictor’s algorithm is given in Section 3.3.

Our work is validated through a simulation study, including two case studies devoted to two benchmark OCPs. Section 4 sets the simulation objectives and describes the general simulation algorithm for a control action during the control horizon.

The first case study—presented in Section 5—deals with a monovariable process, and its objective function is only a final cost. The second one—illustrated in Section 6—considers a multivariable Process with two control inputs, and its objective function is an integral term added to a final cost.

Both OCPs have been analyzed and implemented in previous works [15,16,18] within the same context: the RHC and similar EAs integrated into the controllers. The present work is still directed to another objective: to propose a new Controller using a predefined CP, which will adapt the control ranges. After implementing the new Controller, we conducted simulation series with the old and new Controllers and made a comparative analysis. The conclusion undoubtedly is that the proposed Controller decreases the computational complexity significantly.

Our paper is not devoted to controlling a specific Process; our presentation emphasizes the purpose of the proposed method is to reduce the computational complexity. This method can be the solution to control many kinds of processes and is subjected to the general motivation of our work, which is time constraint fulfilment: the Controller’s run time must be less than the sampling period.

To help the reader understand exhaustively and use the algorithms presented in this paper, we attached all the written programs as Supplementary Materials and gave all the necessary details in the Appendix A, Appendix B, Appendix C and Appendix D.

2. Optimal Control Problems—Defining Elements

Because the structure of OCPs is well-known, we are content with giving only a few details just for presenting the notations rather than developing the subject. We shall also avoid some mathematical details to simplify the presentation.

2.1. Process Model

As in many works, for the algebraic and ordinary differential equations model, the process is as below:

$$\begin{array}{c}\dot{X}(t)={[f_1(X,U,W)\cdots f_n(X,U,W)]}^T—\mathrm{ordinary} \mathrm{differential} \mathrm{equations}\hfill \\ g_i(X,U,W)=0,i=1,\cdots ,p—\mathrm{algebraic} \mathrm{equations}; W: \mathrm{process}\mathrm{parameters}\mathrm{set}\hfill \end{array}$$

(1)

where the variables have the usual meaning:

$$X(t)={[x_1(t)\cdots x_n(t)]}^T—\mathrm{a} \mathrm{vector} \mathrm{with} n \mathrm{state} \mathrm{variables};$$

$$U(t)={[u_1(t)\cdots u_m(t)]}^T—\mathrm{a} \mathrm{vector} \mathrm{with} m \mathrm{control} \mathrm{variables}.$$

Examples of Process Models are Equation (20) from Section 5.1 and Equation (26) from Section 6.1.

2.2. Constraints

Constraints to guide the process evolution accompany the ordinary differential equations. There are many constraint types in the general systems theory, but we are content with mentioning only those used in the two case studies presented in the sequel.

There is a minimum set of constraints that have to be met:

$$\mathit{Control}\mathit{horizon}: \hspace{1em}\hspace{1em}\hspace{1em}t\in [t_0,t_{final}]; t_0=0,$$

(2)

where $t_{final}$ denotes the final moment of the control horizon.

$$\mathit{Initial}\mathit{state}:\hspace{1em}\hspace{1em} X(0)=X_0\in {\mathbf{R}}^n.$$

(3)

$$\mathit{Bound}\mathit{constraints}: \hspace{1em}\hspace{1em}{u}_j(t)\in {\Omega }_j\triangleq \left[u_{\min }^j, u_{\max }^j\right]; j=1,\cdots ,m; 0<t<t_{final}.$$

(4)

We consider that the values $u_{\min }^j, u_{\max }^j$ are the technological limits of the control variable $u_j(t)$, which are given in the OCP’s statement.

If the control system’s sampling period is T, then the control horizon will have H sampling periods, and it holds:

$$t_{final}=H\times T$$

Solving OCPs using EAs involves discretizing the control horizon and control variables’ values. Therefore, we consider a supplementary constraint: the control variables’ values are constant during each sampling period. So, each control variable is a step function.

2.3. Performance Index

An OCP consists in determining the control profile $U(\cdot )$ that optimizes (maximizes or minimizes) a specific objective function (or cost function) J having the general form Equation (5):

$$J(U(\cdot ),X_0)={\displaystyle {\int }_0^{t_{final}}L(X(\tau ),U(\tau ))d\tau }+J_{final}.$$

(5)

The specific function L characterizes the integral component of the objective function $J$. When the controller makes predictions, the presence of a final cost in the objective function’s expression [17] involves a particularity of the prediction horizon.

Remark 1.

When the objective function has a final cost (Mayer term), no matter if there is or is not an integral term (Lagrange term), the prediction horizon must include the control horizon’s final time, the most difficult situation related to the computational complexity.

That is why, for the moment, we shall consider only the final cost:

$$J\left(U(\cdot ),X_0\right)\triangleq J_{final}=J\left(X\left(t_{final}\right)\right)$$

The OCP’s solution related to an initial state is the function $U(\cdot )$ that optimizes the cost function. The optimal value $J_0$ is called the performance index. For maximization, this value is given hereafter:

$$J_0=\underset{U(\cdot )}{max}J\left(X\left(t_{final}\right)\right)\triangleq \underset{U(\cdot )}{max}J\left(U(\cdot ),X_0\right).$$

We are interested in generating and recording the best CP, which is the one that engenders the open-loop performance index, as we shall see in our approach.

As stated before, this problem has no connection with how its solution is implemented in a closed loop. Therefore we shall refer to it as open-loop OCP, its solution being an open-loop solution, that is, just a CP calculated offline. In fact, an open-loop solution is useless; it is a mathematical solution “on paper” and cannot be used to control the Process because the PM has a different dynamic than the real Process, however small the difference would be.

3. Controller with Predictions Based on EAs and Control Profiles

3.1. Main Idea of the Proposed Controller

As we have already mentioned, the objective of the proposed Controller is to decrease the computational complexity when the predictions are computed using EAs. In other words, the optimal problem solved at each sampling period is to determine the optimal prediction, which has exponential complexity. A stochastic algorithm, the EA, will determine a quasi-optimal prediction with moderate computational complexity. Moreover, this computational complexity can be decreased by adapting the control ranges as in the sequel.

The technological limits cover the control variables’ domain for all sampling periods, but generally speaking, these limits are too large for a specific sampling period. Therefore, these limits can be adjusted for each sampling period. This paper proposes to find out the control ranges around the values belonging to the predefined CP.

The individuals of the EA populations are predictions, that is, sequences of control variables placed inside the technological limits (Equation (4)). The main idea is to adapt the intervals ${\Omega }_i\triangleq [u_{\min }^i, u_{\max }^i]; i=1,\cdots ,m$ for each sampling period such that a large enough neighbourhood of a good CP would become the new control variables’ domain.

Figure 1 illustrates this approach for the control variable $u_i(t),1\le i\le m, t\in [0,t_{final}]$. The blue curve corresponds to the optimal (or quasi-optimal) solution, which is the open-loop solution of the OCP at hand. The open-loop OCP can be solved offline using an appropriate tool, including an EA. What we obtain can constitute a good reference for the Process Evolution, namely the reference solution, state trajectory and performance index. All these data can be recorded into a file and used in our approach. The blue curve corresponds to a good solution, not mandatorily the optimum one. This optimal or quasi-optimal solution could be considered a reference solution for our proposed Controller.

The red curve corresponds to a real solution achieved by the closed-loop control structure. Because our desideratum is to generate quasi-optimal behaviour of the Process, it is supposed that the red solution would be placed in a neighbourhood of the blue curve and would generate a close performance index. The dashed patterns indicate the maximum and minimum control values and delimit this neighbourhood, which is, in fact, a “tube” around the blue curve (reference solution). These patterns replicate the blue curve translated with $\pm \Delta$, which is a parameter defining the new control variable’s range. For example:

$${\Delta }^i=\alpha \left(u_{\max }^i-u_{\min }^i\right);\alpha =20 \%.$$

(6)

In this way, the new range has a smaller extent than the technological range, stretching between the technological limits. That will still be true for all the control variables.

The EA’s implementation led us to consider the following discretization constraint:

$$U\left(t\right)=U\left(kT\right) \mathrm{for} k\cdot T\le t<(k+1)\cdot T; k=0,\cdots ,H-1.$$

(7)

To simplify the notations and equations’ form and because the context does not allow any mathematical confusion, we adopt the following convention in the sequel: the time moment $k\cdot T$ will be simply denoted by $k$. For example, it holds:

$$U(kT)\equiv U(k)\triangleq {\left[u_1(k),\cdots ,u_m(k)\right]}^T$$

The discretization constraint means the control vector $U(k)$ has constant elements and acts inside the sampling period $[kT,(k+1)T)$.

Remark 2.

A control profile (CP) is a sequence of $H$ control vectors $U(0), U(1), \cdots U(H-1)$ having constant values, as in Equation (7).

In the sequel, the reference solution will be a step function called CP (see Remark 2). Therefore, all the curves from Figure 1 will, in fact, be step functions, as in the example presented in Figure 7. This fact does not change the effect of the range adaptation.

The control ranges $R_k, k=0,\cdots ,H-1$ can be determined before the control loop works.

Let us consider the reference CP as below:

$$U^r(k)={\left[u_1^r(k) u_2^r(k)\cdots u_m^r(k)\right]}^T; k=0,\cdots ,H-1;$$

(8)

In the context presented before, the control range for the sampling period k is the following set:

$$R_k=I^1(k)\times I^2(k)\times \cdots \times I^m(k).$$

(9)

The shrunken intervals $I^i(k)$ are defined below.

$$I^i(k)\triangleq \left[u_{\min }^i(k), u_{\max }^i(k)\right]; 1\le i\le m;$$

(10)

$$u_{\min }^i(k)\triangleq u_i^r(k)-{\Delta }^i; u_{\max }^i(k)\triangleq u_i^r(k)+{\Delta }^i;$$

(11)

A control range $R_k$ will limit the searching domain for the control variables $U(k)$, as illustrated in Figure 2 (similar to Figure 3 of the paper [18]).

In this way, the bound constraint Equation (4) is replaced with the relation Equation (12) that adapts the control range to each sampling period.

$$U(k)\in R_k; k=0,\cdots ,H-1$$

(12)

The EA’s convergence toward the optimal prediction can be impeded when some ranges are too narrow, and there is insufficient search space for the control values. In other words, the optimal prediction must be entirely included in the blue zone from Figure 1.

Nota Bene:

The smaller the value ${\Delta }^i$, the smaller the extent of the new control variables’ range. This implication seems favourable for our objective and encourages us to choose a small value ${\Delta }^i$. However, a too-small value ${\Delta }^i$ will cause the EA’s convergence to be lost, that is, the value J to be smaller than the reference value. That is why a practical approach is to choose ${\Delta }^i$ following a few simulations.

3.2. Controller Structure with the Adaptation of Control Variables’ Range

The proposed controller is a specific version of the Receding Horizon Controller (Appendix A). To illustrate the specificity of our work, Figure 3 presents details of the proposed Controller. Let us notice the following aspects of this structure.

• The prediction and optimization module—called Predictor in the sequel—is implemented using an EA, not mandatorily a certain version of the EA. The sole request is to assure good efficiency in solving the optimization problem.
• The prediction’s length depends on the difference between the current ($k,0\le k<H$) and final ($H$) moments. So, a prediction has $H-k$ control output values for the remaining sampling periods.
• The objective function computation module uses the PM and an integration procedure. The Predictor sends a candidate prediction $\overline{U}(k)$ and the current state vector $X(k)$ toward this module, both needed by the numerical integration.
• The Controller adapts the control variables’ range $R_k$ online to the sampling period’s rank ($U(k)\in R_k, k=0,\cdots ,H-1$). These ranges are determined offline using the reference CP, that is, the CP yielding the optimal or quasi-optimal process evolution from an initial state vector.
• The online control ranges’ adaptation will cause the computational complexity to decrease, keeping the EA’s convergence.
• At the moment $k$, the Controller sends the first element $U^{\ast }(k)$ of the predicted sequence toward the Process.
• The grey arrows correspond to data obtained offline and available to the Controller, while the red arrows indicate the data exchanged between the Process and the Controller.

The EA is expected to find the optimal prediction for the current step $k$, using the control variables range $R_k$, more efficiently than when the technological limits are adopted.

Remark 3.

A good tuning of the Controller’s parameters will generate a suitable Process evolution. The closed-loop state trajectory will be in the neighbourhood of the reference state trajectory owing to two factors:

-

The closed-loop CP, which is effectively achieved by the control structure, will be in the neighbourhood of the reference CP;

-

The PM accuracy is good enough to involve the state predictions resembling the process state trajectories.

A simulation study presented in the next sections will prove that the desideratum stated by Remark 3 is reachable, and the Controller will work such that the Process will evolve quasi-optimally.

3.3. Implementation of the Controller

To describe clearly the meaning of predicted sequences, we recall hereafter their structure. In the sampling period $[k,k+1]$, a candidate prediction has the following structure:

$$\overline{U}(k)=\langle U\left(k|k\right)\dots ..U\left(H-1|k\right)\rangle .$$

(13)

The value $U(k+i/k)$ is the prediction $U(k+i)$ using the knowledge up to $k$. Using Equations (1) and (13), the Controller also predicts the state sequence.

$$\overline{X}(k)=\langle X\left(k|k\right)\dots .X\left(H |k\right)\rangle .$$

(14)

The value $X(k+i/k)$ is the prediction for the state $X(k+i)$.

The EA generates candidate predictions covering the prediction horizon and calculates the cost function’s value for each one. At convergence, the optimal prediction is returned to the Controller.

$${\overline{U}}^{\ast }(k)=\langle U^{\ast }\left(k|k\right) \dots . U^{\ast }\left(H-1|k\right)\rangle .$$

(15)

The controller’s output is the predicted sequence’s first element:

$$U^{\ast }(k)=U^{\ast }(k|k).$$

This value is sent toward the control input of the Process.

Figure 4 presents the general activities of the Controller, including the call to the function implementing the Predictor.

Variable $k$ stores the current sampling period’s rank against the beginning of the control action (when k = 0). The function “Predictor_EA” returns the predicted sequence, whose first element will give the current control values ${\overline{U}}^{\ast }(k)$.

Algorithm 1 presents the pseudocode of the function “Predictor_EA”. This algorithm is suitable for any EA implementation due to its general structure. Appendix B details the EA’s special version that we have employed for this paper. Emphasis is placed only on the parameters’ initialization and employment of the control ranges (in lines #7–#9).

Algorithm 1 Pseudocode of the function “Predictor_EA”

Predicted_sequence$\leftarrow \mathrm{Predictor}\_EA(k,X_0)$  /* This function determines ${\overline{U}}^{\ast }(k)$.*/ 1 # Global parameters’ initialization. 2 N$\leftarrow$N0          /*N0: number of individuals*/ 3 NGen$\leftarrow$ Ngen0;        /*NGen0: number of generations */ 4 ngene$\leftarrow$H-k;          /*ngene: number of genes */ 5 for i = 1…N. 6   for j = 1…ngene; 7     pop (i, j) $\leftarrow$random($U_{\min }(k+j), U_{\max }(k+j)$).     /*The control variables’ ranges are $R_k, R_{k+1},\dots ,R_{H-1}$ */ 8    end. 9 end. 10  # Objective function’computation for the initial population 11 g$\leftarrow$1; 12 Ncalls$\leftarrow$0; 13 F $\leftarrow$0; 14 while (g ≤ Ngen) and (F = 0) 15   #Selection of chromozomes; 16   #Crossover; 17   #Mutation; 18   #Replacement; 19   #Update F. 20   g $\leftarrow$ g + 1. 21 end. 22 Predicted_sequence $\leftarrow$pop(1)

The columns of $m\times H$ matrices $U_{\min } \mathrm{and} U_{\max }$ are constructed offline using Equations (11) and (16).

$$U_{\min }(k)={\left[u_{\min }^1(k)\dots u_{\min }^m(k)\right]}^T; U_{\max }(k)={\left[u_{\max }^1(k)\dots u_{\max }^m(k)\right]}^T; 0\le k\le H-1$$

(16)

The matrices $U_{\min } \mathrm{and} U_{\max }$ that join these columns (Equation (17)) could be accessible to this function as global data and used for initial population generation.

$$U_{\min }=\left[U_{\min }(0)\dots U_{\min }(H-1)\right]; U_{\max }=\left[U_{\max }(0)\dots U_{\max }(H-1)\right].$$

(17)

The columns $U_{\min }(k) \mathrm{and} U_{\max }(k)$ indicate the range limits $R_k$ for each of the control variables. On the other side, the individuals of the initial population generated by the EA are control variables sequences that are included in $R_k, R_{k+1},\dots ,R_{H-1}$. Therefore, line #8 yields elements randomly inside these sets for each individual of the population.

Line #13 initializes at 0 the variable “Ncalls” that will count the objective function calls’ number. This variable is updated inside the procedure implementing the objective function calculation and can be considered as a computational complexity measure during the control action.

In our implementation, the code of the EA is included in the “Predictor_EA” program. The reader can see Appendix B, which gives the most important details concerning the EA version employed in this work.

4. Simulation Study

4.1. Study’s Objectives and Preliminaries

In the previous sections, we have proposed the algorithms “Controller_EA” and “Predictor_EA” that can be used in real-time for control structure implementation. Of course, this can be conducted in conjunction with a real process.

The sequel presents a simulation study using these two algorithms devoted to predictions based on predefined control profiles. The study’s main objective is to ascertain that this technique decreases the computational complexity compared to EAs using technological bounds.

More specifically, the simulation study has the following objectives:

• To implement and simulate the RHC and an EA that makes optimal predictions. The authors have already achieved this objective in previous works, but there is a new context related to predefined control profiles requiring the adjustment of old implementations.
• To integrate reference control profiles and adaptation of control variables’ ranges within predictions based on EA. The new ranges are involved in the initial population generation and mutation operator.
• To validate the hypothesis that the new technique can reduce the computational complexity of the Controller through the results’ analysis. The computational complexity is the major impediment to the Controller’s feasibility when using metaheuristics.

The study will be achieved using a simulation program that emulates the closed-loop working. The program’s modules have a realistic implementation and can be used in a real-time closed loop. Only a few adjustments have been performed owing to the simulation.

The Controller integrates the PM and sends the control value to the real Process. The simulation model of the Process will be constructed in two ways:

• The simulation model of the Process and PM are identical. This situation is not trivial because there will be differences between the reference CP and the CP achieved by the closed loop. These differences are due to the closing of the control loop. The reference CP is an open-loop solution.
• The simulation model of the Process is considered equivalent to the PM to which a uniformly distributed perturbation is added. If the vectors $X_k$ and $X_k^{\prime }$ are the state variables of the Process and PM, respectively, it holds:

$$X_k^{\prime }=X_k+\mathrm{perturbation}; 0\le k\le H-1.$$

(18)

A realistic measure of the Controller’s computational complexity is the number of objective function calls during the control horizon. This number is relevant, especially when the objective function’s evaluation entails numerical integrations, which are complex and time-consuming. This measure is adequate when comparing versions of the same application, especially in a statistical context. The simulation program will totalize the number of calls for all sampling periods.

We are not interested in controlling a specific Process; our presentation emphasizes the method, not the Process. The proposed combination of the RHC with an EA can be the solution to control many kinds of processes. Of course, time constraints could reduce this combination’s applicability to processes with relatively large time constants.

We shall consider two case studies covering both monovariable and multivariable Processes and Mayer and Boltza (Mayer plus Lagrange terms)-type performance indexes. Both cases deal with the most difficult case concerning the predictions (Remark 1).

4.2. Algorithm of the Closed-Loop Simulation

The simulation series are carried out by a general program that emulates the closed-loop running. The control horizon is implemented through a sequence of sampling periods $k=0,\dots ,H-1$. For the specific moment k, the main action is calling the Controller. The algorithm of this simulation program is presented in Figure 5.

In the sequel, we add some comments to this flowchart to help understand the simulation algorithm.

“state” is a vector with (H + 1) components that will store the Process’ trajectory.

• At step $k$, the procedure “Controller_EA” is called to calculate the optimal control value $U^{\ast }(k)$ considering the current state $state(k)$. This control value is memorized as a vector “uRHC” element.
• The procedure “RealProcessStep” uses the Process’s simulation model and returns the next state of the Process.
• This state is memorized and considered a new current state for the next sampling period.
• The variable “Ncalls_C” cumulates the numbers Ncalls of all steps and, in this way, evaluates the computational complexity on the entire control horizon.

Let us remark that the Controller prediction ${\overline{X}}^{\ast }(0)$, made at the moment $k=0$, and the sequence of real control values “uRHC” are different things. The Controller prediction is computed using the PM, while each control value $uRCH(k)$ assures the transition between two real Process states:

$$X_0=X(k) \stackrel{uRHC(k)}{\to }{X}_{\mathrm{next}}=X(k+1).$$

(19)

The function “RealProcessStep” integrates the Process’s simulation model to determine the transition Equation (19).

Because the prediction is based on an EA, the simulation program was conducted 40 times to allow statistical analysis. Appendix C gives details concerning our simulation program.

5. Case Study #1—Process with a Single Control Variable

5.1. Optimal Control Problem # 1

The Park–Ramirez problem (PRP) is a benchmark problem [15,20,21] describing an OCP that has been addressed in many papers to study various approaches, from integration methods to metaheuristic employment. The nonlinear process models a fed-batch reactor, which produces secreted protein. Besides many works that have communicated quasi-optimal evolutions, the authors have also presented solutions of the PRP in different research contexts. In paper [15], the authors have proposed an RHC solution integrating an EA. The main objective was to evaluate how much the optimal evolution degrades when the PM and Process differ. The present work is the continuation of a previous paper’s topic [18], which was to decrease the computational complexity of the Controller due to a short-time Estimator. This module defines shrunken intervals for the control variables that cause the search process to be more efficient. The Estimator can or cannot be used according to a “quality criterion” that does or does not exist. This work defines shrunken intervals, starting from a control profile that always exists. Because the closed-loop solution of PRP is already constructed and a version of the EA is tested, we shall study now the possibility of decreasing the computational complexity.

We present hereafter the three parts defining the PRP as an OCP.

Process Model

$$\begin{gathered} {\dot{x}}_1=g_1\cdot \left(x_2-x_1\right)-\frac{u}{x_5}\cdot x_1. \\ {\dot{x}}_2=g_2\cdot x_3-\frac{u}{x_5}\cdot x_2. \\ {\dot{x}}_3=g_3\cdot x_3-\frac{u}{x_5}\cdot x_3 \\ {\dot{x}}_4=-7.3\cdot g_3\cdot x_3+\frac{u}{x_5}\cdot \left(20-x_4\right) \\ {\dot{x}}_5=u \\ {g}_1=\frac{4.75\cdot g_3}{0.12+g_3} \\ {g}_2=\frac{x_4}{0.1+x_4}\cdot e^{-5.0\cdot x_4} \\ {g}_3=\frac{21.87\cdot x_4}{(x_4+0.4)(x_4+62.5)} \end{gathered}$$

(20)

The state vector regroups the following physical parameters:

• $x_1(t)$—concentration of secreted protein.
• $x_2(t)$—concentration of total protein.
• $x_3(t)$—density of culture cell.
• $x_4(t)$—concentration of substrate.
• $x_5(t)$—holdup volume.

Constraints

The control horizon, initial state and bound constraints are given below.

$$\hspace{1em}t\in [t_0,t_{final}]; t_0=0, t_{final}=15h$$

(21)

$$\hspace{1em}X(0)=X_0={\left[0, 0, 1, 5, 1\right]}^T\in R^5.$$

(22)

$$u(t)\in \Omega \triangleq \left[0, 2\right]; 0<t<t_{final}.$$

(23)

Performance Index

$$J_0=\underset{u(t)}{\max }J\left(x\left(t_{final}\right)\right)=\underset{u(t)}{\max }{X}_1(t_{final})\cdot X_5(t_{final})$$

(24)

5.2. The Reference Control Profile

The EA presented in Appendix B has also been used to solve open-loop PRP with few modifications (see Appendix B). An open-loop solution is produced and considered the optimal or quasi-optimal solution of the PRP. The control profile, the best trajectory and the performance index have been saved in a data file (for example, the workspace file “WSBestEvolution.mat”).

Figure 6 shows the best CP and best trajectory; this data will help us improve the closed-loop solution’s computational complexity. For the Controller, the open-loop CP will be the reference.

We remark that the performance index of the so-called “best” solution is $J_{opt}=31.930$, but we know from previous works that there are solutions better than this one, having values of J superior to 32. This fact does not matter because we may choose as reference a quite good open-loop solution.

Equations (7)–(11) define the control ranges for the Controller. For this case, it holds:

$$m=1;{ u}_{\max }=2; u_{\min }=0; {\Delta }^1=0.4;$$

In Figure 7, the values $u_{\min }^1(k)$ and $u_{\max }^1(k)$ are depicted for all values of k as step functions. For a specific k, the control range stretches between the two values and is a blue rectangle. According to the k value, the unique shrunken interval ranges from the green line to the black line:

$$I^1(k)=[u_{\min }^1(k), u_{\max }^1(k)]$$

The blue and red step functions are placed between these marks along the control horizon. The dashed lines correspond to technological limits. Let us note that the control ranges cannot intersect with the technological limits.

5.3. Simulation Results

5.3.1. Simulation without Range Adaptation

The first simulation series concerns the implementation of the closed loop using the Controller_EA that does not adapt the control range and, consequently, considers only the technological limits. The data in

Table 1. Simulation series for control loop without range adaptation.

Run #	J0	Ncalls	Run #	J0	Ncalls
1	31.867	103,893	21	31.869	115,447
2	31.960	101,867	22	31.940	106,851
3	31.939	97,526	23	31.904	114,410
4	31.962	105,024	24	31.887	91,767
5	31.915	78,322	25	31.994	107,582
6	31.894	94,649	26	31.804	109,752
7	31.873	104,159	27	31.813	97,892
8	31.961	87,972	28	31.971	95,134
9	32.054	99,993	29	31.852	93,878
10	31.885	121,889	30	31.863	92,734
11	31.868	102,303	31	31.971	99,226
12	31.863	112,949	32	31.823	110,145
13	31.866	97,014	33	31.908	121,515
14	32.115	112,166	34	31.844	111,978
15	31.845	115,368	35	31.964	90,920
16	31.889	116,501	36	31.912	106,542
17	31.896	102,256	37	31.913	101,207
18	31.872	109,789	38	31.986	89,939
19	31.838	102,723	39	31.945	97,510
20	31.884	118,861	40	31.927	111,641

are generated following the procedure indicated in Appendix C.1.

Each line displays the performance index (J₀) and the cumulated number of objective function calls (denoted here as Ncalls). The latter counts all the calls during the control horizon.

When a particular simulation produces the closest value to the average performance index, we consider it the typical execution. We determined the typical run as the 33rd: Jtypical = 31.9082.

Table 2. Statistics on the performance index.

Jmin	Javg	Jmax	Sdev	Jtypical
31.04	31.908	32.114	0.064	31.9082

presents a statistic for the performance index (J₀), including the minimum, maximum and average values and the standard deviation (Sdev).

The typical evolution of the closed loop is described in Figure 8. The controller with predictions based on the EA generates the control values in Figure 8a. The Process’s state vector evolves as in Figure 8b.

5.3.2. Simulation with Range Adaptation (Predefined Control Profile)

The second simulation series concerns the implementation of the closed loop using the Controller_EA with control range adaptation. In addition, the Process’s simulation model is also the PM (the model inside the Controller). The data in

Table 3. Simulation of the control loop with range adaptation.

Run #	J	Ncalls	Run #	J	Ncalls
1	31.836	43,972	21	31.937	42,637
2	31.837	30,398	22	32.030	31,717
3	32.043	29,163	23	32.045	35,026
4	31.877	29,030	24	31.935	36,893
5	31.916	49,406	25	31.870	49,992
6	31.958	28,613	26	31.862	35,178
7	31.989	37,383	27	31.978	39,042
8	31.819	41,684	28	31.910	39,695
9	31.955	40,298	29	31.802	43,910
10	32.128	30,774	30	32.030	33,208
11	31.920	36,427	31	31.879	43,079
12	31.863	31,645	32	31.899	36,875
13	32.197	28,777	33	31.867	27,142
14	31.880	41,636	34	31.972	34,325
15	31.873	35,715	35	31.842	31,758
16	31.914	35,094	36	31.841	47,849
17	31.878	42,261	37	32.071	30,180
18	31.853	42,556	38	31.962	31,795
19	31.878	46,671	39	31.923	53,447
20	32.058	31,562	40	31.909	57,182

are generated following the procedure indicated in Appendix C.2.

This time, the typical run is the 24th and has Jtypical = 31.9354.

Table 4. Statistics on the performance index.

Jmin	Javg	Jmax	Sdev	Jtypical
31.802	31.930	32.196	0.088	31.935

presents a statistic of this simulation series.

The typical evolution of the closed loop is described in Figure 9. With predictions based on the EA and a predefined CP, the controller generates the control values in Figure 9a. The control profile stays inside the blue zones representing the control ranges. The Process’s state vector evolves as in Figure 9b.

5.3.3. Simulation with Range Adaptation and Perturbated Process

To prove that the closed loop is working well in a real context, we have to simulate the situation when the Process differs from the PM. We carried out a few closed-loop simulations using the control range adaptation and perturbated Process.

We shall apply Equation (18) from Section 4.1, namely, the Process state will be affected by an additive perturbation. Practically, a perturbation $\Delta P_i$—equivalent to all influences—is added to the PM’s vector state, as below:

$$x_i(k)\leftarrow x_i(k)+\Delta P_i, i=1,\dots ,5$$

(25)

We have considered the perturbation $\Delta P_i$ as a random variable with uniform distribution in the interval [−L,L], where:

$$L=p\cdot \left|x_i(k)\right|, 0<p<1$$

Hence, $\Delta P_i$ depends on the absolute value of the considered state variable to avoid annulling its influence. We have used a constant value, p = 5%, in our simulation, which means placing perturbations in a 10% interval.

The data in

Table 5. Results of the simulation with control range adaptation and perturbated state vector.

k	uRHC(k)	J0	Ncalls
0	0.396	31.94	8460
1	0.135	32.299	798
2	0.116	30.820	51,740
3	0.452	28.906	47,940
4	0.318	29.635	43,747
5	0.913	30.379	39,620
6	1.057	30.197	35,991
7	1.133	30.578	31,816
8	1.738	31.900	1610
9	1.972	32.147	354
10	0.107	32.578	354
11	0.870	32.252	232
12	0.933	33.062	171
13	0.914	34.687	110
14	1.313	34.698	59

are generated following the procedure indicated in Appendix C.3 and refer to a single simulation.

The evolution of the closed loop is described in Figure 10. With predictions based on the EA and the predefined CP, the controller generates the control values from Figure 10a. The Process’s state vector evolves as in Figure 10b, where we remark small discontinuities of the trajectories. These small jumps are caused by the perturbations applied to the state variables at the moment k, as in Equation (25).

We denote, by A(X₀, r), r = 1, …, H, the accessibility set in r step, the set of all state vectors reached from the initial state X₀ in r steps, respecting the MP’s dynamic and discretization constraint. For example, the maximum value of J(X) when X$\in$A(X₀, H) is the performance index, which is around the value of 32.

Due to the perturbation (Equation (25), the simulation does not respect the MP’s dynamic that was actually our desideratum ($P\ne MP$). The simulated Process now has stochastic behaviour and discontinuous trajectories. Consequently, it is possible for the final state to not belong to A(X₀, H). The trajectory leads to new final states having possible greater values (for example, J₀ = 34.69). The EA now has to search for optimal predictions in a new state subspace, which may be a harder problem involving a very large value of Ncalls. In any case, this is a new task, different from the previous ones, which makes the comparison irrelevant. That is why we did not conduct a third simulation series designated to be compared with the previous two series. Among the simulations with perturbations that we carried out, there are also simulations in which all states belong to A(X₀), and the performance index has values accordingly.

5.4. Discussion of Case #1

Figure 7, Figure 9 and Figure 10 prove the quasi-optimal behaviour of the Process in the three cases. A statistic of the simulation series (without and with range adaptation) is presented in

Table 6. Comparison among simulation series.

	J0		Ncalls
	Typical Value	Sdev	Min	Avg	Max	Sdev
without control range	31.908	0.064	78,322	103,782	121,889	9912
with control range	31.935	0.088	27,142	37,850	57,182	7401
with control range and perturbation	34.69		a single simulation

. First of all, let us remark that the range adaptation does not worsen the closed-loop behaviour:

• The aspects of process evolution are very alike.
• The Controller shows convergences in all simulations.
• The optimal behaviour is not degraded. The J₀ and standard deviation values are very similar because the shrunken ranges do not impede the convergence process. Practically the solutions converge toward the same optimal region of the searching space.

On the other side, the Controller with range adaptation greatly influences the computational complexity. Columns referring to Ncalls of Table 6 show this improvement.

The decreasing ratio of the average number of calls is calculated below.

$$\mathrm{decreasing} \mathrm{ratio}=\frac{103,782-37,850}{103,782}=0.636$$

Due to the control range adaptation, the computational complexity decreased by 63.6%, which is a spectacular reduction.

The simulations with control range adaptation and perturbed Process are not comparable to those from the previous series because the research state-space could differ. The predefined control profile is still not appropriate for the new accessibility set. This situation is tributary to how the Process is constructed using the PM and a uniformly distributed perturbation. Nevertheless, the controller works well even in this situation, and the EA converges to a very good performance index but uses non-accessible states from X₀.

6. Case Study #2: Multivariable Process and Complete Objective Function

6.1. OCP’s Statement

In this section, we address another case study that will complete the first one, owing to the following aspects:

-

The Process is multivariable; it has two control variables.

-

Besides the final cost, the objective function also has an integral component (Remark 2).

Presented in many papers [6,14], we consider an OCP that the authors have already solved using the EA and RHC in a previous paper [14]. In the present work, we studied through simulations only whether a predefined control profile could decrease the computational complexity and to what extent.

The OCP treated in the sequel will be referred to as Protein Production Problem (PPP). The following state equations describe its nonlinear PM:

$$\begin{gathered} {\dot{x}}_1=u_1\left(t\right)+u_2\left(t\right) \\ {\dot{x}}_2=g_1\cdot x_2-\left(u_1\left(t\right)+u_2\left(t\right)\right)\cdot \left(\frac{x_2}{x_1}\right) \\ {\dot{x}}_3=\frac{100\cdot u_1}{ x_1}-\left(u_1\left(t\right)+u_2\left(t\right)\right)\left(\frac{x_3}{x_1}\right)-g_1\cdot \frac{x_2}{0.51} \\ {\dot{x}}_4=g_2\cdot x_2-\left(u_1\left(t\right)+u_2\left(t\right)\right)\cdot \left(\frac{x_4}{x_1}\right) \\ {\dot{x}}_5=\left(\frac{4\cdot u_2\left(t\right)}{x_1}\right)-\left(u_1\left(t\right)+u_2\left(t\right)\right)\cdot \left(\frac{x_5}{x_1}\right) \\ {\dot{x}}_6=\left(-\frac{0.09\cdot x_5}{0.034+x_5}\right)\cdot x_6 \\ {\dot{x}}_7=\left( \frac{0.09\cdot x_5}{0.034+x_5}\right)\cdot \left(1-x_7\right) \\ {g}_1=\frac{x_3}{14.35+x_3\cdot \left(1+x_3/111.5\right)} \cdot \left(x_6+\frac{0.22\cdot x_7}{0.22+x_5}\right)\cdot \\ {g}_2=\left(\frac{0.233\cdot x_3}{14.35+x_3\cdot \left(1+x_3/111.5\right)}\right)\cdot \left(\frac{0.005 +x_5}{0.022 +x_5}\right) \end{gathered}$$

(26)

We have a system with n = 7 state variables and m = 2 control variables. All constraints presented in Section 2.2 are available for this OCP, including the discretization constraint.

$$\begin{array}{c}t\in \left[t_0,t_f\right]; t_0=0; t_f=H=10\mathrm{hours}\hfill \\ X(0)=X_0={\left[1, 0.1, 40, 0, 0, 1, 0\right]}^T.\hfill \\ 0\le u_1\left(t\right), u_2\left(t\right)\le 1 \mathrm{i}.\mathrm{e}., u_j(t)\in {\Omega }_j\triangleq \left[0,1\right]; j=1,2.\hfill \end{array}$$

The objective function and the performance index are given in Equation (27):

$$\begin{gathered} J\left(u_1(t),u_2(t),X_0\right)=-Q\cdot {{\displaystyle \int }}_{t_0}^{t_f}{u}_2\left(t\right)\cdot dt+x_1\left(t_f\right)\cdot x_4\left(t_f\right);Q=1.5. \\ {J}_0=\underset{u_1\left(t\right), u_2\left(t\right)}{max}J \end{gathered}$$

(27)

As we mentioned before, we have solved and obtained near-optimal solutions for the PPP in the previous work. One of these solutions has been considered the best Process evolution relating to the initial state, described by three components: the predefined control profile, trajectory and performance index. These components were stored in the file “WSBestEvolution” using the procedure given in Appendix D.

Two simulation series, each with 40 runnings, were used to compare the controllers without and with control range adaptation.

6.2. Results and Discussion of Case #2

The first simulation series used the Controller constructed in our previous work, which the reader can understand and use following the details given in Appendix D.1. When the EA initializes its individuals, the two control variables take values inside the domains determined by the technological limits:

$${\Omega }_1=[0,1]; {\Omega }_2=[0,1]$$

The performance index J₀ and the number Ncalls are given in

Table 7. Results of the Controller without predefined CP.

Run #	J0	Ncalls	Run #	J0	Ncalls
1	5.624	15,420	21	5.648	14,100
2	5.628	14,460	22	5.772	14,940
3	5.761	16,020	23	5.684	14,040
4	5.640	36,000	24	5.708	16,200
5	5.637	17,580	25	5.638	23,580
6	5.720	15,540	26	5.670	14,700
7	5.760	14,040	27	5.697	17,340
8	5.719	12,480	28	5.684	16,260
9	5.707	14,280	29	5.712	12,780
10	5.672	14,100	30	5.459	36,600
11	5.707	11,880	31	5.675	12,540
12	5.721	13,800	32	5.696	14,580
13	5.643	14,520	33	5.730	13,380
14	5.622	13,380	34	5.589	12,840
15	5.618	17,580	35	5.758	13,680
16	5.641	17,880	36	5.677	18,960
17	5.781	9120	37	5.732	13,800
18	5.689	13,020	38	5.670	10,200
19	5.705	12,240	39	5.6845	16,260
20	5.719	11,640	40	5.6590	14,040

for all the executions.

Execution #23 is the typical one whose evolution is depicted in Figure 11. The Controller calls 14,040 times the objective function during the control horizon and guides the Process toward its final state $x_1(H)=2.359; x_4(H)=2.484$. The value J_opt = 5.685 is also calculated considering the integral component.

A statistical analysis of J₀ and Ncalls is presented in

Table 8. Statistics regarding the performance index and number of calls.

	Min	Avg	Max	Sdev	Typical
J0	5.459	5.682	5.781	0.06	5.685
Ncalls	9120	15645	36600	5409.3

, where the columns’ names have the usual meaning. “Sdev” denotes the standard deviation of J₀.

Let us note that the performance index recorded inside the file giving the best evolution (see Appendix D) is J_opt = 5.752. At the same time, the maximum value achieved by the control loop without range adaptation is J₀ = 5.781. This fact confirms the conclusion drawn in previous works that the RHC does not degrade the loop behaviour significantly.

The second simulation series uses a new version of the Controller, whose EA is modified to adapt the control variables’ ranges (as in Algorithm 1). Instead of Figure 8 illustrating the single-interval ranges for the PRP,

Table 9. Range adaptation of the control variables.

k	${\mathit{R}}_{\mathit{k}}={\mathit{I}}^1(\mathit{k})\times {\mathit{I}}^2(\mathit{k})$
0	$[0,0.2]\times [0,0.2]$
1	$[0,0.2]\times [0,0.2]$
2	$[0,0.292]\times [0,0.2]$
3	$[0,0.246]\times [0,0.2]$
4	$[0,0.346]\times [0,0.2]$
5	$[0,0.246]\times [0,0.3]$
6	$[0,0.211]\times [0,0.2]$
7	$[0,0.3]\times [0,0.2]$
8	$[0,0.2]\times [0,0.2]$
9	$[0,0.261]\times [0,0.2]$

presents the two-interval ranges for the PPP.

Details about how this simulation series is carried out are in Appendix D.2.

Table 10. Simulations of the Controller with predefined CP.

Run #	J0	Ncalls	Run #	J0	Ncalls
1	5.773	8640	21	5.630	18,900
2	5.793	8100	22	5.752	10,440
3	5.786	8820	23	5.783	6720
4	5.791	10,560	24	5.754	9720
5	5.818	9120	25	5.746	8580
6	5.694	10,080	26	5.659	6000
7	5.735	8520	27	5.706	10,140
8	5.611	9660	28	5.735	7980
9	5.761	7560	29	5.708	9660
10	5.709	8820	30	5.684	12,240
11	5.862	9660	31	5.812	6960
12	5.716	10,020	32	5.743	12,300
13	5.670	10,440	33	5.751	5460
14	5.736	8040	34	5.785	7500
15	5.816	6180	35	5.926	7380
16	5.824	6000	36	5.826	7800
17	5.728	10,200	37	5.691	10,620
18	5.637	9540	38	5.818	6000
19	5.905	7860	39	5.917	9360
20	5.810	9840	40	5.692	9120

gives the values J₀ and Ncalls for all the runnings.

Execution #9 is the typical one whose evolution is depicted in Figure 12 The Controller calls 7560 times the objective function during the control horizon and guides the Process toward its final state $x_1(H)=1.922; x_4(H)=3.131$. The value J_opt = 5.761 is calculated by adding the integral component to the final cost.

Statistical analysis for J₀ and Ncalls is presented in

Table 11. Statistics regarding the performance index and objective function’s calls number.

	Min	Avg	Max	Sdev	Typical
J0	5.611	5.758	5.927	0.07	5.761
Ncalls	5460	9013	18,900	2300.7	-

. Compared with Table 8, in Table 11, the average value of J₀ equals the reference performance index (J_opt = 5.752), but there are many other better results.

Figure 11 and Figure 12 show that the two Controllers cause the two closed loops to have similar behaviours. As in case #1, the J₀ and standard deviation values are very similar because the shrunken ranges are wide enough to assure convergence. Because the value of Ncalls decreases globally, its standard deviation will decrease as well. The range adaptation has a great beneficial influence on computational complexity. The columns referring to Ncalls from

Table 12. Comparison between simulation series’ results.

	J0		Ncalls
	Typical Value	Sdev	Min	Avg	Max	Sdev
without control range	5.685	0.06	9120	15,645	36,600	5409.3
with control range	5.761	0.07	5460	9013	18,900	2300.7

prove this improvement.

The decreasing ratio of the average number of calls is calculated below.

$$\mathrm{decreasing} \mathrm{ratio}=\frac{15,645-9013}{15,645}=0.43$$

Due to the control range adaptation, the computational complexity decreased by 43%, which is a very important reduction.

7. Conclusions

In this paper, we presented the continuation of our previous work concerning the implementation of the OCPs using predictions based on EAs. The emphasis was placed on how to decrease the computational complexity of prediction generation. We proposed the so-called predefined control profile, which means the control range adaptation along the control horizon. The following conclusions can be drawn considering the objectives stated in Section 4.1.

• In the new context related to predefined control profiles, we reconsidered the implementation and simulation of closed loops based on RHC and EAs to make optimal predictions. The necessary adjustments have been made.
• The control profiles were determined and integrated into the EA, and the control range adaptation was involved in the initial population generation at each sampling period.
• The computational complexity is a bar for using metaheuristics to optimize inside control structures. The simulation analysis of the two case studies proved that the proposed technique is effective, and the computational complexity decrease is large (63% and, respectively, 43%).

The Controller converged in all simulations and gave good control loop results for the performance index, especially the number of calls. The latter’s decrease was reflected, obviously, in the reduction of the Controller’s execution time at each sampling period.

In the first case study, we also simulated a Process different from the PM, using a perturbated PM. Although the simulations carried out using this technique cannot be compared with the preceding ones, they proved that the Controller is robust and could conduct such a Process.