**5. The Synthesized Optimal Control**

According to this approach, it is necessary to find such a control function (7) that the system without perturbations would always have a stable equilibrium point in the state space. Together with that, in the control function a parameter vector is introduced. The value of this parameter vector affects on position of the equilibrium point in the states space

$$\mathbf{u} = \mathbf{g}(\mathbf{x}, \mathbf{q}^\*),\tag{41}$$

where **q**∗ is a parameter vector.

Control function (41) provides for the system without perturbations

$$
\dot{\mathbf{x}} = \mathbf{f}(\mathbf{x}, \mathbf{g}(\mathbf{x}, \mathbf{q}^\*)) \tag{42}
$$

existence of the equilibrium point

$$\mathbf{f}(\mathbf{x}^\*(\mathbf{q}^\*), \mathbf{g}(\mathbf{x}^\*(\mathbf{q}^\*), \mathbf{q}^\*)) = 0,\tag{43}$$

where **x**∗(**q**∗) is a vector of coordinates of equilibrium point, depending on the parameter vector **q**∗. The system (42) satisfies conditions (34)–(36) in the point **x**∗(**q**∗).

Algorithmically, the method of synthesized optimal control first solves the problem of stabilization system synthesis. For solving the synthesis problem, the functional (6) is not used. Purpose of the control synthesis problem is to receive such control function (41) to provide existence of the stable equilibrium point in the state space.

Once the function (41) is found, the optimal control problem is solved next for the mathematical model (42) with the initial conditions (3) and the terminal conditions (4), and with the quality criterion

$$J\_1 = \int\_0^{t\_f} f\_0(\mathbf{x}(t), \mathbf{g}(\mathbf{x}(t), \mathbf{q}^\*(t)))dt + p\_1 \|\mathbf{x}^f - \mathbf{x}(t\_f)\| \to \min\_{\mathbf{q}^\* \in \mathbf{Q}} \tag{44}$$

where *Q* is a compact set in the space of parameters.

In general case, the vector of parameters **q**∗ can be some function **q**∗(*t*). The properties of this function and methods for finding it requires additional studies. In this work this function is found for the original optimal control problem (1)–(6) as a piece-wise constant one.

Thus, in the synthesized optimal control approach, the uncertainty in the right parts is compensated by the stability of the system relative to a point in the state space. Near the equilibrium point, all solutions converge and feasibility principle is satisfied. This first step of stabilization system synthesis is a key idea of the approach, it provides achievement of better results in the tasks with complex environment and noise. However, this approach could not be previously presented as a single computational method, since there was no general numerical approach for solving the problem of control synthesis. Formally, the problem of synthesis of stabilization system involves the construction of such a feedback control module described by some functions that produces control basing on the received data about the object's state and this control makes the object achieve the terminal goal with the optimal value of some given criterion. In the overwhelming majority of cases, the control synthesis problem is solved analytically or technically taking into account the specific properties of the mathematical model. But now modern numerical methods of symbolic regression can be applied to find a solution without reference to specific model equations. Let us consider the issue in more details.

#### **6. The Problem of Control System Synthesis**

Consider the problem statement of the general numerical synthesis of the control system. The mathematical model is

$$
\dot{\mathbf{x}} = \mathbf{f}(\mathbf{x}, \mathbf{u}),
\tag{45}
$$

where **<sup>x</sup>** <sup>∈</sup> <sup>R</sup>*n*, **<sup>u</sup>** <sup>∈</sup> <sup>U</sup> <sup>⊆</sup> <sup>R</sup>*m*.

The domain of initial conditions is given

$$\mathbb{X}\_0 = \{ \mathbf{x}^{0,1}, \dots, \mathbf{x}^{0,K} \} \subseteq \mathbb{R}^n. \tag{46}$$

The terminal condition is given

$$\mathbf{x}^\* = [\mathbf{x}\_1^\* \dots \mathbf{x}\_n^\*]^T \in \mathbb{R}^n. \tag{47}$$

The quality criterion is given

$$J\_3 = \sum\_{i=1}^{K} t\_{f,i} + p\_1 \| \mathbf{x}^\* - \mathbf{x}(t\_{f,i}, \mathbf{x}^{0,i}) \| \to \min\_{\mathbf{u}},\tag{48}$$

where *tf* ,*<sup>i</sup>* is a time of achieving the terminal condition from the initial condition **x**0,*<sup>i</sup>* . It is necessary to find a control in the form (41).

The general formulation of the synthesis problem was posed by V.G. Boltyanskiy in the 60s of the last century [25]. One of the ways to solve it is to reduce the problem to the partial differential equation of Bellman [26,27], who also proposed a method for its solution in the form of a dynamic programming method [26,28]. Bellman's equation in the general case has no solution; therefore, most often it is solved numerically for one initial condition, which in our case is not enough to ensure stability.

To solve the synthesis problem and obtain an equilibrium point, methods of modal control [29] can be applied for linear systems, as well as other analytical methods such as backstepping [30], analytical design of aggregated controllers [31,32], or synthesis based on the application of the Lyapunov function [21,33]. Note that all known analytical synthesis methods for nonlinear systems, when implemented, are associated with a specific type of model, therefore they cannot be considered universal. In practice, linear controllers, such as PID or PI controllers, are often used to ensure stability. Their use is also associated with a specific model, which is linearized in the neighbourhood of the equilibrium point, and their use is not related to the formal statement of the considered synthesis problem.

To solve the synthesis problem in the considered mathematical formulation, it is necessary to find the control function in the form (41). Most of the known methods specify the control function with an accuracy of the parameter values, for example, methods associated with the solution of the Bellman equation, like analytical design of optimal controllers [34], as well as the use of various controllers, including controllers based on very popular now artificial neural networks [35].

This paper proposes to solve the addressed problem numerically. For a solution of the synthesis problem we apply numerical methods of symbolic regression. These methods can look for a structure of the function in the form a special code by some genetic algorithm and also search for the optimal values of parameters in the desired function.

#### **7. Symbolic Regression Methods**

To encode a mathematical expression, it is necessary to define sets of arguments of the mathematical expression and elementary functions. To decode a code of the mathematical expression it is enough to know how many arguments has each elementary function. For encoding elementary function, it is enough to use integer vector with two components. The first component is the number of arguments of the elementary function. The second component is the function number. Arguments of mathematical expression are elementary functions without arguments, therefore the first component of an argument code is zero.

For the control synthesis problem (45)–(48) it is necessary to find a mathematical expression of the control function (41).

Let us define sets of elementary functions.

A set of mathematical expression arguments or elementary functions without arguments includes variables, parameters, and unit elements for elementary functions with two arguments,

$$\mathcal{F}\_0 = \{f\_{0,1} = \mathbf{x}\_1, \dots, f\_{0,n} = \mathbf{x}\_{\mathbb{N}}, f\_{0,n+1} = \mathbf{c}\_1, \dots, f\_{0,n+p}, f\_{0,n+p+1} = \mathbf{c}\_1, \dots, f\_{0,n+p+r} = \mathbf{c}\_V\}, \tag{49}$$

where *xi* is a component of the state vector, *i* = 1, ... , *n*, **x** = [*x*<sup>1</sup> ... *xn*] *<sup>T</sup>*, *ci* is a component of the parametric vector, *i* = 1, ... , *p*, **c** = [*c*<sup>1</sup> ... *cp*] *<sup>p</sup>*, *ei* is a unit element for function with two arguments.

A set of functions with one argument includes an identity function

$$\mathcal{F}\_1 = \{f\_{1,1}(z) = z, f\_{1,2}(z), \dots, f\_{1,\mathcal{W}}(z)\}. \tag{50}$$

A set of functions with two arguments includes such functions, that are associative, commutative and have a unit element

$$\mathbf{F}\_2 = \{f\_{2,1}(z\_1, z\_2), \dots, f\_{2,V}(z\_1, z\_2)\},\tag{51}$$

where each element from the set *F*<sup>2</sup> has the following properties:

− associative

$$f\_{2,j}(f\_{2,j}(z\_1, z\_2), z\_3) = f\_{2,j}(z\_1, f\_{2,j}(z\_2, z\_3)), \; j = 1, \ldots, V,\tag{52}$$

− commutative

$$f\_{2,j}(z\_1, z\_2) = f\_{2,j}(z\_2, z\_1), \; j = 1, \ldots, V,\tag{53}$$

− existing of a unit element

$$f\_{2,j}(z\_1, e\_j) = z\_1, \ f\_{2,j}(e\_j, z\_2) = z\_2, \ j = 1, \ldots, V. \tag{54}$$

To describe the most common mathematical expressions, it is enough functions with one and two arguments. Functions with three and more arguments may not be used.

Any element of the sets (49)–(51) is encoded by integer vector with two arguments

$$\mathbf{s} = \begin{bmatrix} s\_1 \ s\_2 \end{bmatrix}^T,\tag{55}$$

where *s*<sup>1</sup> is the number of arguments, *s*<sup>2</sup> is a function number.

A code of the mathematical expression is a set of codes of elementary functions

$$\mathbf{S} = \mathbf{s}^1 \dots \mathbf{s}^L,\tag{56}$$

where **s***<sup>j</sup>* = [*s j* 1 *s j* 2] *<sup>T</sup>*, *s j* <sup>1</sup> ∈ {0, 1, 2},

$$s\_2^j \in \begin{cases} \{1, \dots, n+p+V\}, \text{if } s\_1^j = 0\\ \{1, \dots, \mathcal{W}\}, \text{if } s\_1^j = 1\\ \{1, \dots, V\}, \text{otherwise} \end{cases} \tag{57}$$

**Theorem 4.** *For the mathematical expression code (57) with L elements to be correct, it is necessary and enough that the following formulas are valid*

$$1 + \sum\_{i=1}^{j} s\_1^i \le L, \ j = 1, \dots, L - 1,\tag{58}$$

$$1 - L + \sum\_{i=1}^{L} s\_1^i = 0.\tag{59}$$

**Proof.** Consider the Formula (58) and add there −*j* in the left and right sides

$$-j + 1 + \sum\_{i=1}^{j} s\_1^i \le L - j. \tag{60}$$

Consider left side of the inequation (60)

$$T(j) = -j + 1 + \sum\_{i=1}^{j} s\_1^i. \tag{61}$$

This equation calculates how many elements from the set of arguments (49) should be after element *j*. The value *T*(*j*) is increasing on 1 after each *s j* <sup>1</sup> = 2, it is not changing after each *s j* <sup>1</sup> = 1, and it is decreasing on 1 after *s j* <sup>1</sup> = 0.

At *j* = *L*, we receive the Equation (59). After the last element *j* = *L* it must be no elements on the right from element *L*.

Assume that the inequation (57) fails. Then from (61) we receive for *j* = *L*

$$T(j) = -L + 1 + \sum\_{i=1}^{j} s\_1^i > 0. \tag{62}$$

This means, that after the last element there are some elements. This does not allow to decode the code. Therefore conditions (57) and (58) are necessary.

Let the inequation (57) and Equation (58) be satisfied. If the element after the element *j* is an argument from the set (49), then *T*(*j*) is decreasing on 1, if it is the function number with one argument, then *T*(*j*) is not changed, if it is the function number with two arguments, then *T*(*j*) increases on 1. Equation (58) shows that the last element from the set (49) does not need arguments. The formula is decoded. Therefore, performing the Formulas (57) and (58) is enough. QED.

From Equation (58) it follows

$$\sum\_{i=1}^{L} s\_1^i = L - 1.\tag{63}$$

Such direct encoding is in the genetic programming [36]. This method of symbolic regression does not include extra elements, therefore codes of different mathematical expressions have different lengths. It is not very comfortable for programming and implementing crossover in genetic programming. For crossover it is necessary to find in the code (55) the sub-code of mathematical expression with the properties (57) and (58). Crossover operation in genetic programming is performed as exchanging sub-codes of mathematical expressions. Searching for sub-codes and exchanging them takes significant time of the algorithm. Other symbolic regression methods that can be effectively used to find a mathematical expression, such as the network operator method [37,38], or Cartesian genetic programming [39,40] have codes of equal length for different mathematical expressions due to redundant elements.

An effective tool in the search for an optimal mathematical expression is the principle of small variations of the basic solution [41]. According to this principle, the search for the mathematical expression can begin in the neighbourhood of one given basic solution. This solution is coded by some symbolic regression method. Other possible solutions are obtained using sets of codes of small variations of the basic solution. Each small variation slightly modifies the basic solution code so that a new code corresponds to some kind of mathematical expression.

To find the optimal mathematical expression by any method of symbolic regression, a special genetic algorithm is used. Depending on the code of symbolic regression, this

genetic algorithm has its own crossover and mutation operations. Using the principle of small variations of the basic solution, crossover and mutation operations are performed on the sets of small variations.

In the numerical solution of control synthesis problems by symbolic regression methods, together with the search of the structure of the mathematical expression, it is advisable to look for the optimal values of the parameter vector **c** = [*c*<sup>1</sup> ... *cp*] *<sup>T</sup>*, which is included in this mathematical expression in the form of its additional arguments (49). For this purpose, it is convenient to use the same genetic algorithm as for finding the structure. In this case, a possible solution is a pair including the code for structure of the mathematical expression and the vector of parameters. When performing a crossover operation, we get not two, but four offsprings. Two offsprings have new mathematical expression structures and new parameter values, and two others inherit parent structures and have only new parameter values. The crossover operation for parameters is performed as in the classical genetic algorithm, by exchanging codes after the crossover point.

It can be seen that the methods of symbolic regression can automate the process of synthesis of control systems, but very little of them are used in this direction. Only few scientific groups [42–44] are developing these approaches for solving the problem of control system synthesis in view of a number of difficulties, such as non-numerical search space and the absence of a metric on it, the complexity of the program code and the absence of publicly available software packages, and so forth.
