A Second-Order Numerical Method for a Class of Optimal Control Problems

Abstract

The numerical solution of optimal control problems through second-order methods is examined in this paper. Controlled processes are described by a system of nonlinear ordinary differential equations. There are two specific characteristics of the class of control actions used. The first one is that controls are searched for in a given class of functions, which depend on unknown parameters to be found by minimizing an objective functional. The parameter values, in general, may be different at different time intervals. The second feature of the considered problem is that the boundaries of time intervals are also optimized with fixed values of the parameters of the control actions in each of the intervals. The special cases of the problem under study are relay control problems with optimized switching moments. In this work, formulas for the gradient and the Hessian matrix of the objective functional with respect to the optimized parameters are obtained. For this, the technique of fast differentiation is used. A comparison of numerical experiment results obtained with the use of first- and second-order optimization methods is presented.

1. Introduction

High-order methods for solving optimization and optimal control problems continue to attract the attention of specialists both from a theoretical point of view and in terms of their application in numerical calculations [1,2,3]. In particular, second-order methods are widely used not only in optimization problems but also in a wide range of related problems (in finding the roots of nonlinear equations using the procedure of weight functions in fractional iterative modifications of the Newton–Raphson method [4,5]; in optimal control problems [6]; in computational fluid dynamics [7,8,9]; etc.).

Their high speed of convergence and finiteness for certain classes of problems is an obvious advantage. However, their widespread use for solving practical optimization problems, and especially optimal control problems in a general formulation, is complicated, mainly for the following two reasons: (1) the necessity of calculating the matrices of second derivatives and (2) the large amount of required computer memory.

The first reason may be crucial if it is not possible to obtain exact expressions for derivatives. As is known, for difficult-to-compute functions, their finite-difference approximations are, firstly, labor-intensive from a computational point of view. Secondly, the accuracy of the approximated values of derivatives is low, and that is why the methods fail to demonstrate their theoretical efficiency.

The second reason is less restrictive. Modern computing tools and operating systems and the availability of large RAM and its dynamic allocation make it possible to solve problems of a fairly large dimension. Nevertheless, there are practical problems that can be reduced to the class of problems under consideration, the solution of which, even in modern computer systems, has certain difficulties. Such problems are, for example, problems of the optimal control of multidimensional (two- and three-dimensional) objects described by partial differential equations.

Numerous works, including overviews, are devoted to the study of various aspects of second-order numerical methods. Therefore, we will not focus on the analysis of these methods. The interested reader can see, for example, [10,11,12,13,14,15].

In this paper, we study a numerical solution to the class of problems of the optimal control of a process described by a system of ordinary differential equations. The control actions are defined as follows: The time interval is divided into subintervals, at each of which the control actions are defined on a class of parametrically defined functions, the parameters of which are not specified. These parameters are required to be optimized in relation to the given criterion of the optimal control problem. The points that define the boundaries of the intervals of constancy of the control action parameters are also optimized.

The problem under study can be classified as a parametric optimal control problem, in which what is being optimized is a finite-dimensional vector that includes the parameters of control actions on each interval, as well as the boundaries of these intervals. To solve the problem, the technique of fast differentiation is used [16,17,18,19,20,21]. Note that this technique, known as “backpropagation”, has enjoyed wide application in artificial intelligence systems, in particular, image recognition systems [20,21].

In this paper, we apply a finite-difference method and obtain an approximated optimal control problem for which we derive the formulas for the gradient and the Hessian matrix of the objective function. These formulas make it possible to use both efficient numerical methods of optimization and available software tools.

The number of intervals is considered given, but the paper provides an algorithm for minimizing the number of switching points and, thus, the number of intervals themselves. The practical advisability of the considered control problem consists in the importance of reducing the number of changes in the modes of functioning of actual dynamic objects. Reducing the number of switching of the control actions leads to a decrease in the wear and tear of technological equipment and the stability of functioning of both the controlled process and the control unit. As it is known, the exact realization of often time-varying control actions is usually associated, as a rule, with technical difficulties.

The use of second-order numerical methods for the class of control actions under consideration is justified by the relatively small dimension of the optimized vector of parameters that determine the control actions. As is known [2,22], second-order methods are most efficient for problems of relatively small dimension of space of the optimized parameters.

Note that the simplest special case of the problems under consideration is the relay control problem with optimized switching times [23].

This paper presents the results of computer experiments obtained when solving a model problem using first- and second-order methods.

2. Statement of the Problem

Let us consider an optimal control problem described by a system of nonlinear differential equations:

$${\dot{x}}_i\left(t\right)=f_i\left(t, x\left(t\right), u\left(t\right)\right), t\in \left(t_0,t_f\right.], i=1,\dots n,$$

(1)

with the initial conditions:

$$x_i\left(t_0\right)=x_{0i}, i=1,\dots n,$$

(2)

and an objective functional, which is to be minimized:

$$J\left(u\right)=\underset{t_0}{\overset{t_f}{\int }}{f}_0\left(t, x\left(t\right), u\left(t\right)\right)dt+\mathsf{\Phi }\left(x\left(t_f\right)\right)\to \underset{u\left(t\right)}{\mathrm{m}\mathrm{i}\mathrm{n}}.$$

(3)

Here, $f_0\left(t, x, u\right)$, $f{\left(t, x, u\right)={(f}_1\left(t, x, u\right),\dots ,f_n\left(t, x, u\right))}^{\mathsf{{\rm T}}}$ are the given functions that are twice continuously differentiable with respect to the second and third arguments and piecewise continuous with respect to the first argument; $\mathsf{\Phi }\left(\bullet \right)$ is a given twice continuously differentiable function; and $x_{0i}, i=1,\dots ,n, t_0, t_f$ are given.

It is required to determine a piecewise continuously differentiable vector function of the phase state $x\left(t\right)=\left(x_1\left(t\right),\dots ,x_n\left(t\right)\right)$ and a piecewise continuous vector function of control actions $u\left(t\right)=\left(u_1\left(t\right),\dots ,u_r\left(t\right)\right)$, satisfying (1) and (2) and minimizing (3).

Let the control actions $u_i\left(t\right), i=1,\dots ,r$ meet the conditions:

$${\underset{\_}{u}}_i\le u_i\left(t\right)\le {\overline{u}}_i, i=1,\dots ,r, t\in \left[t_0,t_f\right],$$

(4)

and be searched in the following form:

$$u_i\left(t\right)=\sum _{j=1}^m{\beta }_{ij}^k{\phi }_j\left(t-{\alpha }_{k-1}\right)={\tilde{\beta }}_i^k{\tilde{\phi }}^k\left(t\right) \mathrm{at} t\in \left[\left.{\alpha }_{k-1},{\alpha }_k\right)\right.,$$

(5)

where ${\tilde{\phi }}^k\left(t\right)={\left({\phi }_1\left(t-{\alpha }_{k-1}\right),\dots ,{\phi }_m\left(t-{\alpha }_{k-1}\right)\right)}^{\mathsf{{\rm T}}}$ at $t\in [{\alpha }_{k-1}$,${\alpha }_k)$, k = 1,…,$\mu$ are the given linearly independent piecewise continuous (basis) functions; $m, {\underset{\_}{u}}_i,{\overline{u}}_i, i=1,\dots r,$ are given; and $\mathsf{{\rm T}}$ is a transposition sign. In (5), we introduced the product of vectors ${\tilde{\beta }}_i^k$ and ${\tilde{\phi }}^k\left(t\right).$ Let us introduce two vectors $\widehat{\alpha }={\left({\alpha }_0,{\alpha }_1,\dots ,{\alpha }_{\mu }\right)}^{\mathsf{{\rm T}}}$ and $\alpha ={\left({\alpha }_1,\dots ,{\alpha }_{\mu -1}\right)}^{\mathsf{{\rm T}}}$, where ${\alpha }_0=t_0,$ ${\alpha }_{\mu }=t_f$ and

$${t_0\le \alpha }_{k-1}\le {\alpha }_k\le t_f, k=1,\dots ,\mu .$$

(6)

For the sake of brevity, we will use the following notation for $t\in [{\alpha }_{k-1}$,${\alpha }_k)$:

$$\tilde{\phi }\left(t\right)={({\phi }_1^k\left(t\right),\dots ,{\phi }_m^k\left(t\right))=({\phi }_1\left(t-{\alpha }_{k-1}\right),\dots ,{\phi }_m\left(t-{\alpha }_{k-1}\right))}^{\mathsf{{\rm T}}},$$

$$\tilde{\beta }\left(t\right)={\beta }^k, k=1,\dots ,\mu ,$$

where ${\beta }^k=\left(\left({\beta }_{ij}^k\right)\right),i=1,\dots ,r,j=1,\dots ,m,\beta ={\left({\beta }^1,\dots ,{\beta }^{\mathrm{\mu }}\right)}^{\mathsf{{\rm T}}}$.

Values $\alpha$ and coefficients $\beta$ are optimized values obeying (4) and (6).

In the formulas given below, we will use both the component-wise form of notation and the vector-matrix form, if this does not cause problems in understanding the formulas.

Thus, in the optimal control problem under consideration, it is required to find optimal values of the finite-dimensional vector $\nu =\left(\alpha ,\beta \right)\in R^{\left(mr+1\right)\mu -1}$, $\alpha \in R^{\mu -1},$ $\beta \in R^{mr\mu }$. In doing this, a corresponding control function $u\left(t\right)$, defined from (5) and satisfying conditions (4) and (6), together with the corresponding phase vector $x\left(\bullet \right)\in R^n$, defined from Cauchy problem (1), (2), must minimize functional (3).

On the one hand, the posed problem can be attributed to the class of parametric optimal control problems; on the other hand, it is a finite-dimensional optimization problem.

It is possible to prove the differentiability and (strict) convexity of the functional of problem (1)–(6) with respect to the optimized parameters, provided that the functional of the optimal control problem (1)–(4) is differentiable and (strictly) convex. The existence and uniqueness of the solution to the optimal control problem (1)–(6), as well as the convergence of numerical optimization procedures based on first- and second-order methods [22], follow from the fulfillment of these conditions.

The simplest special case of the problem under consideration is a control problem in a class of piecewise constant functions with the optimization of control switching times. To do this, it is enough to assume $m=1$, ${\phi }_1\left(t\right)\equiv 1$.

The problems of the optimal control of objects described by partial differential equations can be reduced to the problems under consideration if Rothe’s method is used to approximate them using spatial variables [24,25].

An obvious advantage of the considered class of control actions is the absence of the need to frequently change the values of the control action parameters, which leads to the stability and robustness of the control system and, consequently, the reliability of its functioning.

3. Numerical Solution Approach to the Problem

To numerically solve the problem under consideration, we use its finite-difference approximation. Next, to solve the resulting finite-dimensional optimization problem using second-order numerical methods, we obtain formulas for the derivatives of the objective functional of the first and second orders in the space of the optimized control action parameters $\nu =\left(\alpha ,\beta \right)$. To obtain these formulas, we will use the technique of fast differentiation [16,17,18,19,20,21].

For the finite-difference approximation of Cauchy problem (1) (2) and functional (3), we apply any known formulas with a high order of accuracy.

Let us divide the segment $\left[t_0,t_f\right]$ into $N$ segments with regular points ${\overline{t}}_j=t_0+jh$, $j=0,\dots ,N$, $\overline{h}=\left(t_f-t_0\right)/N$, ${\overline{t}}_N=t_f$. Let the $j$-th switching point ${\alpha }_j$ belong to the ${\eta }_j$-th regular half-interval:

$${\alpha }_j\in \left[\left.{\overline{t}}_{{\eta }_j-1},{\overline{t}}_{{\eta }_j}\right)\right., j=1,\dots ,\mu .$$

Let us add switching points ${\alpha }_j$, $j=1,\dots ,\mu -1$ to $\left(N+1\right)$ points ${\overline{t}}_0,{\overline{t}}_1\dots ,{\overline{t}}_N$ and increase the number of segments to $N_{\mu }=\left(N+\mu -1\right)$. Let us denote the obtained points by $t_0,t_1,\dots ,t_{N_{\mu }},h_{j-1}=t_j-t_{j-1}$, $j=1,\dots ,N_{\mu }$ as the steps of the non-uniform partition of the segment $\left[t_0,t_f\right]$. It is clear that ${\overline{t}}_N=t_{N_{\mu }}$.

Let point ${\alpha }_j$ be the ${\varkappa }_j$-th point of the non-uniform partition of the segment $\left[t_0,t_f\right]:t_{{\varkappa }_j}={\alpha }_j,j=1,\dots ,\mu -1,t_0={\alpha }_0,t_N={\alpha }_{\mu }$. Then, $h_{{\varkappa }_j-1}={\alpha }_j-t_{{\varkappa }_j-1}=t_{{\varkappa }_j}-t_{{\varkappa }_j-1}$ and $h_{{\varkappa }_j}=t_{{\varkappa }_j+1}-{\alpha }_j=t_{{\varkappa }_j+1}-t_{{\varkappa }_j}.$

In the most generalized form, the finite-dimensional approximation of problem (1), (2) can be written as follows:

$$\begin{gathered} x_i^{j+1}={\mathcal{F}}_i^j\left(t_j,x^j,\tilde{\beta }\left(t_j\right)\tilde{\phi }\left(t_j\right)\right)=x_i^j+h_j{F}_i\left(t_j,x^j,\tilde{\beta }\left(t_j\right)\tilde{\phi }\left(t_j\right)\right), \\ {x}_i^0=x_{0i}, i=1,\dots n,j=0,\dots ,N_{\mu }-1. \end{gathered}$$

(7)

In (7), the $n$-dimensional vector-function $\mathcal{F}\left(.,.,.\right)$ is determined by the approximation scheme, which is used for Cauchy problem (1), (2). For example, if an explicit Euler method scheme is applied, then

$$F\left(t_j,x^j,\tilde{\beta }\left(t_j\right)\tilde{\phi }\left(t_j\right)\right)=f\left(t_j,x^j,\tilde{\beta }\left(t_j\right)\tilde{\phi }\left(t_j\right)\right)$$

(8)

For the fourth-order Runge–Kutta method, we obtain

$$F\left(t_j,x^j,\tilde{\beta }\left(t_j\right)\tilde{\phi }\left(t_j\right)\right)=\left(k_1^j+2k_2^j+2k_3^j+k_4^j\right)/6,$$

(9)

$$k_1^j=f\left(t_j,x^j,\tilde{\beta }\left(t_j\right)\tilde{\phi }\left(t_j\right)\right),$$

$$k_2^j=f\left(t_j+h_j/2,x^j+h_j{k}_1^j/2,\tilde{\beta }\left(t_j\right)\tilde{\phi }\left(t_j+h_j/2\right)\right),$$

$$k_3^j=f\left(t_j+h_j/2,x^j+h_j{k}_2^j/2,\tilde{\beta }\left(t_j\right)\tilde{\phi }\left(t_j+h_j/2\right)\right),$$

$$k_4^j=f\left(t_j+h_j,x^j+h_j{k}_3^j,\tilde{\beta }\left(t_j\right)\tilde{\phi }\left(t_j+h_j\right)\right),$$

$k_1^j,k_2^j,k_3^j,k_4^j$ are $n$-dimensional vector functions of three arguments.

It is evident that virtually all known numerical methods applicable to the Cauchy problem can be described by (7).

To simplify the given formulas when approximating the functional, we use only regular points ${\overline{t}}_j$, $j=0,\dots ,N$:

$$I\left(v\right)=\overline{h}\sum _{j=0}^N{\sigma }_j{f}_0\left({\overline{t}}_j,x^j,\tilde{\beta }\left(t_j\right)\tilde{\phi }\left({\overline{t}}_j\right)\right)+\mathsf{\Phi }\left(x\left({\overline{t}}_N\right)\right).$$

(10)

In particular, for values ${\sigma }_j$, $j=0,\dots ,N$, the well-known Simpson method of fourth-order accuracy can be used.

It is important that under the imposed conditions on the functions involved in problem (1)–(4), the solution of the approximated difference optimal control problem (7)–(10) converges to the solution of the original problem when N → ∞ [2,22].

For the numerical solution of the discrete optimal control problem (7)–(10), we use iterative methods based on Newton’s method. With this purpose, let us find the formulas for the first- and second-order derivatives $\partial I\left(v\right)/\partial \beta ,$ $\partial I\left(v\right)/\partial \alpha ,$ ${\partial }^{2}I\left(v\right)/\partial {\alpha }^2,$ ${\partial }^{2}I\left(v\right)/\partial {\beta }^2,$ ${\partial }^{2}I\left(v\right)/(\partial \beta \partial \alpha )$, applying the technique of fast differentiation and the specifics of the dependencies between the phase variables $x^j$, $j=0,\dots ,N_{\mu },$ and the optimized variables $v=\left(\alpha ,\beta \right)$.

Let us introduce the following index sets corresponding to dependencies (7):

$$L=\left\{0, 1, \dots ,N\right\}, \overline{L}=\left\{0, 1, \dots ,N,\dots ,N_{\mu }\right\},$$

$$L_{x^j}=\left\{s\in \overline{L}: x^s=\mathcal{F}\left(\dots ,x^j, \dots \right)\right\}, j\in \overline{L,}$$

$$L_{{\alpha }_j}=\left\{s\in \overline{L}: x^s=\mathcal{F}\left(\dots ,{\alpha }_j, \dots \right)\right\}, j=1,\dots ,\mu -1,$$

$$L_{{\beta }^k}=\left\{s\in \overline{L}: x^s=\mathcal{F}\left(\dots ,{\beta }_{ij}^k, \dots \right)\right\}, k=1,\dots ,\mu .$$

As follows from the definition of the sets $L_{x^j}$, $L_{{\alpha }_j}$ and $L_{{\beta }^k}$, they are determined by the indices of those variables $x^s$, $s=0,\dots ,N_{\mu }$, the values of which are directly affected by $x^j$, ${\alpha }_j$ and ${\beta }^k$, respectively.

For example, in the case of using the Euler (8) and Runge–Kutta methods, we obtain:

$$L_{x^j}=\left\{j+1\right\}, L_{{\alpha }_j}=\left\{{\varkappa }_j-1,{\varkappa }_j\right\},$$

$$h_{{\varkappa }_j-1}={\alpha }_j-t_{{\varkappa }_j-1}, h_{{\varkappa }_j}=t_{{\varkappa }_j+1}-{\alpha }_j, j=1,\dots ,N_{\mu }-1,$$

$$L_{{\beta }^k}=\left\{{\varkappa }_k,{\varkappa }_k+1,\dots ,{\varkappa }_{k+1}-1\right\}.$$

Let us introduce the following $n$-dimensional vector ${\psi }^j$=${\left({\psi }_1^j,\dots ,{\psi }_n^j\right)}^{\mathsf{{\rm T}}}:$

$${\psi }_i^j={\displaystyle \frac{dI\left(v\right)}{d{x}_i^j}}, i=1,\dots ,n, j=0,\dots ,N_{\mu }.$$

(11)

Here, the derivative is understood as the total, with regard for the dependence (7), i.e., the influence on the functional of both the values of vector $x^j$ and vectors $x^{\xi }$, $\xi \in L_{x^j}$, depending on $x^j$. This implies

$${\psi }_i^j={\displaystyle \frac{\partial I\left(v\right)}{\partial x_i^j}}+\sum _{\xi \in L_{x^j}}\sum _{s=1}^n{\displaystyle \frac{\partial x_s^{\xi }}{\partial x_i^j}}{\psi }_s^{\xi }, i=1,\dots ,n, j=0,\dots ,N_{\mu }.$$

(12)

Here and below, expressions of the vector and matrix derivatives $\partial I\left(v\right)/\partial x^j,$ $\partial x^{\xi }/\partial x^j,$ are understood as partial derivatives, namely, the derivatives with respect to the variables explicitly involved in the dependencies. Particularly, in the case of using the Euler method (8), we obtain

$$\begin{gathered} {\psi }^j=h_j{\sigma }_j{\displaystyle \frac{\partial f_0\left(t_j,x^j,\tilde{\beta }\left(t_j\right)\tilde{\phi }\left(t_j\right)\right)}{\partial x}} \\ +\left(E+h_j{\displaystyle \frac{\partial f\left(t_j,x^j,\tilde{\beta }\left(t_j\right)\tilde{\phi }\left(t_j\right)\right)}{\partial x}}\right){\psi }^{j+1}, j=N-1,\dots ,0, \\ i=N_{\mu }-1,\dots ,0, \end{gathered}$$

(13)

$${\psi }^{N_{\mu }}={\overline{h}}_N{\sigma }_N{\displaystyle \frac{\partial f_0\left({\overline{t}}_N, x^N, \tilde{\beta }\left({\overline{t}}_N\right)\tilde{\phi }\left({\overline{t}}_N\right)\right)}{\partial x}}+{\displaystyle \frac{\partial \mathsf{\Phi }\left(x^N\right)}{\partial x}}.$$

(14)

In (14), it is considered that ${\overline{t}}_N=t_{N_{\mu }}=t_f.$

In the case of using the Runge–Kutta method (9) to approximate (1), (2), it is easy to see that

$$\begin{gathered} {\psi }^j=h_j{\sigma }_j{\displaystyle \frac{\partial f_0\left(t_j,x^j,\tilde{\beta }\left(t_j\right)\tilde{\phi }\left(t_j\right)\right)}{\partial x}} \\ +\left({\mathrm{E}}_n+{\displaystyle \frac{h_j}{6}}{\displaystyle \frac{\partial k_1^j}{\partial x^j}}+{\displaystyle \frac{h_j}{3}}{\displaystyle \frac{\partial k_2^j}{\partial x^j}}+{\displaystyle \frac{h_j}{3}}{\displaystyle \frac{\partial k_3^j}{\partial x^j}}+{\displaystyle \frac{h_j}{2}}{\displaystyle \frac{\partial k_4^j}{\partial x^j}}\right){\psi }^{j+1}, j=N_{\mu }-1,\dots ,0, \end{gathered}$$

(15)

where

$${\displaystyle \frac{\partial k_1^j}{\partial x^j}}={\displaystyle \frac{\partial f\left(t_j,x^j,\tilde{\beta }\left(t_j\right)\tilde{\phi }\left(t_j\right)\right)}{\partial x}},$$

$${\displaystyle \frac{\partial k_2^j}{\partial x^j}}={\displaystyle \frac{\partial f(t^j+h_j/2,x^j,k_1{h}_j/2,\tilde{\beta }\left(t_j\right)\tilde{\phi }\left(t_j\right))}{\partial x}}[{\mathrm{E}}_n+{\displaystyle \frac{1}{2}}{\displaystyle \frac{\partial k_1^j}{\partial x^j}}],$$

$${\displaystyle \frac{\partial k_3^j}{\partial x^j}}={\displaystyle \frac{\partial f(t^j+h_j/2,x^j,k_1{h}_j/2,\tilde{\beta }\left(t_j\right)\tilde{\phi }\left(t_j\right))}{\partial x}}[{\mathrm{E}}_n+{\displaystyle \frac{1}{2}}{\displaystyle \frac{\partial k_2^j}{\partial x^j}}],$$

$${\displaystyle \frac{\partial k_4^j}{\partial x^j}}={\displaystyle \frac{\partial f(t^j+h_{j,},x^j+h_j{k}_3^j,\tilde{\beta }\left(t_j\right)\tilde{\phi }\left(t_j\right))}{\partial x}}[{\mathrm{E}}_n+h_j{\displaystyle \frac{\partial k_1^j}{\partial x^j}}],$$

$\partial k_s^j/\partial x, s=1,\dots ,4$ are n-dimensional square matrix functions, and ${\mathrm{E}}_n$ is an n-dimensional unit square matrix.

Then, from (7), (9) and (15), we obtain:

$${\psi }^{N_{\mu }}={\overline{h}}_N{\sigma }_N{\displaystyle \frac{\partial f_0\left({\overline{t}}_N, x^N, \tilde{\beta }\left({\overline{t}}_N\right)\tilde{\phi }\left({\overline{t}}_N\right)\right)}{\partial x}}+{\displaystyle \frac{\partial \mathsf{\Phi }\left(x^N\right)}{\partial x}}.$$

(16)

It is evident that condition (16) for the adjoint system at the right end of time interval in the case of using the Runge–Kutta method coincides with condition (14) obtained when applying the Euler’s method.

Then, the following formulas hold for the components of the gradient of functional $I\left(\nu \right)$:

$${\displaystyle \frac{dI\left(v\right)}{{d\beta }_{ij}^k}}={\displaystyle \frac{\partial I\left(v\right)}{\partial {\beta }_{ij}^k}}+\sum _{s\in L_{{\beta }^k}}\sum _{\theta =1}^n{\displaystyle \frac{\partial x_{\theta }^s}{\partial {\beta }_{ij}^k}}{\displaystyle \frac{dI\left(v\right)}{d{x}_{\theta }^s}}={\displaystyle \frac{\partial I\left(v\right)}{\partial {\beta }_{ij}^k}}+\sum _{s\in {\varkappa }_k}^{{\varkappa }_{k+1}-1}\sum _{\theta =1}^n{\displaystyle \frac{\partial x_{\theta }^s}{\partial {\beta }_{ij}^k}}{\psi }_{\theta }^s,$$

(17)

$k=1,\dots ,\mu ,i=1,\dots ,r,j=1,\dots ,m.$

$${\displaystyle \frac{dI(v)}{d{\alpha }_k}}={\displaystyle \frac{\partial I(v)}{\partial {\alpha }_k}}+\sum _{s\in L_{{\alpha }_k}}\sum _{\theta =1}^n{\displaystyle \frac{\partial x_{\theta }^s}{\partial {\alpha }_k}}{\displaystyle \frac{dI(v)}{d{x}_{\theta }^s}}=\sum _{\theta =1}^n[({\alpha }_k-t_{{\varkappa }_{k-1}}){\psi }_{\theta }^{{\varkappa }_{k-1}}-(t_{{\varkappa }_k}-{\alpha }_k){\psi }_{\theta }^{{\varkappa }_k}],$$

(18)

$k=1,\dots ,\mu -1.$

In (17), the derivatives $\partial x_s^k/\partial {\beta }_{ij}^k,$ $\partial I\left(\nu \right)/\partial {\beta }_{ij}^k$ are determined by the approximation schemes of problem (1), (2) and functional (3). Since only regular points $t_i, i=1,\dots , N_{\mu },$ are used when approximating functional (3), then $\partial I\left(v\right)/\partial {\alpha }_k=0$ in (18).

Let us consider obtaining calculation formulas for the elements of the matrix $d^{2}I\left(v\right)/d{\beta }^2$:

$${\displaystyle \frac{d}{d{\beta }_{i_2{j}_2}^{k_2}}}\left({\displaystyle \frac{dI\left(v\right)}{d{\beta }_{i_1{j}_1}^{k_1}}}\right)={\displaystyle \frac{\partial }{\partial {\beta }_{i_2{j}_2}^{k_2}}}\left({\displaystyle \frac{dI\left(v\right)}{d{\beta }_{i_1{j}_1}^{k_1}}}\right)+\sum _{s_2\in L_{{\beta }^{k_2}}}\sum _{{\theta }_2=1}^n{\displaystyle \frac{\partial x_{{\theta }_2}^{s_2}}{\partial {\beta }_{i_2{j}_2}^{k_2}}}{\displaystyle \frac{d^{2}I\left(v\right)}{d{\beta }_{i_1{j}_1}^{k_1}d{x}_{{\theta }_2}^{s_2}}},$$

$$k_1, k_2=1,\dots ,\mu -1, i_1, i_2=1,\dots ,r, j_1, j_2=1,\dots ,m,$$

$${\displaystyle \frac{\partial }{\partial {\beta }_{i_2{j}_2}^{k_2}}}\left({\displaystyle \frac{dI\left(v\right)}{d{\beta }_{i_1{j}_1}^{k_1}}}\right)={\displaystyle \frac{\partial }{\partial {\beta }_{i_2{j}_2}^{k_2}}}({\displaystyle \frac{dI\left(v\right)}{d{\beta }_{i_1{j}_1}^{k_1}}}+\sum _{s_1\in L_{{\beta }^{k_1}}}\sum _{{\theta }_1=1}^n{\displaystyle \frac{\partial x_{{\theta }_1}^{s_1}}{\partial {\beta }_{i_2{j}_2}^{k_2}}}{\displaystyle \frac{dI\left(v\right)}{d{x}_{{\theta }_1}^{s_1}}})={\displaystyle \frac{{\partial }^{2}I\left(\nu \right)}{\partial {\beta }_{i_2{j}_2}^{k_2}\partial {\beta }_{i_1{j}_1}^{k_1}}}$$

$$+{\sum }_{s_1\in L_{{\beta }^{k_1}}\cap L_{{\beta }^{k_2}}}{\sum }_{{\theta }_1=1}^n{\displaystyle \frac{{\partial }^2{x}_{{\theta }_1}^{s_1}}{\partial {\beta }_{i_1{j}_1}^{k_1}\partial {\beta }_{i_2{j}_2}^{k_2}}}{\displaystyle \frac{dI\left(v\right)}{d{x}_{{\theta }_1}^{s_1}}}+{\sum }_{s_1\in L_{{\beta }^{k_1}}}{\sum }_{{\theta }_1=1}^n{\displaystyle \frac{\partial x_{{\theta }_1}^{s_1}}{\partial {\beta }_{i_1{j}_1}^{k_1}}}{\displaystyle \frac{\partial }{{\partial \beta }_{i_2{j}_2}^{k_2}}}{\displaystyle \frac{dI\left(v\right)}{d{x}_{{\theta }_1}^{s_1}}},$$

$${\displaystyle \frac{d^{2}I\left(v\right)}{d{\beta }_{i_1{j}_1}^{k_1}d{x}_{{\theta }_2}^{s_2}}}={\displaystyle \frac{\partial }{{\partial \beta }_{i_1{j}_1}^{k_1}}}{\displaystyle \frac{dI\left(v\right)}{d{x}_{{\theta }_2}^{s_2}}}+{\sum }_{s_3\in L_{{\beta }^{k_2}}}{\sum }_{{\theta }_3=1}^n{\displaystyle \frac{\partial x_{{\theta }_3}^{s_3}}{\partial {\beta }_{i_1{j}_1}^{k_1}}}{\displaystyle \frac{d^{2}I\left(v\right)}{d{x}_{{\theta }_2}^{s_2}d{x}_{{\theta }_3}^{s_3}}}.$$

Finally, for the elements of the matrix $d^{2}I\left(v\right)/d{\beta }^2$, we obtain:

$$\begin{gathered} {\displaystyle \frac{d^{2}I\left(v\right)}{d{\beta }_{i_1{j}_1}^{k_1}d{\beta }_{i_2{j}_2}^{k_2}}}={\displaystyle \frac{{\partial }^{2}I\left(v\right)}{\partial {\beta }_{i_2{j}_2}^{k_2}\partial {\beta }_{i_1{j}_1}^{k_1}}}+{\sum }_{s_1\in L_{{\beta }^{k_1}}}{\sum }_{{\theta }_1=1}^n{\displaystyle \frac{{\partial }^2{x}_{{\theta }_1}^{s_1}}{\partial {\beta }_{i_1{j}_1}^{k_1}{\partial \beta }_{i_2{j}_2}^{k_2}}}{\displaystyle \frac{dI\left(v\right)}{d{x}_{{\theta }_1}^{s_1}}} \\ +{\sum }_{s_1\in L_{{\beta }^{k_1}}}{\sum }_{{\theta }_1=1}^n{\displaystyle \frac{\partial x_{{\theta }_1}^{s_1}}{\partial {\beta }_{i_1{j}_1}^{k_1}}}{\displaystyle \frac{\partial }{{\partial \beta }_{i_2{j}_2}^{k_2}}}{\displaystyle \frac{dI\left(v\right)}{d{x}_{{\theta }_1}^{s_1}}}+{\sum }_{s_2\in L_{{\beta }^{k_2}}}{\sum }_{{\theta }_2=1}^n{\displaystyle \frac{\partial x_{{\theta }_2}^{s_2}}{\partial {\beta }_{i_2{j}_2}^{k_2}}}{\displaystyle \frac{\partial }{{\partial \beta }_{i_1{j}_1}^{k_1}}}{\displaystyle \frac{dI\left(v\right)}{d{x}_{{\theta }_2}^{s_2}}} \\ +\sum _{s_2\in L_{{\beta }^{k_2}}}\sum _{{\theta }_2=1}^n\sum _{s_3\in L_{{\beta }^{k_1}}}\sum _{{\theta }_3=1}^n{\displaystyle \frac{\partial x_{{\theta }_2}^{s_2}}{\partial {\beta }_{i_2{j}_2}^{k_2}}}{\displaystyle \frac{\partial x_{{\theta }_3}^{s_3}}{\partial {\beta }_{i_1{j}_1}^{k_1}}}{\displaystyle \frac{d^{2}I\left(v\right)}{d{x}_{{\theta }_2}^{s_2}d{x}_{{\theta }_3}^{s_3}}}. \end{gathered}$$

(19)

As can be seen from (19), when calculating matrix $d^{2}I\left(v\right)/d{\beta }^2$, it is required to calculate matrices $\partial (dI\left(v\right)/dx)/\partial \beta$ and $d^{2}I\left(v\right)/d{x}^2$. Let us derive recurrent formulas for these matrices. Let us introduce the matrices:

$$A_{lgsq}={\displaystyle \frac{d^{2}I\left(v\right)}{d{x}_l^{g}d{x}_s^q}}, l,s=1,\dots ,n, g,q=1,\dots ,N_{\mu }.$$

$$B_{ijklg}={\displaystyle \frac{\partial }{\partial {\beta }_{ij}^k}}{\displaystyle \frac{dI\left(v\right)}{d{x}_l^g}}, i=1,\dots ,r, j=1,\dots ,m, k=1,\dots ,\mu , l=1,\dots ,n, g=1,\dots ,N_{\mu }.$$

It is clear that

$${\displaystyle \frac{d^{2}I\left(v\right)}{d{x}_l^{g}d{x}_s^q}}={\displaystyle \frac{\partial }{\partial x_l^g}}{\displaystyle \frac{dI\left(v\right)}{d{x}_s^q}}+{\sum }_{\theta =1}^n{\sum }_{i\in L_{x^g}}{\displaystyle \frac{\partial x_{\theta }^i}{\partial x_l^g}}{\displaystyle \frac{d^{2}I\left(v\right)}{d{x}_{\theta }^{i}d{x}_s^q}}.$$

For the elements of matrix $A$, we obtain recurrent relations:

$$A_{lgsq}={\displaystyle \frac{\partial }{\partial x_l^g}}{\displaystyle \frac{dI\left(v\right)}{d{x}_s^q}}+\sum _{\theta =1}^n\sum _{i\in L_{x^{\theta }}}{\displaystyle \frac{\partial x_{\theta }^i}{\partial x_l^g}}{A}_{\theta isq}$$

Let us find similar relations for matrix $\partial (dI\left(v\right)/dx)/\partial x.$ Let us denote:

$$C_{lgsq}={\displaystyle \frac{\partial }{\partial x_l^g }}{\displaystyle \frac{dI\left(v\right)}{d{x}_s^q}}={\displaystyle \frac{\partial }{\partial x_l^g }}\left({\displaystyle \frac{\partial I\left(v\right)}{\partial x_s^q }}+\sum _{\theta =1}^n\sum _{i\in L_{x^q}}{\displaystyle \frac{\partial x_{\theta }^i}{\partial x_s^q}}{\displaystyle \frac{dI\left(v\right)}{d{x}_{\theta }^i}}\right)$$

$$={\displaystyle \frac{{\partial }^{2}I\left(v\right)}{\partial x_l^g\partial x_s^q}}+\sum _{\theta =1}^n\sum _{i\in L_{x^q}\cap L_{x^g}}{\displaystyle \frac{{\partial }^2{x}_{\theta }^i}{\partial x_l^g\partial x_s^q}}{\displaystyle \frac{dI\left(v\right)}{d{x}_{\theta }^i}}+\sum _{\theta =1}^n\sum _{i\in L_{x^q}}{\displaystyle \frac{\partial x_{\theta }^i}{\partial x_s^q}}{\displaystyle \frac{\partial }{\partial x_l^g}}{\displaystyle \frac{dI\left(v\right)}{d{x}_{\theta }^i}}$$

$$={\displaystyle \frac{{\partial }^{2}I\left(v\right)}{\partial x_l^g\partial x_s^q}}+\sum _{\theta =1}^n\sum _{i\in L_{x^q}\cap L_{x^g}}{\displaystyle \frac{{\partial }^2{x}_{\theta }^i}{\partial x_l^g\partial x_s^q}}{\displaystyle \frac{dI\left(v\right)}{d{x}_{\theta }^i}}+\sum _{\theta =1}^n\sum _{i\in L_{x^q}}{\displaystyle \frac{\partial x_{\theta }^i}{\partial x_s^q}}{C}_{lg\theta i}$$

Finally, we obtain recurrent relations for matrix $B$.

$${\displaystyle \frac{\partial }{\partial {\beta }_{ij}^k }}{\displaystyle \frac{dI\left(v\right)}{d{x}_l^g}}={\displaystyle \frac{\partial }{\partial {\beta }_{ij}^k }}\left({\displaystyle \frac{\partial I\left(v\right)}{\partial x_l^g }}+\sum _{\theta =1}^n\sum _{s\in L_{x^g}}{\displaystyle \frac{\partial x_{\theta }^s}{\partial x_l^g}}{\displaystyle \frac{dI\left(v\right)}{d{x}_{\theta }^s}}\right)={\displaystyle \frac{{\partial }^{2}I\left(v\right)}{\partial {\beta }_{ij}^k\partial x_l^g}}$$

$$+\sum _{\theta =1}^n\sum _{s\in L_{x^g}\cap L_{{\beta }^k}}{\displaystyle \frac{{\partial }^2{x}_{\theta }^s}{\partial {\beta }_{ij}^k\partial x_l^g}}{\displaystyle \frac{dI\left(v\right)}{d{x}_{\theta }^s}}+\sum _{\theta =1}^n\sum _{s\in L_{x^g}}{\displaystyle \frac{\partial x_{\theta }^s}{\partial x_l^g}}{\displaystyle \frac{\partial }{\partial {\beta }_{ij}^k}}{\displaystyle \frac{dI\left(v\right)}{d{x}_{\theta }^s}}$$

$$={\displaystyle \frac{{\partial }^{2}I\left(v\right)}{\partial {\beta }_{ij}^k\partial x_l^g}}+\sum _{\theta =1}^n\sum _{s\in L_{x^g}\cap L_{{\beta }^k}}{\displaystyle \frac{{\partial }^2{x}_{\theta }^s}{\partial {\beta }_{ij}^k\partial x_l^g}}{\displaystyle \frac{dI\left(v\right)}{d{x}_{\theta }^s}}+\sum _{\theta =1}^n\sum _{i\in L_{x^g}}{\displaystyle \frac{\partial x_{\theta }^s}{\partial x_l^g}}{C}_{ljk\theta s}.$$

Using the obtained recurrent relations, we calculate the matrices $A, C and B$ and then matrix $d^{2}I\left(v\right)/d{\beta }^2$.

The matrices $d^{2}I\left(v\right)/\left(d\alpha d\beta \right)$ and $d^{2}I\left(v\right)/d{\alpha }^2$ are calculated in a similar way.

Formulas (11)–(18) make it possible to use any known numerical scheme to approximate Cauchy problem (1), (2) within the frame of (7) and apply first-order method optimization to solve the approximated discrete control problem (5), (7), (10) under constraints (4), (6) [2]. The obtained formulas for the Hessian matrix can be used when applying second-order methods, which are more efficient in terms of the number of iterations and the accuracy of the solutions.

It is essential that the obtained formulas for the gradient and Hessian matrix of the objective function are accurate and consistent with the used schemes of methods for the finite-dimensional approximation of Cauchy problems with respect to the direct and adjoint systems of differential equations for the continuous case. For example, (14) and (16) are more accurate in comparison with the frequently used formula of approximation of the condition for the adjoint variable using the formula ${\psi }^N=\partial \mathsf{\Phi }\left(x\left(t_{N_{\mu }}\right)\right)/\partial x$ [18,24,25].

4. Procedure for Optimizing the Number of Switchings

As mentioned above, reducing the number of switchings, i.e., the dimension of the vector $\alpha \mathsf{ϵ}{R}^{\mu }$, is important in practice. This leads to the simplification, reliability and robustness of the control system. Let us denote by $J_{\mu }^{*}\left(\nu \right)$, the optimal value of the minimized functional (10) with respect to $\nu$ for a given value of $\mu$.

Let $J^{*}=J\left(u^{*}\right)$ denote the minimum value of the functional of problem (1)–(4) in the class of piecewise continuous control functions $u\left(t\right)$.

Then, it is clear that the following relation holds

$$J^{*}\le J_{{\mu }_1}^{*}\left(\nu \right)\le J_{{\mu }_2}^{*}\left(\nu \right) \mathrm{for} {\mu }_2<{\mu }_1.$$

and

$$J^{*}=\underset{\mu \to \infty }{\lim }{J}_{\mu }^{*}\left(\nu \right).$$

(20)

At the same time, from (20), it follows that for an arbitrarily small $\delta >0$, there is such ${\mu }^{*}$ that the following relation holds:

$$J_{\mu }^{*}\left(\nu \right)-J^{*}\le \delta \mathrm{at} \mu \ge {\mu }^{*}.$$

Hence, for real problems, to select a wise number of switchings $\mu$ for a given value of $\delta$, algorithms based on the optimal methods of one-dimensional minimization (golden section, dichotomy, etc.) can be used.

The number of control action switching points can also be reduced based on the results of solving problem (1)–(5) for a given value of $\mu$. In case all parameters ${\beta }_{ij}^k$ and ${\beta }_{ij}^{k-1}$, $i=1,\dots ,$ $r, j=1,\dots ,m,$ or ${\alpha }_k$ and ${\alpha }_{k-1}$ coincide or are close enough, i.e.,

$$\left|{\beta }_{ij}^k-{\beta }_{ij}^{k-1}\right|\le \epsilon , i=1,\dots ,r, j=1,\dots ,m,$$

or

$$\left|{\alpha }_k-{\alpha }_{k-1}\right|\le \epsilon$$

for any $k, k=1,\dots ,\mu ,$ then the interval of parameter constancy $\left[\left.{\alpha }_{k-1},{\alpha }_k\right)\right.$ can be removed from consideration, and $\mu$ can be reduced by one unit.

5. Results of Computer Experiments

To illustrate the efficiency of the proposed approach and formulas for the numerical solution of optimal control problems, consider the following problem:

$${\dot{x}}_1\left(t\right)=x_2\left(t\right), {\dot{x}}_2\left(t\right)=u\left(t\right)-\sin x_1\left(t\right), t\in \left(\mathrm{0,5}\right],$$

$$x_1\left(0\right)=5, x_2\left(0\right)=0,$$

$$J\left(u\right)=x_1^2\left(5\right)+x_1^2\left(5\right)\to min, \left|u\left(t\right)\right|\le 1.$$

Control was sought in the class of piecewise constant functions with optimized switching points. The exact solution to the problem is unknown due to its nonlinearity, but the results obtained for this problem in [26,27,28] are close to each other and, apparently, are almost optimal.

To solve the direct and adjoint Cauchy problems, the Runge–Kutta method (9) with step h = 0.01 was used.

The test problem was solved for six initial points considered in [27]. The results are given in

Table 1. Numerical results of solving test problem for initial number of switchings set to 2.

Case	$\mathit{v}$0	$\mathit{v}$*	$\mathit{J}$*	Niter
1.1	α0 = (0.780, 3.460) β0 = (0.70, −0.60, 0.50)	α* = (0.982, 4.550) β* = (1.0, −1.0, 1.0)	11.908	17
		α* = (0.950, 4.511) β* = (1.0, −1.0, 1.0)	11.962	24
1.2	α0 = (0.780, 3.460) β0 = (2.00, −2.00, 0.50)	α* = (0.982, 4.550) β* = (1.0, −1.0, 1.0)	11.908	18
		α* = (0.950, 4.501) β* = (1.0, −1.0, 1.0)	11.967	19
1.3	α0 = (0.520, 2.730) β0 = (0.80, −0.80, 0.40)	α* = (0.982, 4.550) β* = (1.0, −1.0, 1.0)	11.908	24
		α* = (0.950, 4.528) β* = (1.0, −1.0, 1.0)	11.944	40
1.4	α0 = (0.280, 3.260) β0 = (0.26, −0.40, 0.32)	α* = (0.982, 4.550) β* = (1.0, −1.0, 1.0)	11.908	23
		α* = (0.950, 4.512) β* = (1.0, −1.0, 1.0)	11.961	22

and

Table 2. Numerical results of solving test problem for initial number of switchings set to 4.

Case	$\mathit{v}$0	$\mathit{v}$*	$\mathit{J}$*	Niter
2.1	α0 = (0.620, 0.780, 2.260, 3.760) β0 = (0.400, 0.500, −0.600, −0.800, 0.800)	α* = (0.561, 0.982, 2.156, 4.550) β* = (1.000, 1.000, −1.000, −1.000, 1.000)	11.908	27
		α* = (0.949, 0.950, 1.858, 4.512) β* = (1.000, 0.584, −1.000, −1.000, 1.000)	11.961	9
2.2	α0 = (0.110, 0.760, 0.810, 4.760) β0 = (0.450, 0.850, −0.700, −0.800, 0.800)	α* = (10-17, 0.982, 0.982, 4.550) β* = (0.450, 1.000, −0.687, −1.000, 1.000)	11.908	21
		α* = (0.950, 0.950, 0.950, 4.509) β* = (1.000, 0.835, −0.657, −1.000, 1.000)	11.965	3

, where $\mathit{v}$⁰ is a guess point, $\mathit{v}$* is the solution point.

The initial number of switchings was set to two (cases 1.1–1.4) and four (cases 2.1–2.2). Both tables show the comparison of the results obtained by applying Newton’s method (highlighted in bold) with the results obtained in [27]. Optimal values of the objective functional [27] were recalculated by solving the direct Cauchy problem with the use of the Runge–Kutta method. Initial and optimal values of the trajectory $(x_1\left(t\right), x_2\left(t\right))$ and control function $u\left(t\right)$ for cases 2.1 and 2.2 are shown in Figure 1, Figure 2 and Figure 3.

Let us analyze the obtained numerical results. When the number of switchings is initially set to two (cases 1.1–1.4), the same solution is obtained for all the guess points.

As for the case of four initial switchings, we have two technically different results. However, using the algorithm proposed in Section 4 for case 2.1, we can merge intervals [0, 0.561] and [0.561, 0.982] because the control action equals 1 at both of them. The same holds for intervals [0.982, 2.156] and [2.156, 4.5]. This results in the solution obtained for cases 1.1–1.4.

Now, let us analyze the solution obtained in case 2.2. The intervals where control actions equal 0.450 and −0.687 are negligibly small (see Figure 3, the control action is linked to the right axis), and they can be removed from consideration. So, the solution obtained in case 2.2 also coincides with that in cases 1.1–1.4.

A comparison with the results provided by first-order methods [27] shows that the second-order method achieves the minimum value of the objective functional, which is less than in [27]. This refinement is the result of the more exact values of the optimal switchings.

6. Conclusions

This paper provides formulas for the use of second-order methods for solving optimal control problems in parametrically specified classes of control actions. In the problem, both the switching times of the control parameters and the parameters themselves are being optimized. Despite the awkwardness of the obtained formulas, their use is efficient when it is necessary to repeatedly solve a specific control problem for different values of the parameters involved in the problem. Such parameters can be the values of the coefficients involved in differential Equation (1) or the values of the initial conditions.

This paper presents the results of numerical experiments and a comparison with the results obtained by first-order optimal control methods, including those given in other works.