The Meaning and Accuracy of the Improving Functions in the Solution of the CBQR by Krotov’s Method

Abstract

A new solution to the continuous-time bilinear quadratic regulator optimal control problem (CBQR) was recently developed using Krotov’s Method. This paper provides two theoretical results related to the properties of that solution. The first discusses the equivalent representation of the cost-to-go performance index. The second one breaks down this equivalence into smaller identities referencing the components of the performance index. The paper shows how these results can be used to verify the numerical accuracy of the computed solution. Additionally, the meaning of the improving function and the equivalent representation, which are the main elements in the discussed CBQR’s solution, are explained according to the derived notions. A numerical example of structural control application exemplifies the significance of these results and how they can be applied to a specific CBQR problem.

1. Introduction

The challenge of “choosing the best path” is a prevalent problem that has a significant presence in applied science and technology [1]. Not surprisingly, its solutions find their way into a wide range of applications, including automatic control of systems [2]. For the latter, the problem is addressed in the framework of optimal control theory. The fundamental objective in this field is to maximize the return from, or minimize the cost of, the operation of physical [3,4], biological [5,6], social [7,8,9], economic processes [10,11], etc.

Many studies have been conducted over the years in optimal control theory, yielding diverse results for many types of problems, and the research is still ongoing. An important tool for solving various optimal control problems is Krotov’s method [2], sometimes referred to by its original name: “a global method of successive improvements of control”. The method, which stems from the fundamental extension principle [2], is a successive algorithm aimed at the computerized solution of optimal control problems. It is a well-known instrument for constructing optimal control for quantum systems [12,13]. Additionally, its efficiency was demonstrated for a class of structural control problems [14,15,16], iron and steel manufacturing processes [17], and biological systems [18].

However, it should be noted that even though Krotov’s method furnishes a rigorous methodology for solving optimal control problems, its formulation is general and requires additional effort when addressing a specific issue. In order to apply the method to a given optimal control problem, one should solve another problem. That is, one should reshape the given performance index to a form that points out a clear way to obtain a better, improved process and the feedback generating it [2]. The reshaped form of the performance index is called an equivalent representation, and the key object for obtaining it is an object called an improving function [2].

One of the recent studies that utilized Krotov’s method deals with the solution for the continuous-bilinear quadratic regulator problem (CBQR) [19]. Here, the aim is to support that solution by furnishing some interesting theoretical comments on the equivalent representation related to this problem. These results can be used to verify the numerical accuracy of the computed improving function and the solution obtained by it. Additionally, in the same context, they shed light on the meaning of the elements of the equivalent representation and the improving function. Section 2 provides some background needed for understanding the main results. The latter are presented in Section 3. Finally, a numerical example is given to illustrate the significance of the results and how they can be used to verify the numerical accuracy of an improving function obtained by the method suggested in [19].

2. Methods

In order to put things in context and facilitate the main derivations, it is beneficial to review several notions and theories.

$\mathbf{x}:\mathbb{R}\to {\mathbb{R}}^n$ denotes a state trajectory, and $\mathbf{u}:\mathbb{R}\to {\mathbb{R}}^{n_u}$ denotes a control trajectory. Here, $\mathbf{x}\left(t\right)$ is an intersection of $\mathbf{x}$ at t. That is, $\mathbf{x}\left(t\right)$ is a specific vector in ${\mathbb{R}}^n$ that expresses the system’s state at t, whereas $\mathbf{x}$ refers to the entire trajectory. ${\mathcal{U}}_i$ is a set of control signals admissible to the i-th control device.

The CBQR is an optimal control problem that consists of a bilinear system [20] and a quadratic performance index of the form:

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill \dot{\mathbf{x}}\left(t\right)=& \mathbf{A}\left(t\right)\mathbf{x}\left(t\right)+\mathbf{B}\left(t\right)\mathbf{u}\left(t\right)+\left\{\mathbf{u}\mathbf{N}\right(t\left)\right\}\mathbf{x}\left(t\right)+\mathbf{g}\left(t\right)\hfill \\ & \mathbf{x}\left(0\right),\mathbf{u}\in \mathcal{U},\forall t\in (0,t_f)\hfill \end{array}\end{array}$$

(1)

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill J(\mathbf{x},\mathbf{u})=& \frac{1}{2}\underset{0}{\overset{t_f}{\int }}\mathbf{x}{\left(t\right)}^T\mathbf{Q}\left(t\right)\mathbf{x}\left(t\right)+\mathbf{u}{\left(t\right)}^T\mathbf{R}\left(t\right)\mathbf{u}\left(t\right)\mathrm{d}t+\frac{1}{2}\mathbf{x}{\left(t_f\right)}^T\mathbf{H}\mathbf{x}\left(t_f\right)\hfill \end{array}\end{array}$$

(2)

Here, $\left\{\mathbf{u}\mathbf{N}\left(t\right)\right\}\triangleq {\sum }_{i=1}^{n_u}{u}_i\left(t\right){\mathbf{N}}_i\left(t\right)$; $\mathbf{A},{\mathbf{N}}_i:\mathbb{R}\to {\mathbb{R}}^{n\times n}$ and $\mathbf{B}:\mathbb{R}\to {\mathbb{R}}^{n\times n_u}$. $\mathbf{g}:\mathbb{R}\to {\mathbb{R}}^n$ is a trajectory of external excitations; $\mathbf{Q}:\mathbb{R}\to {\mathbb{R}}^{n\times n}$ such that $\mathbf{Q}\left(t\right)\ge 0$; $\mathbf{R}:\mathbb{R}\to {\mathbb{R}}^{n_u\times n_u}$ such that $\mathbf{R}\left(t\right)\ge 0$ and $\mathbf{H}\ge 0$. A pair $(\mathbf{x},\mathbf{u})$ that satisfies Equation (1) is called an admissible process. The set $\mathcal{X}\subseteq [\mathbb{R}\to {\mathbb{R}}^n]$ comprises state trajectories, reachable from $\mathcal{U}$ and the specified $\mathbf{x}\left(0\right)$. The solution to this problem is required in the form of a state feedback that synthesizes an optimal admissible process, i.e., one that minimizes J.

A successive method was utilized in order to solve this problem [19], that is, to obtain an admissible process that minimizes J. The method, which is named Krotov’s method after its founder Prof. V. F. Krotov [2], furnishes the solution as a sequence called an improving sequence. Krotov defined this sequence to be a sequence of admissible processes—$\left\{({\mathbf{x}}_k,{\mathbf{u}}_k)\right\}$—such that $J({\mathbf{x}}_k,{\mathbf{u}}_k)\ge J({\mathbf{x}}_{k+1},{\mathbf{u}}_{k+1})$; i.e., the performance of each element in the improving sequence is better than or equal to its former. In addition to this monotonous improvement, the method has the advantages of not being limited to small variations in $\mathbf{u}$ and obtaining the solution in the form of feedback [2]. A succinct introduction to this method is given below.

Consider a class of optimal control problems defined by a state equation, set of admissible control trajectories, and performance index:

$$\begin{array}{cc}\hfill \dot{\mathbf{x}}\left(t\right)=& \mathbf{f}(t,\mathbf{x}\left(t\right),\mathbf{u}\left(t\right));\mathbf{x}\left(0\right),\forall t\in (0,t_f);\mathbf{u}\in \mathcal{U}\hfill \end{array}$$

(3)

$$\begin{array}{cc}\hfill J(\mathbf{x},\mathbf{u})=& l_f\left(\mathbf{x}\left(t_f\right)\right)+\underset{0}{\overset{t_f}{\int }}l(t,\mathbf{x}\left(t\right),\mathbf{u}\left(t\right))\mathrm{d}t\hfill \end{array}$$

(4)

Here, $\mathbf{f}:\mathbb{R}\times {\mathbb{R}}^n\times {\mathbb{R}}^{n_u}\to {\mathbb{R}}^n$, $l:\mathbb{R}\times {\mathbb{R}}^n\times {\mathbb{R}}^{n_u}\to \mathbb{R}$ and $l_f:{\mathbb{R}}^n\to \mathbb{R}$. The goal is to find an admissible $(\mathbf{x},\mathbf{u})$ that minimizes J.

One of the key concepts in Krotov’s theory is the equivalent representation [2]. It was shown that the optimal control problem can be reformulated by transforming the performance index J into an equivalent one, $J_{eq}$. The rationale behind this transformation lies in the potential for a thoughtfully selected $J_{eq}$ to simplify the solution process. The equivalent representation plays a crucial role in various results put forth by Krotov, notably in Krotov’s method [2]. The subsequent theorem introduces the relevant equivalent representation for our context. $\mathsf{\xi }$ and $\mathsf{\nu }$ denote vectors in ${\mathbb{R}}^n$ and ${\mathbb{R}}^{n_u}$, respectively.

Theorem 1

([2]). Let $q:\mathbb{R}\times {\mathbb{R}}^n\to \mathbb{R}$ be a piecewise-smooth function, upon which the next functions and performance index are constructed:

$$\begin{array}{cc}\hfill s(t,\mathsf{\xi },\mathsf{\nu })\triangleq & q_t(t,\mathsf{\xi })+q_{\mathbf{x}}(t,\mathsf{\xi })\mathbf{f}(t,\mathsf{\xi },\mathsf{\nu })+l(t,\mathsf{\xi },\mathsf{\nu })\hfill \end{array}$$

(5)

$$\begin{array}{cc}\hfill s_f\left(\mathsf{\xi }\right)\triangleq & l_f\left(\mathsf{\xi }\right)-q(t_f,\mathsf{\xi })\hfill \end{array}$$

(6)

$$\begin{array}{cc}\hfill J_{eq}(\mathbf{x},\mathbf{u})\triangleq & s_f\left(\mathbf{x}\left(t_f\right)\right)+q(0,\mathbf{x}\left(0\right))+\underset{0}{\overset{t_f}{\int }}s(t,\mathbf{x}\left(t\right),\mathbf{u}\left(t\right))\mathrm{d}t\hfill \end{array}$$

(7)

If $(\mathbf{x},\mathbf{u})$ is an admissible process, then $J(\mathbf{x},\mathbf{u})=J_{eq}(\mathbf{x},\mathbf{u})$.

Hence, the challenge is to find a q that forms a beneficial $J_{eq}$. In the following theorem, Krotov points out a way of finding such a q. The theorem states the properties of q and an improving feedback, $\widehat{\mathbf{u}}$, which allow for the improvement of a given admissible process.

Theorem 2

([2]). Let a given $({\mathbf{x}}_1,{\mathbf{u}}_1)$ be admissible, and let $q:\mathbb{R}\times {\mathbb{R}}^n\to \mathbb{R}$. If the following statements hold:

1. 

q grants s and $s_f$ the property:

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill s(t,{\mathbf{x}}_1\left(t\right),{\mathbf{u}}_1\left(t\right))=& \underset{\mathsf{\xi }\in \mathcal{X}\left(t\right)}{max}s(t,\mathsf{\xi },{\mathbf{u}}_1\left(t\right));\forall t\in (0,t_f)\hfill \\ \hfill s_f\left({\mathbf{x}}_1\left(t_f\right)\right)=& \underset{\mathsf{\xi }\in \mathcal{X}\left(t_f\right)}{max}{s}_f\left(\mathsf{\xi }\right)\hfill \end{array}\end{array}$$

(8)

2. 

$\widehat{\mathbf{u}}$ is a feedback

$$\begin{array}{c}\hfill \widehat{\mathbf{u}}(t,\mathsf{\xi })=arg\underset{\mathsf{\nu }\in \mathcal{U}\left(t\right)}{min}s(t,\mathsf{\xi },\mathsf{\nu })\end{array}$$

(9)

for all $t\in [0,t_f],\mathsf{\xi }\in \mathcal{X}\left(t\right)$.

3. 

${\mathbf{x}}_2$ is a state trajectory that solves:

$$\begin{array}{c}{\dot{\mathbf{x}}}_2\left(t\right)=\mathbf{f}(t,{\mathbf{x}}_2\left(t\right),\widehat{\mathbf{u}}(t,{\mathbf{x}}_2\left(t\right)));{\mathbf{x}}_2\left(0\right)=\mathbf{x}\left(0\right),\hfill \end{array}$$

(10)

at any $\forall t\in (0,t_f)$, and ${\mathbf{u}}_2$ is a control trajectory such that ${\mathbf{u}}_2\left(t\right)=\widehat{\mathbf{u}}(t,{\mathbf{x}}_2\left(t\right))$,

then $({\mathbf{x}}_2,{\mathbf{u}}_2)$ is an improved process.

A q that meets the requirements listed in the above theorem is called an improving function. By repeating the process improvement over and over, an improving sequence is obtained.

However, while the process of successive improvement proves highly beneficial, this alone does not guarantee optimality. Additional considerations are necessary to ensure that the obtained solution is indeed optimal. For instance, even assuming convergence of the improving sequence, the optimality of its limit process remains uncertain. In the case of an optimum, we anticipatethat it will satisfy conditions of optimality, such as Pontryagin’s minimum principle. Krotov has also addressed this question.

Assume that at some step k of the algorithm, the gradients $q_{k\mathbf{x}}$ and $q_{(k-1)\mathbf{x}}$ are equal for all $t\in (0,t_f)$, and $q_k$ grants the gradient of $s_k$ the following property

$$s_{k\mathbf{x}}(t,{\mathbf{x}}_k\left(t\right),{\mathbf{u}}_k\left(t\right))=0;\forall t\in (0,t_f)$$

at the process $({\mathbf{x}}_k,{\mathbf{u}}_k)$. Krotov showed that, in this case, $({\mathbf{x}}_k,{\mathbf{u}}_k)$ satisfies Pontryagin’s minimum principle with the costate $q_{k\mathbf{x}}(t,{\mathbf{x}}_k\left(t\right))$ [2]. In this context, Krotov’s method provides a convenient instrument for finding a solution that satisfies Pontryagin’s minimum principle, rather than solving it directly. Furthermore, while solving problems directly via Pontryagin’s minimum principle typically yields an open-loop control trajectory, Krotov’s method offers a solution in a feedback form [21].

The major difficulty faced by a control designer intending to apply Krotov’s method is formulating an improving function suitable for the addressed optimal control problem. For the CBQR problem, given an admissible process, $(\mathbf{x},\mathbf{u})$, an improving function can be formulated as $q(t,\mathsf{\xi })={\mathsf{\xi }}^T\mathbf{P}\left(t\right)\mathsf{\xi }+{\mathsf{\xi }}^T\mathbf{p}\left(t\right)$ [19]. Here, $\mathbf{P}:\mathbb{R}\to {\mathbb{R}}^{n\times n}$ is the solution of the following differential Lyapunov equation [22]:

$$\begin{array}{c}\hfill \dot{\mathbf{P}}\left(t\right)=-\mathbf{P}\left(t\right)(\mathbf{A}\left(t\right)+\left\{\mathbf{u}\mathbf{N}\left(t\right)\right\})-{(\mathbf{A}\left(t\right)+\left\{\mathbf{u}\mathbf{N}\left(t\right)\right\})}^T\mathbf{P}\left(t\right)-\mathbf{Q}\left(t\right)\end{array}$$

(11)

to $\mathbf{P}\left(t_f\right)=\mathbf{H}$, and $\mathbf{p}:\mathbb{R}\to {\mathbb{R}}^n$ is the solution of:

$$\begin{array}{c}\hfill \dot{\mathbf{p}}\left(t\right)=-{\left(\mathbf{A}\left(t\right)+\left\{\mathbf{u}\mathbf{N}\right(t\left)\right\}\right)}^T\mathbf{p}\left(t\right)-\mathbf{P}\left(t\right)(\mathbf{B}\left(t\right)\mathbf{u}\left(t\right)+\mathbf{g}\left(t\right))\end{array}$$

(12)

to $\mathbf{p}\left(t_f\right)=\mathbf{0}$. In these equations, $\mathbf{u}$ is the control trajectory specified by the given admissible process.

As mentioned above, the improving function q is associated with some $J_{eq}$, whose importance lies in the fact that it underpins the process improvement. This relation, between the improving function and equivalent representation, is discussed below in Section 3 in the context of the CBQR problem.

3. Results

The equivalent representation of the performance index refers to the entire time domain $[0,t_f]$. However, in some cases, there is an interest in the performance of the system over a sub-interval, $[t_1,t_f]\subseteq [0,t_f]$, also known as the cost-to-go. Following Theorem 1 one can easily obtain the equivalent representation of the cost-to-go. The next corollary provides an equivalent representation of the cost-to-go of a given admissible process, $(\mathbf{x},\mathbf{u})$.

Corollary 1.

Let $\overline{J}:\mathcal{X}\times \mathcal{U}\times \mathbb{R}\to \mathbb{R}$ and ${\overline{J}}_{eq}:\mathcal{X}\times \mathcal{U}\times \mathbb{R}\to \mathbb{R}$ be the functionals:

$$\begin{array}{cc}\hfill \overline{J}(\mathbf{x},\mathbf{u},t_1)\triangleq & \underset{t_1}{\overset{t_f}{\int }}l(t,\mathbf{x}\left(t\right),\mathbf{u}\left(t\right))\mathrm{d}t+l_f\left(\mathbf{x}\left(t_f\right)\right)\hfill \end{array}$$

(13)

$$\begin{array}{cc}\hfill {\overline{J}}_{eq}(\mathbf{x},\mathbf{u},t_1)\triangleq & q(t_1,\mathbf{x}\left(t_1\right))+\underset{t_1}{\overset{t_f}{\int }}s(t,\mathbf{x}\left(t\right),\mathbf{u}\left(t\right))\mathrm{d}t+s_f\left(\mathbf{x}\left(t_f\right)\right)\hfill \end{array}$$

(14)

where $t_1\in [0,t_f]$ and $(\mathbf{x},\mathbf{u})$ is admissible. Then, $\overline{J}(\mathbf{x},\mathbf{u},t_1)={\overline{J}}_{eq}(\mathbf{x},\mathbf{u},t_1)$ for any $0\le t_1\le t_f$.

The proof is similar to Krotov’s proof to Theorem 1, differing only by setting the time domain to $[t_1,t_f]$ rather than $[0,t_f]$. It is given below for the reader’s convenience.

Proof. 

Let the hypothesis hold. By substituting Equations (5) and (6) into ${\overline{J}}_{eq}-\overline{J}$, we obtain:

$$\begin{array}{cc}\hfill {\overline{J}}_{eq}(\mathbf{x},\mathbf{u},t_1)-\overline{J}(\mathbf{x},\mathbf{u},t_1)=& q(t_1,\mathbf{x}\left(t_1\right))+(l_f\left(\mathbf{x}\left(t_f\right)\right)-q(t_f,\mathbf{x}\left(t_f\right))-l_f\left(\mathbf{x}\left(t_f\right)\right)\hfill \\ & +\underset{t_1}{\overset{t_f}{\int }}\left(q_t(t,\mathbf{x}\left(t\right))+q_{\mathbf{x}}(t,\mathbf{x}\left(t\right))\mathbf{f}(t,\mathbf{x}\left(t\right),\mathbf{u}\left(t\right))\right)\mathrm{d}t\hfill \end{array}$$

As $(\mathbf{x},\mathbf{u})$ is an admissible process, $\dot{\mathbf{x}}\left(t\right)=\mathbf{f}(t,\mathbf{x}\left(t\right),\mathbf{u}\left(t\right))$ holds, leading to:

$$\begin{array}{cc}\hfill {\overline{J}}_{eq}(\mathbf{x},\mathbf{u},t_1)-\overline{J}(\mathbf{x},\mathbf{u},t_1)=& q(t_1,\mathbf{x}\left(t_1\right))-q(t_f,\mathbf{x}\left(t_f\right))\hfill \\ & +\underset{t_1}{\overset{t_f}{\int }}\left(q_t(t,\mathbf{x}\left(t\right))+q_{\mathbf{x}}(t,\mathbf{x}\left(t\right))\dot{\mathbf{x}}\left(t\right)\right)\mathrm{d}t\hfill \\ & =q(t_1,\mathbf{x}\left(t_1\right))-q(t_f,\mathbf{x}\left(t_f\right))+\underset{t_1}{\overset{t_f}{\int }}\mathrm{d}q(t,\mathbf{x}\left(t\right))=0\hfill \end{array}$$

by the virtue of Newton–Leibnitz formula. □

Remark 1.

 

${\overline{J}}_{eq}$ can be used for evaluating performance over sub-trajectories by:

$$\begin{array}{cc}\hfill \underset{t_1}{\overset{t_2}{\int }}l(t,\mathbf{x}\left(t\right),& \mathbf{u}\left(t\right))\mathrm{d}t=\overline{J}(\mathbf{x},\mathbf{u},t_2)-\overline{J}(\mathbf{x},\mathbf{u},t_1)\hfill \end{array}$$

(15)

$$\begin{array}{cc}\hfill =& {\overline{J}}_{eq}(\mathbf{x},\mathbf{u},t_2)-{\overline{J}}_{eq}(\mathbf{x},\mathbf{u},t_1)\hfill \end{array}$$

(16)

$$\begin{array}{cc}\hfill =& q(t_1,\mathbf{x}\left(t_1\right))-q(t_2,\mathbf{x}\left(t_2\right))+\underset{t_1}{\overset{t_2}{\int }}s(t,\mathbf{x}\left(t\right),\mathbf{u}\left(t\right))\mathrm{d}t\hfill \end{array}$$

(17)

Assume that $(\mathbf{x},\mathbf{u})$ is an admissible process. By substituting q into s and then into ${\overline{J}}_{eq}$ (Equation (14)), we obtain

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill {\overline{J}}_{eq}(\mathbf{x},\mathbf{u},t_1)& =\frac{1}{2}\mathbf{x}{\left(t_1\right)}^T\mathbf{P}\left(t_1\right)\mathbf{x}\left(t_1\right)+\mathbf{p}{\left(t_1\right)}^T\mathbf{x}\left(t_1\right)\hfill \\ & +\underset{t_1}{\overset{t_f}{\int }}\mathbf{p}{\left(t\right)}^T(\mathbf{B}\left(t\right)\mathbf{u}\left(t\right)+\mathbf{g}\left(t\right))+\frac{1}{2}\mathbf{u}{\left(t\right)}^T\mathbf{R}\left(t\right)\mathbf{u}\left(t\right)\mathrm{d}t\hfill \end{array}\end{array}$$

(18)

Corollary 1 establishes the overall equivalence of J and $J_{eq}$ in a general sense, without delving into their components. The subsequent theorem addresses how this equivalence is reflected through the components of J and $J_{eq}$ for a linear system with an external excitation and no control input, namely:

$$\begin{array}{cc}\hfill \mathbf{f}(t,\mathbf{x}(t),\mathbf{u}(t\left)\right)=& {\mathbf{f}}^{\prime }(t,\mathbf{x}\left(t\right))\triangleq \mathbf{A}\left(t\right)\mathbf{x}\left(t\right)+\mathbf{g}\left(t\right);\mathbf{x}\left(0\right)\hfill \\ \hfill l(t,\mathbf{x}(t),\mathbf{u}(t\left)\right)=& l^{\prime }(t,\mathbf{x}\left(t\right))\triangleq \mathbf{x}{\left(t\right)}^T\mathbf{Q}\left(t\right)\mathbf{x}\left(t\right);l_f\left(\mathbf{x}\left(t_f\right)\right)=\frac{1}{2}\mathbf{x}{\left(t_f\right)}^T\mathbf{H}\mathbf{x}\left(t_f\right)\hfill \end{array}$$

In brief, the following theorem highlights specific properties of $\mathbf{P}$ and $\mathbf{p}$, which are the solutions to Equations (11) and (12), respectively. These properties allow us to deconstruct the overall equivalence, as described in Corollary 1, into smaller identities. The theorem also provides a tool for verifying the accuracy of $\mathbf{P}$ and $\mathbf{p}$ and offers a deeper understanding of their role in $J_{eq}$.

It is worth noting that, despite the differences between the CBQR and the aforementioned system, the result remains relevant to the CBQR case, as elucidated later in Section 4.

Theorem 3.

Let $\mathbf{g}:\mathbb{R}\to {\mathbb{R}}^n$; $\mathbf{A},\mathbf{Q}:\mathbb{R}\to {\mathbb{R}}^{n\times n}$, where $\mathbf{Q}\left(t\right)\ge 0$. If ${\mathbf{x}}_h,{\mathbf{x}}_p,\mathbf{p}:\mathbb{R}\to {\mathbb{R}}^n$ and $\mathbf{P}:\mathbb{R}\to {\mathbb{R}}^{n\times n}$ satisfy the linear ODEs

$$\begin{array}{ccc}{\dot{\mathbf{x}}}_h\left(t\right)=\hfill & \mathbf{A}\left(t\right){\mathbf{x}}_h\left(t\right);\hfill & \hfill {\mathbf{x}}_h\left(t_1\right)=\mathbf{x}\left(t_1\right)\end{array}$$

(19)

$$\begin{array}{ccc}{\dot{\mathbf{x}}}_p\left(t\right)=\hfill & \mathbf{A}\left(t\right){\mathbf{x}}_p\left(t\right)+\mathbf{g}\left(t\right);\hfill & \hfill {\mathbf{x}}_p\left(t_1\right)=\mathbf{0}\end{array}$$

(20)

$$\begin{array}{ccc}\dot{\mathbf{P}}\left(t\right)=\hfill & -\mathbf{P}\left(t\right)\mathbf{A}\left(t\right)-\mathbf{A}{\left(t\right)}^T\mathbf{P}\left(t\right)-\mathbf{Q}\left(t\right);\hfill & \hfill \mathbf{P}\left(t_f\right)=\mathbf{H}\end{array}$$

(21)

$$\begin{array}{ccc}\dot{\mathbf{p}}\left(t\right)=\hfill & -\mathbf{A}{\left(t\right)}^T\mathbf{p}\left(t\right)-\mathbf{P}\left(t\right)\mathbf{g}\left(t\right);\hfill & \hfill \mathbf{p}\left(t_f\right)=\mathbf{0}\end{array}$$

(22)

for $t\in (t_1,t_f)$, then:

(a) 

$\frac{1}{2}\mathbf{x}{\left(t_1\right)}^T\mathbf{P}\left(t_1\right)\mathbf{x}\left(t_1\right)=\frac{1}{2}\underset{t_1}{\overset{t_f}{\int }}{\mathbf{x}}_h{\left(t\right)}^T\mathbf{Q}\left(t\right){\mathbf{x}}_h\left(t\right)\mathrm{d}t+\frac{1}{2}{\mathbf{x}}_h{\left(t_f\right)}^T\mathbf{H}{\mathbf{x}}_h\left(t_f\right)$

(b) 

$\underset{t_1}{\overset{t_f}{\int }}\mathbf{p}{\left(t\right)}^T\mathbf{g}\left(t\right)\mathrm{d}t=\frac{1}{2}\underset{t_1}{\overset{t_f}{\int }}{\mathbf{x}}_p{\left(t\right)}^T\mathbf{Q}\left(t\right){\mathbf{x}}_p\left(t\right)\mathrm{d}t+\frac{1}{2}{\mathbf{x}}_p{\left(t_f\right)}^T\mathbf{H}{\mathbf{x}}_p\left(t_f\right)$

(c) 

$\mathbf{x}{\left(t_1\right)}^T\mathbf{p}\left(t_1\right)=\underset{t_1}{\overset{t_f}{\int }}{\mathbf{x}}_h{\left(t\right)}^T\mathbf{Q}\left(t\right){\mathbf{x}}_p\left(t\right)\mathrm{d}t+{\mathbf{x}}_h{\left(t_f\right)}^T\mathbf{H}{\mathbf{x}}_p\left(t_f\right)$

Proof. 

Consider Equations (1) and (2) over $[t_1,t_f]$, and let $\mathbf{u}\triangleq \mathbf{0}$. Consequently, Equation (1) becomes:

$$\begin{array}{cc}\hfill \dot{\mathbf{x}}\left(t\right)=& \mathbf{A}\left(t\right)\mathbf{x}\left(t\right)+\mathbf{g}\left(t\right);t\in (t_1,t_f),\mathbf{x}\left(t_1\right)\hfill \end{array}$$

$\mathbf{x}$’s solution consists of homogeneous and particular solutions satisfying Equations (19) and (20), respectively. In addition, Equations (11) and (12) become Equations (21) and (22), repectively.

As $(\mathbf{x},\mathbf{0})$ is admissible and by Corollary 1:

$$\begin{array}{cc}\hfill \frac{1}{2}\underset{t_1}{\overset{t_f}{\int }}\mathbf{x}{\left(t\right)}^T\mathbf{Q}\left(t\right)\mathbf{x}\left(t\right)\mathrm{d}t& +\frac{1}{2}\mathbf{x}{\left(t_f\right)}^T\mathbf{H}\mathbf{x}\left(t_f\right)=\hfill \\ & \underset{t_1}{\overset{t_f}{\int }}\mathbf{p}{\left(t\right)}^T\mathbf{g}\left(t\right)\mathrm{d}t+\frac{1}{2}\mathbf{x}{\left(t_1\right)}^T\mathbf{P}\left(t_1\right)\mathbf{x}\left(t_1\right)+\mathbf{x}{\left(t_1\right)}^T\mathbf{p}\left(t_1\right)\hfill \end{array}$$

After substituting $\mathbf{x}={\mathbf{x}}_h+{\mathbf{x}}_p$:

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill \frac{1}{2}\underset{t_1}{\overset{t_f}{\int }}{\left({\mathbf{x}}_h\left(t\right)+{\mathbf{x}}_p\left(t\right)\right)}^T& \mathbf{Q}\left(t\right)\left({\mathbf{x}}_h\left(t\right)+{\mathbf{x}}_p\left(t\right)\right)\mathrm{d}t\hfill \\ & +\frac{1}{2}{\left({\mathbf{x}}_h\left(t_f\right)+{\mathbf{x}}_p\left(t_f\right)\right)}^T\mathbf{H}\left({\mathbf{x}}_h\left(t_f\right)+{\mathbf{x}}_p\left(t_f\right)\right)\hfill \\ \hfill =\underset{t_1}{\overset{t_f}{\int }}& \mathbf{p}{\left(t\right)}^T\mathbf{g}\left(t\right)\mathrm{d}t+\frac{1}{2}\mathbf{x}{\left(t_1\right)}^T\mathbf{P}\left(t_1\right)\mathbf{x}\left(t_1\right)+\mathbf{x}{\left(t_1\right)}^T\mathbf{p}\left(t_1\right)\hfill \end{array}\end{array}$$

(23)

(a)

Let $\mathbf{g}\equiv \mathbf{0}$. Hence, ${\mathbf{x}}_p=\mathbf{0}$, $\mathbf{p}=\mathbf{0}$, and

$$\begin{array}{cc}\hfill \frac{1}{2}\underset{t_1}{\overset{t_f}{\int }}{\mathbf{x}}_h{\left(t\right)}^T\mathbf{Q}\left(t\right){\mathbf{x}}_h\left(t\right)\mathrm{d}t& +\frac{1}{2}{\mathbf{x}}_h{\left(t_f\right)}^T\mathbf{H}{\mathbf{x}}_h\left(t_f\right)=\frac{1}{2}\mathbf{x}{\left(t_1\right)}^T\mathbf{P}\left(t_1\right)\mathbf{x}\left(t_1\right)\hfill \end{array}$$

(b)

Let $\mathbf{x}\left(t_1\right)=\mathbf{0}$. Hence, ${\mathbf{x}}_h=\mathbf{0}$ and

$$\begin{array}{cc}\hfill \frac{1}{2}\underset{t_1}{\overset{t_f}{\int }}{\mathbf{x}}_p{\left(t\right)}^T\mathbf{Q}\left(t\right){\mathbf{x}}_p\left(t\right)\mathrm{d}t& +\frac{1}{2}{\mathbf{x}}_p{\left(t_f\right)}^T\mathbf{H}{\mathbf{x}}_p\left(t_f\right)=\underset{t_1}{\overset{t_f}{\int }}\mathbf{p}{\left(t\right)}^T\mathbf{g}\left(t\right)\mathrm{d}t\hfill \end{array}$$

(c)

By (a), (b), and cancelling terms from Equation (23):

$$\begin{array}{cc}\hfill \underset{t_1}{\overset{t_f}{\int }}{\mathbf{x}}_h{\left(t\right)}^T\mathbf{Q}\left(t\right){\mathbf{x}}_p\left(t\right)\mathrm{d}t& +{\mathbf{x}}_h{\left(t_f\right)}^T\mathbf{H}{\mathbf{x}}_p\left(t_f\right)=\mathbf{x}{\left(t_1\right)}^T\mathbf{p}\left(t_1\right)\hfill \end{array}$$

□

4. Discussion

The fact described in Corollary 1 proves to be useful in certain circumstances. As mentioned earlier, the method proposed for solving the CBQR problem [19] relies on process improvement. However, even though the process is rigorously derived, there is a practical obstacle to its application. The solutions for Equations (11) and (12) are usually obtained through numerical integration methods. In practice, numerical computation errors are always an issue when differential equations are solved numerically [23,24]. Numerical observations show that a successful improvement of a given process—$({\mathbf{x}}_k,{\mathbf{u}}_k)$—is quite sensitive to the accuracy of q. This accuracy is highly affected by that of $\mathbf{P}$ and $\mathbf{p}$, which solves Equations (11) and (12). Furthermore, differential Lyapunov equations tend to be quite stiff [25,26], making the numerical accuracy issue very present. This is particularly true in the case of Equation (11) and when $\mathbf{u}$ is rapidly varying or discontinuous, which is a common situation in many practical optimal control problems [2,16,27,28,29]. Therefore, dealing with numerical errors becomes a common practice in the improvement process. As a result, when numerical errors do exist in $\mathbf{P}$ and $\mathbf{p}$, the structure of q becomes flawed and inaccurate, and the improvement might fail. A verification of q’s numerical accuracy at different t’s can be helpful in such a case as it may pinpoint the root of the problem and, consequently, its solution.

First, note that $\mathbf{P}\left(t_f\right)$ and $\mathbf{p}\left(t_f\right)$ are specified. Hence they are accurately known, and $q(t_f,\mathsf{\xi })$ is accurately known as well. Hence, q’s accuracy at $t_f$ is not of concern. As for q at other t’s, ${\overline{J}}_{eq}$ can be used. By taking advantage of the independence of $\overline{J}$ on q, one can verify q’s accuracy at a certain $t_1\in [0,t_f)$ by examining the difference $|{\overline{J}}_{eq}({\mathbf{x}}_k,{\mathbf{u}}_k,t_1)-\overline{J}({\mathbf{x}}_k,{\mathbf{u}}_k,t_1)|$. Clearly, higher accuracy will be reflected in a smaller difference. This tool can be utilized to identify the sections of $\mathbf{u}$ that contribute to the numerical issues, indicating where additional computational efforts should be focused.

Theorem 3 can be instrumental for a more specific verification of the accuracy of $\mathbf{P}$ and $\mathbf{p}$. Since the right-hand side of expressions (a)–(c) in Theorem 3 depends solely on $\mathbf{x}$, it can be computed and compared to the left-hand side of those equations. This allows us to verify the accuracy of the numerically computed $\mathbf{P}$ and $\mathbf{p}$ at any given t, as follows.

Throughout the improving steps, the control trajectory undergoes repeated alterations. However, at each step, the previously computed process, say $(\mathbf{x},\mathbf{u})$, serves as a fixed starting point. Consequently, at a given step, we treat an admissible control trajectory, $\mathbf{u}$, as a specified input. We can now write the state equation for this step as:

$$\begin{array}{cc}\hfill \dot{\mathbf{x}}\left(t\right)=& \widehat{\mathbf{A}}\left(t\right)\mathbf{x}\left(t\right)+\widehat{\mathbf{g}}\left(t\right);\mathbf{x}\left(0\right),t\in (0,t_f)\hfill \end{array}$$

(24)

where $\widehat{\mathbf{A}}\left(t\right)\triangleq \mathbf{A}\left(t\right)+\left\{\mathbf{u}\mathbf{N}\left(t\right)\right\}$ and $\widehat{\mathbf{g}}\left(t\right)\triangleq \mathbf{B}\left(t\right)\mathbf{u}\left(t\right)+\mathbf{g}\left(t\right)$. Clearly, for the given process, Equation (1) is equivalent to Equation (24). As it is a linear system, the state trajectory can be decomposed into homogeneous and particular solutions. This also applies to each sub-interval $(t_1,t_f)\subseteq (0,t_f)$. Therefore, $\mathbf{x}={\mathbf{x}}_h+{\mathbf{x}}_p$ over $(t_1,t_f)$ where the homogeneous solution ${\mathbf{x}}_h$ solves

$$\begin{array}{c}\hfill {\dot{\mathbf{x}}}_h\left(t\right)=\widehat{\mathbf{A}}\left(t\right){\mathbf{x}}_h\left(t\right);t\in (t_1,t_f),{\mathbf{x}}_h\left(t_1\right)=\mathbf{x}\left(t_1\right)\end{array}$$

(25)

and the particular one, ${\mathbf{x}}_p$, solves

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill {\dot{\mathbf{x}}}_p\left(t\right)=\widehat{\mathbf{A}}\left(t\right){\mathbf{x}}_p\left(t\right)+& \widehat{\mathbf{g}}\left(t\right);t\in (t_1,t_f),{\mathbf{x}}_p\left(t_1\right)=\mathbf{0}\hfill \end{array}\end{array}$$

(26)

On the one hand, substituting this decomposition into the CBQR’s l and $l_f$ and then into Equation (13), yields:

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill \overline{J}({\mathbf{x}}_h+{\mathbf{x}}_p,\mathbf{u},t_1)=& \frac{1}{2}\underset{t_1}{\overset{t_f}{\int }}{\mathbf{x}}_h{\left(t\right)}^T\mathbf{Q}\left(t\right){\mathbf{x}}_h\left(t\right)\mathrm{d}t+\frac{1}{2}{\mathbf{x}}_h{\left(t_f\right)}^T\mathbf{H}{\mathbf{x}}_h\left(t_f\right)\hfill \\ & +\frac{1}{2}\underset{t_1}{\overset{t_f}{\int }}{\mathbf{x}}_p{\left(t\right)}^T\mathbf{Q}\left(t\right){\mathbf{x}}_p\left(t\right)\mathrm{d}t+\frac{1}{2}{\mathbf{x}}_p{\left(t_f\right)}^T\mathbf{H}{\mathbf{x}}_p\left(t_f\right)\hfill \\ & +\underset{t_1}{\overset{t_f}{\int }}{\mathbf{x}}_h{\left(t\right)}^T\mathbf{Q}\left(t\right){\mathbf{x}}_p\left(t\right)\mathrm{d}t+{\mathbf{x}}_h{\left(t_f\right)}^T\mathbf{H}{\mathbf{x}}_p\left(t_f\right)\hfill \\ & +\frac{1}{2}\underset{t_1}{\overset{t_f}{\int }}\mathbf{u}{\left(t\right)}^T\mathbf{R}\left(t\right)\mathbf{u}\left(t\right)\mathrm{d}t\hfill \end{array}\end{array}$$

(27)

On the other hand, according to Equation (18), an admissible process $(\mathbf{x},\mathbf{u})$ can be evaluated over the sub-interval $(t_1,t_f)$ by:

$$\begin{array}{cc}\hfill \overline{J}({\mathbf{x}}_h+{\mathbf{x}}_p,\mathbf{u},t_1)={\overline{J}}_{eq}(\mathbf{x},\mathbf{u},t_1)=& \frac{1}{2}\mathbf{x}{\left(t_1\right)}^T\mathbf{P}\left(t_1\right)\mathbf{x}\left(t_1\right)+\mathbf{p}{\left(t_1\right)}^T\mathbf{x}\left(t_1\right)\hfill \\ & +\underset{t_1}{\overset{t_f}{\int }}\mathbf{p}{\left(t\right)}^T\widehat{\mathbf{g}}\left(t\right)\mathrm{d}t+\underset{t_1}{\overset{t_f}{\int }}\frac{1}{2}\mathbf{u}{\left(t\right)}^T\mathbf{R}\left(t\right)\mathbf{u}\left(t\right)\mathrm{d}t\hfill \end{array}$$

where $\mathbf{P}$ and $\mathbf{p}$ are computed from Equations (11) and (12) to the given $\mathbf{u}$ and $\mathbf{g}$. Obviously, in this case, Equations (21) and (22) solved for $\widehat{\mathbf{A}}$ and $\widehat{\mathbf{g}}$ are equivalent to Equations (11) and (12). Theorem 3 reveals that each state-dependent-term on the right-hand side of Equation (27) is equal to a corresponding component of ${\overline{J}}_{eq}$. Let

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill I_h\left(\tau \right)& \triangleq \frac{1}{2}\underset{\tau }{\overset{t_f}{\int }}{\mathbf{x}}_h{\left(t\right)}^T\mathbf{Q}\left(t\right){\mathbf{x}}_h\left(t\right)\mathrm{d}t+\frac{1}{2}{\mathbf{x}}_h{\left(t_f\right)}^T\mathbf{H}{\mathbf{x}}_h\left(t_f\right)\hfill \\ \hfill I_p\left(\tau \right)& \triangleq \frac{1}{2}\underset{\tau }{\overset{t_f}{\int }}{\mathbf{x}}_p{\left(t\right)}^T\mathbf{Q}\left(t\right){\mathbf{x}}_p\left(t\right)\mathrm{d}t+\frac{1}{2}{\mathbf{x}}_p{\left(t_f\right)}^T\mathbf{H}{\mathbf{x}}_p\left(t_f\right)\hfill \\ \hfill I_{hp}\left(\tau \right)& \triangleq \underset{\tau }{\overset{t_f}{\int }}{\mathbf{x}}_h{\left(t\right)}^T\mathbf{Q}\left(t\right){\mathbf{x}}_p\left(t\right)\mathrm{d}t+{\mathbf{x}}_h{\left(t_f\right)}^T\mathbf{H}{\mathbf{x}}_p\left(t_f\right)\hfill \\ \hfill I_{eq,p}\left(\tau \right)& \triangleq \underset{\tau }{\overset{t_f}{\int }}\mathbf{p}{\left(t\right)}^T\widehat{\mathbf{g}}\left(t\right)\mathrm{d}t\hfill \end{array}\end{array}$$

(28)

The above trajectories can be used to verify the accuracy of $\mathbf{P}$ and $\mathbf{p}$; i.e., according to Theorem 3, a good accuracy should be reflected by a strong agreement in each of the pairs $(I_h,\mathbf{x}\mathbf{P}\mathbf{x})$, $(I_{hp},{\mathbf{x}}^T\mathbf{p})$, and $(I_p,I_{eq,p})$.

Finally, Theorem 3 offers a more profound insight into the meaning of the components in ${\overline{J}}_{eq}$. The identities that are introduced in the theorem reveal that:

• The term $\frac{1}{2}\mathbf{x}{\left(t_1\right)}^T\mathbf{P}\left(t_1\right)\mathbf{x}\left(t_1\right)$ evaluates the performance of the homogeneous solution.
• $\underset{t_1}{\overset{t_f}{\int }}\mathbf{p}{\left(t\right)}^T\widehat{\mathbf{g}}\left(t\right)\mathrm{d}t$ evaluates the performance of the particular solution.
• $\mathbf{p}{\left(t_1\right)}^T\mathbf{x}\left(t_1\right)$ evaluates the cross-performance of the homogeneous-particular solutions.

Additionally,

$$\begin{array}{cc}\hfill q(t_1,\mathbf{x}\left(t_1\right))=& \frac{1}{2}\mathbf{x}{\left(t_1\right)}^T\mathbf{P}\left(t_1\right)\mathbf{x}\left(t_1\right)+\mathbf{p}{\left(t_1\right)}^T\mathbf{x}\left(t_1\right)\hfill \\ \hfill =& I_h\left(t_1\right)+I_{hp}\left(t_1\right)\hfill \end{array}$$

This implies that the improving function, q, is a sum of the homogeneous solution’s performance and the cross-performance of the homogeneous-particular solutions.

5. Numerical Example

A CBQR problem that was previously introduced in [19] is used here in order to exemplify an application of the above results. Unlike the original paper, where it was used to demonstrate a practical application of the CBQR problem and its solution, here, the focus is placed on the solution’s numerical accuracy. It illustrates how the above results can be used to verify the numerical accuracy of an improving function that was obtained by the method suggested in [19]. The problem concerns a structural control problem of a two-story building subjected to external excitation and configured with two semi-active variable stiffness (SAVS) devices [30,31]. Devices of this type consist of a frame that connects adjacent floors through a hydraulic element. This element has two operation modes controlled by a servo-valve having two states: open and closed. When the valve is open, it allows almost free flow of the hydraulic fluid, and the device’s resistance to relative floors’ motion is minimal. In a closed state, the valve prevents the fluid’s flow, locks the device, and turns it into a lateral bracing element. Springs and variable dashpots were used for modeling these SAVS devices. Each variable dashpot is capable of providing either a finite or infinite damping, representing the device’s unlocked or locked state, respectively. Figure 1 depicts the dynamic scheme used for modeling the controlled structural system.

The system dynamics amount to the following bilinear state equation:

$$\begin{array}{c}\dot{\mathbf{x}}\left(t\right)=\mathbf{A}\mathbf{x}\left(t\right)+u_1\left(t\right){\mathbf{N}}_1\mathbf{x}\left(t\right)+u_2\left(t\right){\mathbf{N}}_2\mathbf{x}\left(t\right)+\mathbf{g}\left(t\right)\hfill \\ \mathbf{A}=\left[\begin{array}{cccccc}0& 0& 1& 0& 0& 0\\ 0& 0& 0& 1& 0& 0\\ -2000& 1000& -2.01& 1& 1\dot{1}0-5& -1\dot{1}0-5\\ 500& -500& 0.5& -0.51& 0& 5\dot{1}0-6\\ 0& 0& -2.5\dot{1}07& 0& 0& 0\\ 0& 0& 2.5\dot{1}07& -2.5\dot{1}07& 0& 0\end{array}\right]\hfill \\ {\mathbf{N}}_1=\mathbf{diag}(0,0,0,0,0,-5000)\hfill \\ {\mathbf{N}}_2=\mathbf{diag}(0,0,0,0,-5000,-5000)\hfill \\ \mathbf{g}\left(t\right)={\left[\begin{array}{cccccc}0& 0& -3& -3& 0& 0\end{array}\right]}^T\sin \left(16.55t\right)\hfill \end{array}$$

where the state vector $\mathbf{x}\left(t\right)=(z_1\left(t\right),z_2\left(t\right),{\dot{z}}_1\left(t\right),{\dot{z}}_2\left(t\right),w_1\left(t\right),w_2\left(t\right))$ describes the horizontal displacements and velocities in the dynamic degrees of freedom and the forces applied through the SAVS devices to the floors. $\mathbf{u}\triangleq (u_1,u_2)$ is the control input trajectory, representing the SAVS devices’ locking patterns. Here, the control policy is restricted to one of three unlocking patterns [30]: (1) both devices are unlocked, (2) only the second device is unlocked, or (3) both devices are locked. These settings lead to an admissible set of control inputs:

$$\mathbf{u}\left(t\right)\in \mathcal{U}=\left\{\left[\begin{array}{c}0\\ 1\end{array}\right],\left[\begin{array}{c}1\\ 0\end{array}\right],\left[\begin{array}{c}0\\ 0\end{array}\right]\right\}$$

Every object in this set reflects the SAVS devices’ unlocking patterns: 1, 2 and 3, respectively. Note that $\mathcal{U}$ finiteness inhibits variational methods from being used for $\mathbf{u}$. A horizontal ground acceleration signal of ${\ddot{z}}_g\left(t\right)=3sin\left(16.55t\right)$ was used to simulate a seismic excitation.

The performance evaluation accounts for inter-story drifts and control forces. It is:

$$\begin{array}{cc}\hfill J(\mathbf{x},\mathbf{u})& =(\frac{1}{2}\underset{0}{\overset{5}{\int }}{x}_1{\left(t\right)}^2\dot{1}05+{(x_2\left(t\right)-x_1\left(t\right))}^2\dot{1}05+5(x_5{\left(t\right)}^2\hfill \\ & +x_6{\left(t\right)}^2\left)\mathrm{d}t\right)+\frac{1}{2}(x_1{\left(5\right)}^2\dot{1}04+{(x_2\left(5\right)-x_1\left(5\right))}^2\dot{1}04\hfill \\ & +50(x_5{\left(5\right)}^2+x_6{\left(5\right)}^2))\hfill \end{array}$$

$\mathbf{Q}$ and $\mathbf{H}$ were constructed accordingly. It follows that $\mathbf{R}=\mathbf{0}$ as $\mathbf{u}$ has no weight in J. Additional details of the system are available in the original paper [19].

Here, that problem is revisited and discussed in the context of Corollary 1 and Theorem 3. That is, in this section, the results are exemplified by demonstrating their application in diagnosing numerical issues emerging during the solution of the above CBQR problem.

Two characteristics of this problem allude that numerical issues are likely to be involved in solving it, especially in solving the differential equations that are related to the process improving stage and the feedback synthesis. First, due to the operation’s principle of the SAVS devices, the control signals are binary. Second, the control signals alternate rapidly. Such issues indeed came up during the CBQR iterations, as follows.

Following [19], Krotov’s method was applied to the CBQR problem through MATLAB. A numerical integration algorithm based on the fourth-order Runge-Kutta method was utilized to solve the necessary differential equations. However, here, computations were carried out twice. In the first case, the integration step was set to $0.01\left[\mathrm{s}\right]$, whereas, in the second, it was set to $0.001\left[\mathrm{s}\right]$. Notably, even though the fourth-order Runge-Kutta is not recommended for stiff equations [25], it is sufficient for the point discussed in this paper. Here, the discussion revolves around validating the numerical solution’s accuracy rather than its actual accurate computation.

Fifteen iterations were executed, each one consisting of a single CBQR improvement step. As explained above, these iterations generate a sequence of processes, where each is expected to be better than the previous in terms of J.

Table 1. Peformance index values at each process. Here, $\mathsf{\Delta }{J}_i=J_i-J_{i-1}$.

	Case 1		Case 2
$\mathbf{i}$	$\mathbf{J}$	$\mathsf{\Delta }\mathbf{J}$	$\mathbf{J}$	$\mathsf{\Delta }\mathbf{J}$
0	$4.59\times {10}^{14}$	-	$4.59\times {10}^{14}$	-
1	$4.34\times {10}^{13}$	$-4.16\times {10}^{14}$	$1.54\times {10}^{14}$	$-3.05\times {10}^{14}$
2	$5.84\times {10}^{12}$	$-3.76\times {10}^{13}$	$1.44\times {10}^{13}$	$-1.39\times {10}^{14}$
3	$4.88\times {10}^{12}$	$-9.61\times {10}^{11}$	$5.54\times {10}^{12}$	$-8.9\times {10}^{12}$
4	$4.78\times {10}^{12}$	$-9.94\times {10}^{10}$	$4.66\times {10}^{12}$	$-8.81\times {10}^{11}$
5	$4.82\times {10}^{12}$	$3.51\times {10}^{10}$	$4.65\times {10}^{12}$	$-4.5\times {10}^9$
6	$4.59\times {10}^{12}$	$-2.23\times {10}^{11}$	$4.6\times {10}^{12}$	$-4.9\times {10}^{10}$
7	$4.57\times {10}^{12}$	$-2.59\times {10}^{10}$	$4.53\times {10}^{12}$	$-6.9\times {10}^{10}$
8	$4.45\times {10}^{12}$	$-1.23\times {10}^{11}$	$4.51\times {10}^{12}$	$-2.87\times {10}^{10}$
9	$4.38\times {10}^{12}$	$-6.33\times {10}^{10}$	$4.49\times {10}^{12}$	$-1.38\times {10}^{10}$
10	$4.43\times {10}^{12}$	$5.05\times {10}^{10}$	$4.49\times {10}^{12}$	$-5.34\times {10}^9$
11	$4.36\times {10}^{12}$	$-7.09\times {10}^{10}$	$4.48\times {10}^{12}$	$-8.91\times {10}^9$
12	$4.42\times {10}^{12}$	$5.52\times {10}^{10}$	$4.47\times {10}^{12}$	$-8.72\times {10}^9$
13	$4.38\times {10}^{12}$	$-3.75\times {10}^{10}$	$4.49\times {10}^{12}$	$1.92\times {10}^{10}$
14	$4.5\times {10}^{12}$	$1.23\times {10}^{11}$	$4.47\times {10}^{12}$	$-1.41\times {10}^{10}$
15	$4.41\times {10}^{12}$	$-9.66\times {10}^{10}$	$4.47\times {10}^{12}$	$-7.55\times {10}^9$

provides J’s values, evaluated for each computed process and each case. In this table, i stands for the iteration number. Start with $i=0$ for the initial process, $i=1$ for the process obtained after one improvement, $i=2$ for that obtained after two improvements, and so forth. The table also denotes the relative change in J, signified by $\mathsf{\Delta }{J}_i\triangleq J_i-J_{i-1}$.

It can be seen that J’s values in case 2 are different than those of case 1. This implies that the processes, obtained in each case, differ too. Additionally, for an improving sequence, $\mathsf{\Delta }J$ is expected to be non-positive. However, starting at the fifth iteration of case 1, $\mathsf{\Delta }J$ introduces some positive values, which means that a deterioration was obtained rather than an improvement. Although this issue also exists in case 2, it is milder. In case 2, only one such incident was recorded—after iteration 13. The explanation to this non-monotonous behavior is numerical errors, involved in the computation of $\mathbf{P}$ and $\mathbf{p}$. Obviously, case 2, which benefits from higher accuracy due to a smaller integration step, is more reliable than case 1.

Hence, inspection of the monotonicity of $\left\{{(\mathbf{x},\mathbf{u})}_i\right\}$ can serve as a simple measure of the accuracy of the obtained process. Nevertheless, a deeper inspection can be performed through a comparison of the original cost-to-go $\overline{J}$ with its equivalent counterpart, which was defined in Corollary 1, ${\overline{J}}_{eq}$. Such a comparison is presented in Figure 2. In this figure, case 1’s costs are ${\overline{J}}^{c1}$, ${\overline{J}}_{eq}^{c1}$, and case 2’s are ${\overline{J}}^{c2}$, ${\overline{J}}_{eq}^{c2}$. Corollary 1 states that, theoretically, $\overline{J}$ and ${\overline{J}}_{eq}$ should be identical for any $0\le t_1\le t_f$. However, this property is not observed in case 1. It can be seen that ${\overline{J}}_{eq}^{c1}$ substantially deviates from ${\overline{J}}^{c1}$. In contrast, in case 2, there is a relatively good agreement between ${\overline{J}}^{c2}$ and ${\overline{J}}_{eq}^{c2}$. Additionally, Figure 2 points out another interesting fact. It can be seen that ${\overline{J}}^{c1}$ and ${\overline{J}}^{c2}$ are not equal. Although they do have similar initial and terminal costs, they differ over the majority of time instances. This is another sign of the difference between the solutions obtained in case 1 and 2. However, the good agreement of ${\overline{J}}^{c2}$ and ${\overline{J}}_{eq}^{c2}$ implies that case 2 is the accurate one.

Can we better identify the reason for the poor performance of case 1? Based on Theorem 3, the answer is yes. Consider the identities defined in Theorem 3 and the terms defined in Equation (28). A better identification can be obtained by examining the equivalence of the terms in each one of the pairs: $(I_h,\mathbf{x}\mathbf{P}\mathbf{x})$, $(I_{hp},{\mathbf{x}}^T\mathbf{p})$, and $(I_p,I_{eq,p})$.

Figure 3 inspects this equivalence in each case. As before, a ‘c1’ superscript indicates case 1, and case 2 is indicated by ‘c2’. The correspondence in each of the pairs is conspicuous in case 2. However, even though case 1 shows reasonable similarity in the two first pairs, in the third pair, $I_p^{c1}$ is dissimilar to $I_{eq,p}^{c1}$ for $t\in [0,3.2]$. It follows that in the given problem this element is the reason for the difference between ${\overline{J}}^{c1}$ and ${\overline{J}}_{eq}^{c1}$. Because the pair $(I_p,I_{eq,p})$ consists of $\widehat{\mathbf{g}}$, which is absent from the other pairs, it is suggested that the numerical obstacle in the given problem is most likely related to the components of $\widehat{\mathbf{g}}$, i.e., either the control input $\mathbf{u}$, the external excitation ${\ddot{z}}_g$, or both.

6. Conclusions

This paper furnishes novel theoretical results that are related to a recently published solution to the CBQR problem. Specifically, the results refer to a key element in that solution—the improving function. First, Corollary 1 defines an equivalent cost-to-go performance index. Notably, its formulation is general and not limited to the CBQR problem. A sub-interval’s equivalence is an immediate consequence of this theorem and is discussed too. Next, Theorem 3 continues the idea presented in Corollary 1 but concentrates on the CBQR case.

The theorem breaks the equivalence down into smaller identities in relation to the components of the CBQR improving function.

These results allow us to verify the accuracy of the obtained numerical solution. This can be performed for the overall accuracy of the computed feedback and each of its components. It is an important tool as numerical issues frequently arise in solving ODEs, especially when discontinuous control signals and excitations are involved. Additionally, these theoretical results shed light on the meaning of the equivalent representation, the improving function, and the way in which they are related to the given problem.

A numerical example from structural control is provided to illustrate the above identities and how they can be used to verify the accuracy of the computed solution and pinpoint causes of inaccuracy, if there are such.

Based on the above results, different future research directions can be considered. The following are three possible directions. First, the results were demonstrated to have application in validating the numerical accuracy of improving functions. However, there is still work to be performed on systematically incorporating such a validation into the iteration process and what steps should be taken when errors are encountered. Second, can the identities from Theorem 3 be extended for more complex systems? Third, the derived identities provide novel explanations of the elements of the improving function related to the CBQR problem. It would be beneficial to examine their utilization in improving the computation time and effort required to solve that problem.