*2.3. Hamiltonian System: Minimization*

The Hamiltonian in Equation (8) is a function of the state, co-state, and decision criteria (or control) and allows linkage of the running costs *F*(*x*, *u*) with a linear measure of the behavior of the system dynamics *f*(*x*, *u*). Equation (9) articulates the Hamiltonian of the problem formulation described in Equations (2)–(5). Minimizing the Hamiltonian with respect to the decision criteria vector per Equation (10) leads to conditions that must be true if the cost function is minimized while simultaneously satisfying the constraining dynamics. Equation (11) reveals the optimal decision *u* will be known if the rate adjoint can be discerned.

$$H = F(\mathbf{x}, u) + \lambda^T f(\mathbf{x}, u) \tag{9}$$

$$H = \frac{1}{2}\mu^2 + \lambda\_x \upsilon + \lambda\_v \mu \tag{10}$$

$$\frac{\partial H}{\partial \boldsymbol{\mu}} = 0 \rightarrow \boldsymbol{\mu} + \lambda\_{\boldsymbol{\nu}} = 0 \tag{11}$$

#### *2.4. Hamiltonian System: Adjoint Gradient Equations*

The change of the Hamiltonian with respect to the adjoint *λ* maps to the time-evolution of the corresponding state in accordance with Equations (12) and (13).

$$
\dot{\lambda}\_x = -\frac{\partial H}{\partial x} = 0 \to \lambda\_x(t) = a \tag{12}
$$

$$
\dot{\lambda}\_{\upsilon} = -\frac{\partial H}{\partial v} = \lambda\_x \to \dot{\lambda}\_{\upsilon} = \lambda\_x(t) = a \to \lambda\_{\upsilon}(t) = -at - b \tag{13}
$$

The rate adjoint was discovered to reveal the optimal decision criteria, and the adjoint equations reveal the rate adjoint is time-parameterized with two unknown constants still to be sought. Together, Equations (11)–(13) form a system of differential equations to be solved with boundary conditions (often referred to as a two-point boundary value problem in mathematics).

### *2.5. Terminal Transversality of the Enpoint Lagrangian*

The endpoint Lagrangian *E* in Equation (14) adjoins the endpoint function endpoint cost *E x tf* and the endpoint constraints functions *<sup>e</sup> x tf* in Equation (8) and provides a linear measure for endpoint conditions in Equation (7). The endpoint Lagrangian *E* exists at the terminal (final) time alone. The transversality condition in Equation (15) specifies the adjoint at the final time is perpendicular to the cost at the end point. In this problem, the endpoint cost *E x tf* <sup>=</sup> 0. These Equations (16) and (17) are often useful when seeking a sufficient number of equations to solve the system.

$$E = E + \nu^T \mathcal{e} = \nu^T \mathcal{e} = \nu\_x \left(\mathbf{x}\_f - 1\right) + \nu\_v \left(\mathbf{v}\_f - 0\right) = \nu\_x \left(\mathbf{x}\_f - 1\right) + \nu\_v \mathbf{v}\_f \tag{14}$$

$$\frac{\partial \overline{E}}{\partial \mathbf{x}\_f} = \lambda\_\mathbf{x} \begin{pmatrix} \mathbf{t}\_f \\ \mathbf{t}\_f \end{pmatrix} \tag{15}$$

$$\frac{\partial \overline{E}}{\partial \mathbf{x}\_f} = \lambda\_\mathbf{x} \begin{pmatrix} t\_f \\ \end{pmatrix} = \boldsymbol{\upsilon}\_\mathbf{x} \tag{16}$$

$$\frac{\partial \overline{E}}{\partial \upsilon\_f} = \lambda\_\upsilon \left( t\_f \right) = \upsilon\_\upsilon \tag{17}$$

### *2.6. New Two-Point Boundary Value Problem*

For the two-state system, four equations are required with four known conditions to evaluate the equations. In this instance, two Equations (3)–(10) have been formulated for state dynamics, two more Equations (18) and (19) have been formulated for the adjoints, and two more Equations (20) and (21) have been formulated for the adjoint endpoint conditions. Four known conditions, Equations (22)–(25) have also been formulated. Combining Equations (11) and (13) produce Equation (26).

.

$$
\dot{\mathbf{x}} = \mathbf{v} \tag{18}
$$

$$
\dot{v} = \mu \tag{19}
$$

$$
\dot{\lambda}\_x = 0 \tag{20}
$$

$$
\dot{\lambda}\_{\text{v}} = -\lambda\_{\text{x}} \tag{21}
$$

$$\mathbf{x}(0) = 0\tag{22}$$

*v*(0) = 0 (23)

$$\mathbf{x}(1) = 1\tag{24}$$

$$v(1) = 0\tag{25}$$

Evaluating Equation (27) with Equation (23) produces the value *c* = 0. Evaluating Equation (28) with Equation (22) produces the value *d* = 0. Evaluating Equation (27) with Equation (25) produces Equation (29), while evaluating Equation (28) with Equation (24) produces Equation (30). .

$$
\dot{v} = -\lambda\_v(t) = at + b \tag{26}
$$

$$v = \int \dot{v} dt = \frac{1}{2}at^2 + bt + c \tag{27}$$

$$\propto = \int vdt = \frac{1}{6}at^3 + \frac{1}{2}bt^2 + ct + d \tag{28}$$

$$v(1) = \frac{1}{2}a + b = 0\tag{29}$$

$$\mathbf{x}(1) = \frac{1}{6}a + \frac{1}{2}b = 1\tag{30}$$

Solving the system of two Equations (29) and (30) produces *a* = −12 and *b* = 6. Substituting Equation (26) into Equation (11) with *a* and *b* produces Equation (31), and substitution of *a* and *b* into Equations (27) and (28), respectively, produce Equations (32) and (33) the solution of the trajectory optimization problem.

$$
u^\*(t) = -12t + 6\tag{31}$$

$$v^\*(t) = -3t^2 + 6t\tag{32}$$

$$\mathbf{x}^\*(t) = -2t^3 + 3t^2 \tag{33}$$

Equations (31)–(33) constitute the optimal solution for quiescent initial conditions and the state final conditions (zero velocity and unity scaled position). To implement a *form of feedback* (not classical feedback), consider leaving the initial conditions non-specific in variable-form as described next.

### *2.7. Real-Time Feedback Update of Boundary Value Problem Optimum Solutions*

Classical methods utilize feedback of asserted form *u* = −*Kx* for state variable *x*, where the decision criteria (for control or state estimation/observer) and gains *K* are solved to achieve some stated performance criteria. Such methods are used in Section 3 and their results are established as benchmarks for comparison. So-called modern methods utilize optimization problem formulation to eliminate classical gain tuning substituting optimal gain selection but retaining the asserted form of the decision criteria. Such methods are often referred to as "linear-quadratic optimal" estimators or controllers. These estimators are also presented as benchmarks for comparison, where the optimization problem equally weights state errors and estimation accuracy.

Alternative use of feedback is proposed here (whose simulation is depicted in Figure 3b). Rather than classical feedback topologies asserting *u* = −*Kx* utilization of state feedback in formulating the estimator or control's decision criteria, this section proposes relabeling the current state feedback as the new initial conditions of the two-point boundaryvalue problems used to solve for optimal state estimates or control decision criteria in Equations (22) and (23). The solution of (26)–(28) using the initial values of (22) and (23) manifest in values of the integration constants: *a* = −12 and *b* = 6. As done in real-time optimal control, the values of the integration constants are left "free" in variable form, and their values are newly established for each discrete instance of state feedback (re-labeled as new initial conditions). This notion is proposed in Proposition 1, whose proof expresses the form of the online calculated integration constants that solve the new optimization problem. The two constants *a*ˆ and ˆ *b* are utilized in the same decision Equation (31) where the estimates replace the formerly solved values of the boundary value problem resulting in Equation (40).

**Figure 3.** Simulink systems for noisy sensors and decision criteria (guidance or control) subsystem: (**a**) noisy sensor subsystem; (**b**) decision topology (guidance or control).

**Proposition 1.** *Feedback may be utilized not in closed form to solve the constrained optimization problem in real time.*

$$x = \frac{1}{6}at^3 + \frac{1}{2}bt^2\tag{34}$$

$$v = \frac{1}{2}at^2 + bt\tag{35}$$

$$x\_f = \frac{1}{6}at\_f^3 + \frac{1}{2}bt\_f^2\tag{36}$$

$$v\_f = \frac{1}{2}at\_f^2 + bt\_f\tag{37}$$

**Proof of Proposition 1.** Implementing Equations (34)–(37) in matrix form as revealed in Equation (38) permits solution for the unknown constants as a function of time as displayed in Equation (39), and subsequent use of the unknown constants form the new optimal solution from the current position and velocity per Equation (40).

$$\underbrace{\begin{bmatrix} t\_0^{t\_0} & t\_0^{t\_0} & t\_0 & 1\\ \frac{t\_0^{t\_0}}{2} & t\_0 & 1 & 0\\ \frac{1}{2} & \frac{1}{2} & 1 & 1\\ \frac{1}{2} & 1 & 1 & 0 \end{bmatrix}}\_{T} \underbrace{\begin{Bmatrix} a\\ b\\ c\\ d \end{Bmatrix}}\_{p} = \underbrace{\begin{Bmatrix} x\_0\\ v\_0\\ 1\\ 0 \end{Bmatrix}}\_{q} \tag{38}$$

$$\left\{ \begin{array}{ccc} \dot{a}\\ \dot{b}\\ c\\ d \end{array} \right\} = \begin{bmatrix} \frac{t\_0^3}{6} & \frac{t\_0^2}{2} & t\_0 & 1\\ \frac{t\_0^2}{2} & t\_0 & 1 & 0\\ \frac{1}{2} & \frac{1}{2} & 1 & 1\\ \frac{1}{2} & 1 & 1 & 0 \end{bmatrix}^{-1} \begin{Bmatrix} x\_0\\ v\_0\\ 1\\ 0 \end{Bmatrix} \right\} \tag{39}$$

$$
\mu^\* \equiv \mathfrak{A}t + \hat{b} \tag{40}
$$

In Section 3, estimation of *a*ˆ and ˆ *b* becomes singular due to the inversion in Equation (39) as approaching the terminal endpoint, where switching to Equations (31)–(33) is implemented as depicted in Figure 4d to avoid the deleterious effects of singularity when applying Proposition 1. The cases with switching at singular conditions are suffixes with "*with switching*" in the respective label.

**Figure 4.** SIMULINK subsystems: (**a**) implementation of classical feedback methods; (**b**) calculation of quadratic cost; (**c**) open loop decision topologies (sinusoidal and optimal); (**d**) switching function to disengage use of real-time parameter updates when the matrix in Equation (39) is rank deficient. Notice in subfigure (**c**) sinusoidal wave input is coded using the identical time-index of the rest of the simulation. The next stages of future research will utilize this identical simulation to investigate efficacy of the proposed virtual sensoring amidst unknown wave actions.
