A Theory of Functional Connections-Based hp-Adaptive Mesh Refinement Algorithm for Solving Hypersensitive Two-Point Boundary-Value Problems

Abstract

This manuscript introduces the first hp-adaptive mesh refinement algorithm for the Theory of Functional Connections (TFC) to solve hypersensitive two-point boundary-value problems (TPBVPs). The TFC is a mathematical framework that analytically satisfies linear constraints using an approximation method called a constrained expression. The constrained expression utilized in this work is composed of two parts. The first part consists of Chebyshev orthogonal polynomials, which conform to the solution of differentiation variables. The second part is a summation of products between switching and projection functionals, which satisfy the boundary constraints. The mesh refinement algorithm relies on the truncation error of the constrained expressions to determine the ideal number of basis functions within a segment’s polynomials. Whether to increase the number of basis functions in a segment or divide it is determined by the decay rate of the truncation error. The results show that the proposed algorithm is capable of solving hypersensitive TPBVPs more accurately than MATLAB R2021b’s bvp4c routine and is much better than the standard TFC method that uses global constrained expressions. The proposed algorithm’s main flaw is its long runtime due to the numerical approximation of the Jacobians.

1. Introduction

In recent years, the Theory of Functional Connections (TFC) [1] has garnered much interest from researchers for obtaining numerical solutions to scientific and engineering problems. A few examples of the types of problems the TFC has been used to solve include those from transport theory [2], solid mechanics [3,4], and orbit transfer [5,6,7]. The increase in the TFC’s popularity is due to its successful use of functional interpolation to analytically satisfy linear constraints exactly. Functional interpolation is performed by approximating a solution via a constrained expression, which comprises an arbitrary free function and a summation of products between switching functions and projection functionals. Some constraints that could be handled were immediately apparent during the TFC’s inception by Daniele Mortari [8]. Examples include point, derivative, and relative constraints. The ability of the TFC to analytically satisfy these constraints exactly allowed it to be used to solve one-dimensional nonlinear ordinary differential equations (ODEs) in milliseconds and with machine-level accuracy [9]. In this work, we are concerned with solving linear and nonlinear hypersensitive two-point boundary-value problems (TPBVPs) stemming from optimal control problems (OCPs) involving point and derivative constraints. These problems are challenging because of steep gradients in their solutions, which are difficult to approximate with global constrained expressions. We overcome this challenge by developing the first hp-adaptive mesh refinement algorithm for the TFC. The algorithm splits the constrained expressions into multiple segments and adaptively determines the length, location, and hyperparameters for each segment to accurately approximate a solution.

Three families of optimal control methods exist: dynamic programming (DP) [10], direct [11,12], and indirect methods [13,14]. DP solves an OCP by transforming it into the Hamilton–Jacobi–Bellman (HJB) partial differential equation, which gives the necessary and sufficient conditions for an optimal solution when solved and enables closed-loop control effort. Direct methods convert the OCP into a nonlinear programming (NLP) problem that can be solved by any of the available NLP solvers in the literature but are inherently open-loop. Indirect methods are also open-loop, but they analytically construct the necessary conditions for optimality via Pontryagin’s Minimum Principle (PMP) and solve a resulting dual TPBVP to obtain the OCP solution. Indirect methods tend to solve OCPs more accurately than direct methods but are less likely to converge because they are more sensitive to an initial guess. Whether one class of methods should be used over another is problem-dependent. Hence, the TFC has already been used to solve OCPs via all three families, but some more than others. Schiassi et al. used a subset of the TFC, called the Extreme Theory of Functional Connections (X-TFC), to solve the HJB for finite and infinite horizon OCPs with linear and nonlinear dynamics [15]. For the direct method, Zhang et al. solved several coplanar orbit low-thrust multi-target visit problems, governed by aspherical gravity perturbation, with the TFC to analytically satisfy the orbital multi-target visit constraints in an NLP [16]. The TFC has been used to solve OCPs via the indirect method much more extensively in the literature. For example, by forming a TPBVP via PMP, the TFC has been used to solve many spacecraft energy optimal rendezvous problems in order to avoid keep-out-zones [17], planetary landing problems [18], and the famous Riccati problem of optimal control [19].

Even though the TFC has been shown to solve many OCPs very well via the indirect method, the method fails to converge on solutions that are not smooth (e.g., bang-bang problems). This occurs because the TFC initially utilizes global orthogonal polynomials for the free function, which only converge exponentially on smooth continuous functions [20] (p. 29). This issue was rectified by Johnston et al. by splitting the domain of a fuel-optimal planetary landing OCP into several intervals connected by knot points [18]. The knot points were placed where the discontinuity in the solution occurred, making each interval smooth. The total solution could then be converged upon by using constrained expressions for each segment and tying them together. The locations of the knot points were found via an optimization algorithm, which was feasible because the number of switching points in a fuel-optimal planetary landing OCP is known when the gravity is assumed to be constant [21]. A technique where the domains of TPBVPs could be split into segments and solved with the TFC was eventually formalized by Johnston and Mortari [22] for solving hybrid problems. Nonetheless, Johnston and Mortari provided no insight into how to determine the number of orthogonal polynomial basis terms in an interval and how to determine the number of intervals when they are unknown a priori. Hence, our desire to develop a TFC-based adaptive mesh refinement algorithm for determining these hyperparameters.

Researchers have heavily investigated the use of adaptive mesh refinement algorithms in conjunction with many OCP techniques. One of the more well-known applications in optimal control theory is the coupling with pseudospectral methods for solving OCPs directly [23,24,25,26,27], where the state and control are approximated using Lagrange orthogonal polynomials with Gaussian quadrature collocation points. Most adaptive mesh refinement methods for pseudospectral methods are referred to as hp methods because they achieve better convergence toward the solution by varying the number of intervals on the domain (the h method) and the degree of the orthogonal polynomial (the p method). A more in-depth explanation of the h, p, and hp methods can be found by studying finite-element methods [28,29,30], where they are also widely used. For the mesh refinement algorithm to be adaptive, an a posteriori estimate of the error between the actual and approximate solutions must be quantified to determine the ideal number of intervals, the degree of the approximating polynomials, and the location of the knot points for each interval. For example, Darby et al. approximated the error by computing the residuals on and between collocation points [24]. When the error was above some tolerance, they increased the polynomial degree in intervals with low curvature or uniform error across the collocation points. An interval was divided into subintervals instead when the curvature was high or the error across the collocation points in an interval was not uniform. Unfortunately, the error estimates from Darby et al. [24] created much noise, making them computationally intractable when a highly accurate solution is desired. Patterson et al. developed a more accurate error estimate by using the difference between an interpolated value of the state and a quadrature approximation to the integral of the state dynamics [25]. They performed adaptive mesh refinement based on an error estimate ratio: when a set number of polynomial degrees was used to approximate the state, the degree number was increased by one for interpolation. The spectral convergence properties of pseudospectral methods could then be used to determine how many degrees to increase the polynomial by or to divide the interval into smaller subintervals.

The mesh refinement methods proposed in [24,25] only increased the mesh, which sometimes increased the computational complexity. Liu et al. [26] introduced a technique that both increased and decreased the mesh based on the error estimate from Patterson et al. [25] and used a ratio between the second derivative of the approximation on the current mesh and the previous one to determine the smoothness of an interval. Their algorithm used the power-series representation of the Lagrange polynomial to reduce the degree of the approximating polynomial within a mesh interval for smooth intervals when the estimated error was lower than the tolerance. Smooth neighboring mesh intervals were also combined when the estimated error of the larger interval, which extended into the smaller one, was similar. More recently, Liu et al. [27] discovered a new way to estimate the smoothness of mesh intervals according to the decay rate of a Legendre polynomial approximation and increased the mesh size based on the decay rate. The work performed in this manuscript is heavily inspired by [27].

This paper proposes an hp-adaptive mesh refinement algorithm that the TFC, with a truncated Chebyshev orthogonal polynomial-free function, can use to solve hypersensitive TPBVPs. The error between the actual solution and the constrained expression approximation is estimated using an exponential least-squares fit of the Chebyshev coefficients, the orthogonal property of Chebyshev polynomials, and a rule of thumb that correlates the error associated with Chebyshev polynomials when approximating functions with their truncation error. The decay rate of the exponential least-squares fit of the Chebyshev coefficients can be used to determine the smoothness of the solution on an interval. When the error estimate is greater than some tolerance and the interval is deemed smooth, the super-geometric convergence of the truncation error for Chebyshev polynomials can be used to relate the error to the ideal number of basis functions in the Chebyshev polynomial needed to satisfy the tolerance. If the interval is not smooth, it is divided into subintervals so that the solution in each subinterval is easier to approximate with a constrained expression. Smooth intervals can have the number of basis functions in the Chebyshev polynomial decreased when they already satisfy the error estimate. Furthermore, smooth intervals can also be combined if they are exceptionally smooth or if the number of basis functions in all their constrained expression approximations is some minimum number.

As the name of this manuscript suggests, we only demonstrate how the proposed hp-adaptive mesh refinement algorithm can be used to solve hypersensitive TPBVPs, even though it can also be used for more well-behaved problems. Hypersensitive problems have a very long time domain with respect to the rates of expansion and contraction of the dynamics in specific directions within a neighborhood of the optimal solution [31]. Thus, the TPBVPs solved in this work have sharp increases in the gradients near the boundary conditions and a smooth segment between them. These sharp increases concerning the entire domain make hypersensitive TPBVPs challenging to solve with global constrained expression approximations. Although hypersensitive TPBVPs can stem from the indirect method of optimal control, the ones we solve in this work do not contain inequality constraints on the control or states. We do not solve these types of problems in this work because they require smoothing techniques along with a continuation procedure, drastically decreasing computational efficiency. The computation time of our hp-adaptive mesh refinement algorithm is already long (see Section 6 for more details). Therefore, we hope to speed up the computational runtime before coupling the proposed hp-adaptive mesh refinement algorithm with continuation.

This manuscript is outlined as follows. In the next section, we introduce how to solve a general TPBVP with the TFC, as well as how to split the domain into intervals and solve the solution over each with the TFC. The derivation of an error estimate for constrained expressions with orthogonal polynomial-free function approximations is highlighted in Section 3, along with an error analysis for smooth and nonsmooth functions. The TFC-based hp-adaptive mesh refinement algorithm that uses our derived error estimate is then proposed in Section 4. Numerical results for solving three hypersensitive TPBVPs are then provided and discussed in Section 5, along with a computational time study. Lastly, conclusions and an outlook on future work are presented in Section 7.

2. Theory of Functional Connections

Here, the TFC is first presented for solving a TPBVP with a single ODE and with a global constrained expression. We then show how global constrained expressions can be used to solve a TPBVP composed of a system of ODEs. Lastly, we demonstrate how global constrained expressions can be replaced with piecewise constrained expressions, where each only approximates a domain segment.

2.1. General TPBVP Outline

Solving a TPBVP via the TFC can be accomplished by following a simple set of procedures. Indeed, a “cookbook” guide can be summarized in three steps:

1.

Define the loss equation(s), which are represented by the residuals of the differential equations (DEs).

2.

Derive the constrained expression(s).

3.

Map the problem domain to that of the free function (if necessary) or simply normalize it for numerical stability.

Assume a general TPBVP,

$$\ddot{y}\left(t\right)=f(t,y\left(t\right),\dot{y}\left(t\right))\mathrm{subject}\mathrm{to}:\left\{\begin{array}{c}y\left(t_0\right)=y_0\hfill \\ y\left(t_f\right)=y_f\hfill \end{array}\right.,$$

(1)

where f is some function in terms of time t and the differentiation variable y. As its name suggests, the TPBVP is constrained at the beginning and end of the time domain $t\in [t_0,t_f]$. With the TFC, we approximate the solution with a constrained expression Y,

$$y\left(t\right)\approx Y(t,g\left(t\right))=g\left(t\right)+\sum _{l=1}^{N_{\mathrm{const}}}{\Omega }_l\left(t\right){\varrho }_l(t_l,g\left(t_l\right)),$$

(2)

where $g\left(t\right)$ is a free function, ${\Omega }_l\left(t\right)$ are switching functions, ${\varrho }_l(t_l,g\left(t_l\right))$ are projection functionals, and $N_{\mathrm{const}}$ is the number of linear constraints.

Defining the loss equation for any set of DEs, including the ODE in Equation (1), is easy. Simply bring all terms to one side of the equals sign, like so,

$$\ddot{y}\left(t\right)-f(t,y\left(t\right),\dot{y}\left(t\right))=\hat{f}\left(t,y\left(t\right),\dot{y}\left(t\right),\ddot{y}\left(t\right)\right)=0,$$

(3)

where $\hat{f}$ is the constrained DE residual. To turn the TPBVP into an unconstrained problem (i.e., $\hat{f}$ is no longer subject to $y_0$ and $y_f$), the constrained expressions for y and $\dot{y}$ must be derived.

We start the process of figuring out the constrained expressions by rewriting Equation (2) as

$$Y(t,g\left(t\right))=g\left(t\right)+{\Omega }_1\left(t\right){\varrho }_1(t_0,g\left(t_0\right))+{\Omega }_2\left(t\right){\varrho }_2(t_f,g\left(t_f\right)).$$

The projection functionals (i.e., ${\varrho }_1$ and ${\varrho }_2$) are simply the difference between the constraint value and the free function evaluated at the point where the constraint is defined:

$$\begin{array}{c}\hfill {\varrho }_1=y_0-g\left(t_0\right)\\ \hfill {\varrho }_2=y_f-g\left(t_f\right).\end{array}$$

The most cumbersome part of deriving the constrained expressions is the derivation of the switching functions, which are of the general forms

$${\Omega }_1\left(t\right)=\sum _{i=1}^{N_{\mathrm{const}}=2}{s}_i\left(t\right){\alpha }_{i1}\mathrm{and}{\Omega }_2\left(t\right)=\sum _{i=1}^{N_{\mathrm{const}}=2}{s}_i\left(t\right){\alpha }_{i2}.$$

Only two switching functions exist because the ODE has two boundary constraints. Combining the equations for the switching functions into a compact linear combination where the constraints occur yields

$$\left[\begin{array}{cc}{s}_1\left(t_0\right)& s_2\left(t_0\right)\\ s_1\left(t_f\right)& s_2\left(t_f\right)\end{array}\right]\left[\begin{array}{cc}{\alpha }_{11}& {\alpha }_{12}\\ {\alpha }_{21}& {\alpha }_{22}\end{array}\right]=\left[\begin{array}{cc}{\Omega }_1\left(t_0\right)& {\Omega }_2\left(t_0\right)\\ {\Omega }_1\left(t_f\right)& {\Omega }_2\left(t_f\right)\end{array}\right].$$

To figure out the switching functions, compute the unknown coefficients ${\alpha }_{ij}$. The support functions $s_i$ do not have to be derived but can merely be chosen. However, to ensure the support matrix is invertible, the columns of the support matrix must be linearly independent. For point constraints, like in our case, the selection of $s_1\left(t\right)=1$ and $s_2\left(t\right)=t$ satisfies this condition. A switching function is defined as being equal to 1 when evaluated at the constraint it is referencing and equal to 0 at all other constraints. Therefore, the system can be solved as

$$\begin{array}{cc}\hfill \left[\begin{array}{cc}1& t_0\\ 1& t_f\end{array}\right]\left[\begin{array}{cc}{\alpha }_{11}& {\alpha }_{12}\\ {\alpha }_{21}& {\alpha }_{22}\end{array}\right]& =\left[\begin{array}{cc}1& 0\\ 0& 1\end{array}\right]\hfill \\ \hfill \left[\begin{array}{cc}{\alpha }_{11}& {\alpha }_{12}\\ {\alpha }_{21}& {\alpha }_{22}\end{array}\right]& ={\left[\begin{array}{cc}1& t_0\\ 1& t_f\end{array}\right]}^{-1}\left[\begin{array}{cc}1& 0\\ 0& 1\end{array}\right]\hfill \\ \hfill & =\frac{1}{t_f-t_0}\left[\begin{array}{cc}{t}_f& -t_0\\ -1& 1\end{array}\right].\hfill \end{array}$$

Substituting the coefficients back into the switching functions and simplifying them gives

$${\Omega }_1\left(t\right)=\frac{t_f{s}_1\left(t\right)-s_2\left(t\right)}{t_f-t_0}=\frac{t_f-t}{t_f-t_0}$$

and

$${\Omega }_2\left(t\right)=\frac{s_2\left(t\right)-t_0{s}_1\left(t\right)}{t_f-t_0}=\frac{t-t_0}{t_f-t_0}.$$

With the switching and projection functions defined, all that is left to do to finish deriving the constrained expression is to decide what the free function should be. One commonly selected free function among researchers who utilize the TFC method for solving ODEs is a Chebyshev polynomial expansion of the first kind,

$$g\left(t\right)=\sum _{j=N_{\mathrm{const}}}^{N_{\mathrm{const}}+L-1}{\xi }_j{\mathcal{T}}_j\left(\tau \right)={\mathit{\xi }}^{⊺}\mathcal{T}\left(\tau \right),$$

where ${\xi }_j$ are the coefficients, ${\mathcal{T}}_j\left(\tau \right)$ are the Chebyshev basis functions, and L is the number of chosen basis functions. When one expresses a Chebyshev series, one often starts the summation from the zeroth order (i.e., $j=0$). The TFC process requires the series summation to begin at the order equal to the number of constraints (i.e., $j=N_{\mathrm{const}}$) because the Chebyshev series terms of order less than $N_{\mathrm{const}}$ are linearly dependent on the switching functions. The algebraic system of equations formed via the loss functions must be linearly independent to ensure that the Jacobian products are invertible in the Gauss–Newton algorithm and linear least squares give a unique solution. The loss function eventually derived is guaranteed to be linearly independent when the Chebyshev series begins at $j=N_{\mathrm{const}}$.

The domain of Chebyshev polynomials $\tau \in [-1,1]$ does not coincide with the time domain $t\in [t_0,t_f]$ unless $t_0=-1$ and $t_f=1$. Thus, to ensure all terms within the constrained expressions are on the same domain, the switching functions can be linearly mapped to the $\tau$ domain using the equations,

$$\tau ={\tau }_0+\frac{{\tau }_f-{\tau }_0}{t_f-t_0}(t-t_0)⟷t=t_0+\frac{t_f-t_0}{{\tau }_f-{\tau }_0}(\tau -{\tau }_0).$$

The w-th derivative of the free function can then be defined as

$$\frac{{\mathrm{d}}^{w}g}{\mathrm{d}{t}^w}={\mathit{\xi }}^{⊺}\frac{{\mathrm{d}}^w\mathcal{T}\left(\tau \right)}{\mathrm{d}{\tau }^w}{c}^w,$$

(4)

where the mapping constant c is

$$c=\left(\frac{2}{t_f-t_0}\right).$$

(5)

With the free function and subsequent mapping defined, the constrained expressions for the y, $\dot{y}$, and $\ddot{y}\left(t\right)$ variables that make up the loss equation can be defined as

$$\begin{array}{cc}\hfill Y\left(t\right)=Y\left(\tau \right)& ={\mathit{\xi }}^{⊺}\mathcal{T}\left(\tau \right)+{\Omega }_1\left(\tau \right)\left(y_0-{\mathit{\xi }}^{⊺}\mathcal{T}\left({\tau }_0\right)\right)+{\Omega }_2\left(\tau \right)\left(y_f-{\mathit{\xi }}^{⊺}\mathcal{T}\left({\tau }_f\right)\right)\hfill \\ \hfill & ={\left(\mathcal{T}\left(\tau \right)-{\Omega }_1\left(\tau \right)\mathcal{T}\left({\tau }_0\right)-{\Omega }_2\left(\tau \right)\mathcal{T}\left({\tau }_f\right)\right)}^{⊺}\mathit{\xi }+{\Omega }_1\left(\tau \right)y_0+{\Omega }_2\left(\tau \right)y_f,\hfill \end{array}$$

$$\begin{array}{cc}\hfill \frac{\mathrm{d}Y\left(t\right)}{\mathrm{d}t}=c\frac{\mathrm{d}Y\left(\tau \right)}{\mathrm{d}\tau }& =\left(\frac{2}{t_f-t_0}\right)[{\left(\frac{\mathrm{d}\mathcal{T}\left(\tau \right)}{\mathrm{d}\tau }-\frac{\mathrm{d}{\Omega }_1\left(\tau \right)}{\mathrm{d}\tau }\mathcal{T}\left({\tau }_0\right)-\frac{\mathrm{d}{\Omega }_2\left(\tau \right)}{\mathrm{d}\tau }\mathcal{T}\left({\tau }_f\right)\right)}^{⊺}\mathit{\xi }\hfill \\ \hfill & +\frac{\mathrm{d}{\Omega }_1\left(\tau \right)}{\mathrm{d}\tau }{y}_0+\frac{\mathrm{d}{\Omega }_2\left(\tau \right)}{\mathrm{d}\tau }{y}_f],\hfill \end{array}$$

and

$$\begin{array}{cc}\hfill \frac{{\mathrm{d}}^{2}Y\left(t\right)}{\mathrm{d}{t}^2}=c^2\frac{{\mathrm{d}}^{2}Y\left(\tau \right)}{\mathrm{d}{\tau }^2}& =\left(\frac{2}{t_f-t_0}\right)[{\left(\frac{{\mathrm{d}}^2\mathcal{T}\left(\tau \right)}{\mathrm{d}{\tau }^2}-\frac{{\mathrm{d}}^2{\Omega }_1\left(\tau \right)}{\mathrm{d}{\tau }^2}\mathcal{T}\left({\tau }_0\right)-\frac{{\mathrm{d}}^2{\Omega }_2\left(\tau \right)}{\mathrm{d}{\tau }^2}\mathcal{T}\left({\tau }_f\right)\right)}^{⊺}\mathit{\xi }\hfill \\ \hfill & +\frac{{\mathrm{d}}^2{\Omega }_1\left(\tau \right)}{\mathrm{d}{\tau }^2}{y}_0+\frac{{\mathrm{d}}^2{\Omega }_2\left(\tau \right)}{\mathrm{d}{\tau }^2}{y}_f].\hfill \end{array}$$

The constrained expressions can now be plugged into Equation (3), which transforms the constrained ODE $\hat{f}$ into an unconstrained ODE $\check{f}$ consisting of the basis domain $\tau$, unknown Chebyshev coefficients $\mathit{\xi }$, endpoints of the time domain ($t_0$ and $t_f$), and the boundary conditions of the constrained ODE $\mathit{\beta }$:

$$\check{f}(\tau ,\mathit{\xi },\mathit{\beta },t_f,t_0)=c\frac{{\mathrm{d}}^{2}Y\left(\tau \right)}{\mathrm{d}{\tau }^2}-f\left(\tau ,Y\left(\tau \right),c\frac{\mathrm{d}Y\left(\tau \right)}{\mathrm{d}\tau }\right)=0,$$

where $\mathit{\beta }={\left\{\begin{array}{cc}{y}_0,& y_f\end{array}\right\}}^{⊺}$.

With the constrained expressions plugged into the DE residual, an algebraic system of equations can be formed after the domain is discretized with Chebyshev–Gauss–Lobatto nodes,

$${\tau }_d=-cos\frac{d\pi }{N}\mathrm{for}\mathrm{all}d=(0,\dots ,N-1),$$

where N is the number of discretization points, also called collocation points. Note that having $N\ge L$ ensures that the algebraic system of equations formed via the loss equations is not undetermined. Chebyshev–Gauss–Lobatto nodes are better than uniformly spaced points for discretizing the domain because they avoid the Runge phenomena. Once the domain is discretized, a vector of loss functions at each discretization point can be expressed as

$$\mathcal{L}\left(\mathit{\xi }\right)=\left\{\begin{array}{c}\check{f}({\tau }_0,\mathit{\xi },\mathit{\beta },t_f,t_0)\\ ⋮\\ \check{f}({\tau }_d,\mathit{\xi },\mathit{\beta },t_f,t_0)\\ ⋮\\ \check{f}({\tau }_{N-1},\mathit{\xi },\mathit{\beta },t_f,t_0)\end{array}\right\},$$

(6)

where $\mathit{\xi }$ is the only unknown variable, which is why we write the loss only in terms of it here. If f is linear, then $\mathit{\xi }$ within $\check{f}$ will show up linearly. Equation (6) then yields the form

$$\mathit{A}\mathit{\xi }+\mathit{b}=0,$$

(7)

where the matrix $\mathit{A}$ is a linear combination of terms multiplied by $\mathit{\xi }$, while the vector $\mathit{b}$ is the loss vector evaluated at $\mathit{\xi }=0$. Linear least squares can then be used to solve Equation (7) for $\mathit{\xi }$.

In the case of nonlinear DEs, Equation (6) will be nonlinear in the unknown ${\xi }_j$ coefficients. The Gauss–Newton method, also known as iterative least squares, can then be used to solve for the change in $\mathit{\xi }$ at each p iteration step, which is denoted by

$$\Delta {\mathit{\xi }}_p={\mathit{\xi }}_{p+1}+{\mathit{\xi }}_p.$$

(8)

The Gauss–Newton technique is expressed as

$$\Delta {\mathit{\xi }}_p=-{\left(\mathit{J}{\left({\mathit{\xi }}_p\right)}^{⊺}\mathit{J}\left({\mathit{\xi }}_p\right)\right)}^{-1}\mathit{J}{\left({\mathit{\xi }}_p\right)}^{⊺}\mathcal{L}\left({\mathit{\xi }}_p\right),$$

where $\mathit{J}\left({\mathit{\xi }}_p\right)$ is the Jacobian of the loss vector with respect to the $\mathit{\xi }$ variables and is computed at ${\mathit{\xi }}_p$. Iterations are repeated until some stopping criterion(s) is met, which in this work is

$$\parallel \mathcal{L}\left({\mathit{\xi }}_p\right){\parallel }_{\infty }<{\epsilon }_{\mathrm{GN}},\parallel \mathcal{L}\left({\mathit{\xi }}_{p+1}\right){\parallel }_{\infty }-{\parallel \mathcal{L}\left({\mathit{\xi }}_p\right)\parallel }_{\infty }<{\epsilon }_{\mathrm{GN}},\mathrm{or}p=p_{\max },$$

(9)

where ${\epsilon }_{\mathrm{GN}}$ defines some user-prescribed tolerance, ${\parallel ·\parallel }_{\infty }$ refers to the ${\mathrm{L}}_{\infty }$-norm, and $p_{\max }$ is the maximum number of iterations. When any stopping criterion is met, the solution is obtained by substituting $\mathit{\xi }$ into the constrained expression and mapping back to the problem domain, which in this example is time. If the last two stopping criteria were met before the first, then the Gauss–Newton algorithm failed to converge toward an accurate solution.

2.2. Solving Systems of ODEs

The example of the general TPBVP solved via the TFC method in the previous walkthrough only contained a single ODE. TPBVPs composed of n ODEs can also be solved via the TFC. Consider the following TPBVP:

$${\ddot{y}}_i\left(t\right)=f_i(t,\mathit{y}\left(t\right),\dot{\mathit{y}}\left(t\right))\mathrm{subject}\mathrm{to}{y}_i\left(t_{il}\right)={\beta }_{il},$$

(10)

where $i\in (1,\dots ,n)$, $l\in (1,\dots ,N_{{\mathrm{const}}_i})$ and $\mathit{y}\in {\mathbb{R}}^n$. The ${\beta }_{il}$ constant in Equation (10) is the l-th point constraint on the i-th differentiation variable. Furthermore, $t_{il}$ is the time at which the l-th constraint occurs on the i-th differentiation variable. To solve this problem with the TFC, a constrained expression may be written for each i-th differentiation variable,

$$y_i\left(t\right)\approx Y_i\left(t\right)=g_i\left(t\right)+\sum _{l=1}^{N_{{\mathrm{const}}_i}}{\Omega }_{il}\left(t\right){\varrho }_{il}(t_{il},g_i\left(t_{il}\right)),$$

where $N_{{\mathrm{const}}_i}$ represents the number of constraints on the i-th differentiation variable.

The free function, switching functions, and projection functionals can be derived/chosen following the same procedure as in the previous subsection. Likewise, each constrained expression can be discretized to form a system of algebraic equations,

$$\begin{array}{cc}\hfill {\mathcal{L}}_i(\mathbf{\Xi },\mathit{B},t_0,t_f)& =\left\{\begin{array}{c}{\check{f}}_i({\tau }_0,\mathbf{\Xi },\mathit{B},t_0,t_f)\\ ⋮\\ {\check{f}}_i({\tau }_{N-1},\mathbf{\Xi },\mathit{B},t_0,t_f)\end{array}\right\}\hfill \\ \hfill & =\left\{\begin{array}{c}{c}^2\frac{{\mathrm{d}}^2{Y}_i\left({\tau }_0\right)}{{\mathrm{d}}^2\tau }-f_i\left({\tau }_0,\mathit{Y}\left({\tau }_0\right),c\frac{\mathrm{d}\mathit{Y}\left({\tau }_0\right)}{\mathrm{d}\tau }\right)\\ ⋮\\ c\frac{\mathrm{d}{Y}_i\left({\tau }_{N-1}\right)}{\mathrm{d}\tau }-f_i\left({\tau }_{N-1},\mathit{Y}\left({\tau }_{N-1}\right),c^2\frac{{\mathrm{d}}^2\mathit{Y}\left({\tau }_{N-1}\right)}{{\mathrm{d}}^2\tau }\right)\end{array}\right\},\hfill \end{array}$$

(11)

Consider that the ${\mathit{\xi }}_i$ and ${\mathit{\beta }}_i$ variables refer to the vector of Chebyshev coefficients and vector of point constraints associated with the i-th differentiation variable, respectively. Then, the $\mathbf{\Xi }$ and $\mathit{\beta }$ variables in Equation (11) represent the vectors of all Chebyshev coefficient vectors and point constraint vectors appended together as follows:

$$\mathbf{\Xi }={\left\{\begin{array}{ccccc}{\mathit{\xi }}_1^{⊺},& \dots ,& {\mathit{\xi }}_i^{⊺},& \dots ,& {\mathit{\xi }}_n^{⊺}\end{array}\right\}}^{⊺}$$

(12)

and

$$\mathit{B}={\left\{\begin{array}{ccccc}{\mathit{\beta }}_1^{⊺},& \dots ,& {\mathit{\beta }}_i^{⊺},& \dots ,& {\mathit{\beta }}_n^{⊺}\end{array}\right\}}^{⊺}.$$

Note that the lengths of ${\mathit{\xi }}_i$ and ${\mathit{\beta }}_i$ do not have to be identical. Furthermore, $\mathit{Y}\in {\mathbb{R}}^n$ in Equation (11) is a vector of constrained expressions, where the i-th element approximates the i-th differentiation variable. An augmented algebraic system of equations can be formed as

$$\mathbb{L}\left(\mathbf{\Xi }\right)=\left\{\begin{array}{c}{\mathcal{L}}_1\left(\mathbf{\Xi }\right)\\ ⋮\\ {\mathcal{L}}_i\left(\mathbf{\Xi }\right)\\ ⋮\\ {\mathcal{L}}_n\left(\mathbf{\Xi }\right)\end{array}\right\}.$$

We wrote the final form of the loss only in terms of the unknowns, as in the previous subsection. Substituting $\mathit{\xi }$ with $\mathbf{\Xi }$ and $\mathcal{L}\left(\mathit{\xi }\right)$ with $\mathbb{L}\left(\mathbf{\Xi }\right)$ into Equations (8) and (9) then allows $\mathbf{\Xi }$ to be computed via the Gauss–Newton method. If all of the $f_i$ functions are linear, then linear least squares can be used.

2.3. Domain Decomposition

The domain decomposition technique we employ is the same as that presented by Johnston and Mortari [22]. In their work, the authors developed a TFC algorithm that decomposes the domain for solving TPBVPs related to hybrid systems (i.e., systems where a time-sequence of DEs governs the dynamics), along with those that are not but have dynamics that contain transient or stiff behavior. One example is the fuel-optimal planetary landing problem whose control action is bang-bang [18] (i.e., switches between a maximum and minimum, and vice versa). However, the numerical examples provided only involved splitting the domain into two or three segments. This work uses the same procedure when the domain is split into many more segments. How the domain is decomposed is described here for the reader’s convenience. If the reader has any lingering questions, we recommend they read Refs. [18,22].

Assume a TPBVP with one ODE and two boundary conditions, as in Equation (1). To solve this problem with the domain decomposed, separate constrained expressions must be derived for each segment $S_k$ for $k=(1,\dots ,K)$, where K is the total number of segments. Between each segment, $C^1$ continuity must be enforced and computed together with the Chebyshev coefficients. Figure 1 depicts a solution to the TPBVP where the domain is decomposed into multiple segments. Be aware that $Y_t$ in the figure refers to the derivative of Y with respect to t. First, let us talk about the segments at the ends of the domain, i.e., $S_1$ and $S_K$. The initial segment needs a constrained expression that analytically satisfies three constraints: $y\left(t_0\right)=y_0$, $y\left(t_1\right)=y_1$, and $\dot{y}\left(t_1\right)={\dot{y}}_1$. The boundary condition gives the first constraint, $y_0$, while the last two, $y_1$ and ${\dot{y}}_1$, are continuity conditions with the next segment. Similar to the first segment, the last segment’s constraint expression analytically satisfies three constraints as well: $y\left(t_{K-1}\right)=y_{K-1}$, $y\left(t_f\right)=y_f$, and $\dot{y}\left(t_{K-1}\right)={\dot{y}}_{K-1}$. Following the TFC process already introduced, the constrained expression for the first and last segments can be written as

$$\begin{array}{cc}\hfill { }^{\left(1\right)}Y\left(t,{ }^{\left(1\right)}g\left(t\right)\right)={ }^{\left(1\right)}g\left(t\right)& +{ }^{\left(1\right)}{\Omega }_1\left(t\right)\left(y_0-{ }^{\left(1\right)}g\left(t_0\right)\right)\hfill \\ \hfill & +{ }^{\left(1\right)}{\Omega }_2\left(t\right)\left(y_1-{ }^{\left(1\right)}g\left(t_1\right)\right)+{ }^{\left(1\right)}{\Omega }_3\left(t\right)\left({\dot{y}}_1-{ }^{\left(1\right)}\dot{g}\left(t_1\right)\right)\hfill \end{array}$$

and

$$\begin{array}{cc}\hfill { }^{\left(K\right)}Y\left(t,{ }^{\left(K\right)}g\left(t\right)\right)={ }^{\left(K\right)}g\left(t\right)& +{ }^{\left(K\right)}{\Omega }_1\left(t\right)\left(y_{K-1}-{ }^{\left(K\right)}g\left(t_{K-1}\right)\right)\hfill \\ \hfill & +{ }^{\left(K\right)}{\Omega }_2\left(t\right)\left(y_f-{ }^{\left(K\right)}g\left(t_f\right)\right)+{ }^{\left(K\right)}{\Omega }_3\left(t\right)\left({\dot{y}}_{K-1}-{ }^{\left(K\right)}\dot{g}\left(t_{K-1}\right)\right),\hfill \end{array}$$

where the switching functions are

$$\begin{array}{cc}\hfill { }^{\left(1\right)}{\Omega }_1\left(t\right)& =\frac{1}{{\left(t_1-t_0\right)}^2}\left(t_1^2-t_{1}t+t^2\right)\hfill \\ \hfill { }^{\left(1\right)}{\Omega }_2\left(t\right)& =\frac{1}{{\left(t_1-t_0\right)}^2}\left(t_0\left(t_0-2t_1\right)+2t_{1}t-t^2\right)\hfill \\ \hfill { }^{\left(1\right)}{\Omega }_3\left(t\right)& =\frac{1}{t_1-t_0}\left(t_0{t}_1-\left(t_0+t_1\right)t+t^2\right)\hfill \\ \hfill { }^{\left(K\right)}{\Omega }_1\left(t\right)& =\frac{1}{{\left(t_f-t_{K-1}\right)}^2}\left(t_f\left(t_f-2t_{K-1}\right)+2t_{K-1}t-t^2\right)\hfill \\ \hfill { }^{\left(K\right)}{\Omega }_2\left(t\right)& =\frac{1}{{\left(t_f-t_{K-1}\right)}^2}\left(-t_f{t}_{K-1}+\left(t_f+t_{K-1}\right)t-t^2\right)\hfill \\ \hfill { }^{\left(K\right)}{\Omega }_3\left(t\right)& =\frac{1}{t_f-t_{K-1}}\left(t_{K-1}^2-2t_{K-1}t+t^2\right).\hfill \end{array}$$

The switching functions for the first and last segments are different because the continuity conditions are at the end of the first segment and the beginning of the last segment. On the other hand, the constrained expressions for the interior segments, $S_k$ for all $k=(2,\dots ,K-1)$, have constrained expressions with identical switching functions because they always analytically satisfy three continuity constraints at both ends of their domain: $y\left(t_{k-1}\right)=y_{k-1}$, $y\left(t_k\right)=y_k$, $\dot{y}\left(t_{k-1}\right)={\dot{y}}_{k-1}$, and $\dot{y}\left(t_k\right)={\dot{y}}_k$. Unlike the end segments, none of the constraints for the interior segments are associated with the boundary conditions. Thus, the interior segments only have to deal with enforcing continuity. The constrained expressions for these segments can be written as

$$\begin{array}{cc}\hfill { }^{\left(k\right)}y\left(t,{ }^{\left(k\right)}g\left(t\right)\right)={ }^{\left(1\right)}g\left(t\right)& +{ }^{\left(k\right)}{\Omega }_1\left(t\right)\left(y_k-{ }^{\left(k\right)}g\left(t_{k-1}\right)\right)+{ }^{\left(k\right)}{\Omega }_2\left(t\right)\left(y_k-{ }^{\left(k\right)}g\left(t_k\right)\right)\hfill \\ \hfill & +{ }^{\left(k\right)}{\Omega }_3\left(t\right)\left({\dot{y}}_{k-1}-{ }^{\left(k\right)}\dot{g}\left(t_{k-1}\right)\right)+{ }^{\left(k\right)}{\Omega }_4\left(t\right)\left({\dot{y}}_k-{ }^{\left(k\right)}\dot{g}\left(t_k\right)\right)\hfill \end{array}$$

where the switching functions are

$$\begin{array}{cc}\hfill { }^{\left(k\right)}{\Omega }_1\left(t\right)& =\frac{1}{{\left(t_k-t_{k-1}\right)}^3}\left(-t_k^2\left(2t_{k-1}-t_k\right)+6t_{k-1}{t}_{k}t-3\left(t_{k-1}+t_k\right)t^2+2t^3\right)\hfill \\ \hfill { }^{\left(k\right)}{\Omega }_2\left(t\right)& =\frac{1}{{\left(t_k-t_{k-1}\right)}^3}\left(-t_{k-1}^2\left(t_{k-1}-3t_k\right)-6t_{k-1}{t}_{k}t+3\left(t_{k-1}+t_k\right)t^2-2t^3\right)\hfill \\ \hfill { }^{\left(k\right)}{\Omega }_3\left(t\right)& =\frac{1}{{\left(t_k-t_{k-1}\right)}^2}\left(-t_{k-1}{t}_k^2+t_k\left(2t_{k-1}+t_k\right)t-\left(t_{k-1}+2t_k\right)t^2+t^3\right)\hfill \\ \hfill { }^{\left(k\right)}{\Omega }_4\left(t\right)& =\frac{1}{{\left(t_k-t_{k-1}\right)}^2}\left(-t_{k-1}^2{t}_k+t_{k-1}\left(t_{k-1}+2t_k\right)t-\left(2t_{k-1}+t_k\right)t^2+t^3\right).\hfill \end{array}$$

Now, assume the TPBVP is composed of n ODEs. The free functions within the various constrained expressions can be expressed as truncated Chebyshev polynomials on the domain $\tau \in [-1,1]$. Using Equations (4) and (5), we have

$${ }^{\left(k\right)}{g}_i\left(t\right)={ }^{\left(k\right)}{g}_i\left(\tau \right),\frac{\mathrm{d}{ }^{\left(k\right)}{g}_i\left(\tau \right)}{\mathrm{d}t}={ }^{\left(k\right)}c\frac{\mathrm{d}{ }^{\left(k\right)}{g}_i\left(\tau \right)}{\mathrm{d}\tau },\mathrm{and}{c}_k=\frac{2}{t_k-t_{k-1}}.$$

Then, by discretizing the domains, the augmented loss vector that combines the loss vectors for each differentiation variable in each segment can be written as

$${ }^{\left(k\right)}\mathbb{L}\left({ }^{\left(k\right)}\mathbf{\Xi },{ }^{\left(k\right)}\mathit{B},t_{k-1},t_k\right)=\left\{\begin{array}{c}{ }^{\left(k\right)}{\mathcal{L}}_1\left({ }^{\left(k\right)}\mathbf{\Xi },{ }^{\left(k\right)}\mathit{B},t_{k-1},t_k\right)\\ ⋮\\ { }^{\left(k\right)}{\mathcal{L}}_i\left({ }^{\left(k\right)}\mathbf{\Xi },{ }^{\left(k\right)}\mathit{B},t_{k-1},t_k\right)\\ ⋮\\ { }^{\left(k\right)}{\mathcal{L}}_n\left({ }^{\left(k\right)}\mathbf{\Xi },{ }^{\left(k\right)}\mathit{B},t_{k-1},t_k\right)\end{array}\right\}$$

where

$$\begin{array}{cc}\hfill { }^{\left(k\right)}{\mathcal{L}}_i\left({ }^{\left(k\right)}\mathbf{\Xi },{ }^{\left(k\right)}\mathit{B},t_{k-1},t_k\right)& =\left\{\begin{array}{c}{\check{f}}_i\left({\tau }_0,{ }^{\left(k\right)}\mathbf{\Xi },{ }^{\left(k\right)}\mathit{B},t_{k-1},t_k\right)\\ ⋮\\ {\check{f}}_i\left({\tau }_{N-1},{ }^{\left(k\right)}\mathbf{\Xi },{ }^{\left(k\right)}\mathit{B},t_{k-1},t_k\right)\end{array}\right\}\hfill \\ \hfill & =\left\{\begin{array}{c}{c}_k^2\frac{{\mathrm{d}}^2{ }^{\left(k\right)}{Y}_i\left({\tau }_0\right)}{\mathrm{d}{\tau }^2}-f_i\left({\tau }_0,{ }^{\left(k\right)}\mathit{Y}\left({\tau }_0\right),c_k\frac{\mathrm{d}{ }^{\left(k\right)}\mathit{Y}\left({\tau }_0\right)}{\mathrm{d}\tau }\right)\\ ⋮\\ c_k^2\frac{{\mathrm{d}}^2{ }^{\left(k\right)}{Y}_i\left({\tau }_{N-1}\right)}{\mathrm{d}{\tau }^2}-f_i\left({\tau }_{N-1},{ }^{\left(k\right)}\mathit{Y}\left({\tau }_{N-1}\right),c_k\frac{\mathrm{d}{ }^{\left(k\right)}\mathit{Y}\left({\tau }_{N-1}\right)}{\mathrm{d}\tau }\right)\end{array}\right\}.\hfill \end{array}$$

The unknown variable ${ }^{\left(k\right)}\mathbf{\Xi }$ represents the k-th segment’s appended unknown Chebyshev coefficient vector,

$${ }^{\left(k\right)}\mathbf{\Xi }={\left\{\begin{array}{ccccc}{ }^{\left(k\right)}{\mathit{\xi }}_1^{⊺},& \dots ,& { }^{\left(k\right)}{\mathit{\xi }}_i^{⊺},& \dots ,& { }^{\left(k\right)}{\mathit{\xi }}_n^{⊺}\end{array}\right\}}^{⊺}.$$

Likewise, ${ }^{\left(k\right)}\mathit{B}$ refers to the augmented vector of point constraints for the k-th segment and can be written as

$${ }^{\left(k\right)}\mathit{B}={\left\{\begin{array}{ccccc}{ }^{\left(k\right)}{\mathit{\beta }}_1^{⊺},& \dots ,& { }^{\left(k\right)}{\mathit{\beta }}_i^{⊺},& \dots ,& { }^{\left(k\right)}{\mathit{\beta }}_n^{⊺}\end{array}\right\}}^{⊺}.$$

Not all point constraints are known in constrained expressions that approximate only a segment of the domain. Only the initial conditions on $S_1$ and final conditions on $S_K$ are known. All continuity conditions within a k-th segment and for an i-th differentiation variable are unknown and are represented by ${ }^{\left(k\right)}{\hat{\mathit{\beta }}}_i$. Grouping the continuity conditions in a segment gives

$${ }^{\left(k\right)}\hat{\mathit{B}}={\left\{\begin{array}{ccccc}{ }^{\left(k\right)}{\widehat{{\mathit{\beta }}_1}}^{⊺},& \dots ,& { }^{\left(k\right)}{\widehat{{\mathit{\beta }}_i}}^{⊺},& \dots ,& { }^{\left(k\right)}{\widehat{{\mathit{\beta }}_n}}^{⊺}\end{array}\right\}}^{⊺}.$$

Likewise, grouping the continuity conditions in all segments gives

$$\widehat{\mathit{B}}=\mathrm{unique}\left({\left\{\begin{array}{ccccc}{ }^{\left(1\right)}{\hat{\mathit{B}}}^{⊺},& \dots ,& { }^{\left(k\right)}{\hat{\mathit{B}}}^{⊺},& \dots ,& { }^{\left(K\right)}{\hat{\mathit{B}}}^{⊺}\end{array}\right\}}^{⊺}\right),$$

where $\mathrm{unique}\left(·\right)$ means only consider the unique elements of the $·$ vector. All unknowns can then be grouped to form

$$\mathit{Z}={\left\{\begin{array}{ccccc}{ }^{\left(1\right)}{\mathbf{\Xi }}^{⊺},& \dots ,& { }^{\left(k\right)}{\mathbf{\Xi }}^{⊺},& \dots ,& { }^{\left(K\right)}{\mathbf{\Xi }}^{⊺},{\hat{\mathit{B}}}^{⊺}\end{array}\right\}}^{⊺}.$$

Exactly how $\hat{\mathit{B}}$ can be built for a specific TPBVP is shown in Appendix A.

The complete augmented loss vector that contains all segments and forms the system of algebraic equations to be solved is

$$\mathbb{L}\left(\mathit{Z}\right)=\left\{\begin{array}{c}{ }^{\left(1\right)}\mathbb{L}\left(\mathit{Z}\right)\\ ⋮\\ { }^{\left(k\right)}\mathbb{L}\left(\mathit{Z}\right)\\ ⋮\\ { }^{\left(K\right)}\mathbb{L}\left(\mathit{Z}\right)\end{array}\right\}.$$

Once again, $\mathbb{L}$ is only explicitly written in terms of $\mathit{Z}$ because the boundary conditions within ${ }^{\left(1\right)}\mathit{B}$ and ${ }^{\left(K\right)}\mathit{B}$ are known, and so too are $t_k$ for all $k=(1,\dots ,K)$. The unknown $\mathit{Z}$ values can be computed using linear least squares when every $f_i$ is linear or the Gauss–Newton algorithm when any $f_i$ is nonlinear. The $\mathbb{L}$ algebraic system will be taken in a block diagonal form due to its segment-wise construction, allowing efficient algorithms for computing sparse matrices to be exploited.

3. Error Estimation

The most critical aspect of an hp-adaptive mesh refinement algorithm is how it relates the approximation error to when the order of the approximating polynomial should be increased (p-adaptive mesh refinement) or when an approximating polynomial should be divided into more segments (h-adaptive mesh refinement). For the TFC, increasing the approximating polynomial order corresponds to adding more basis functions to the constrained expression’s free function (a truncated expansion of Chebyshev polynomials). In the previous section, it was shown that the TFC minimizes ${\parallel \mathbb{L}\parallel }_{\infty }$ to obtain an accurate solution. However, a criterion for when to increase the number of basis functions in a constrained expression or split the constrained expression into multiple segments can be made by examining the constrained expression’s truncation error. Therefore, that criterion is presented in this section, along with an error analysis of a few problems to demonstrate that a low truncation error corresponds with a small ${\parallel \mathbb{L}\parallel }_{\infty }$.

3.1. Truncation Error of the Constrained Expression

Let $y\left(\tau \right)$ be a piecewise smooth bounded function on the interval $\tau \in [-1,+1]$ that has $N_{\mathrm{const}}$ point constraints at ${\tau }_l$ with $l=\left(1,\dots ,N_{\mathrm{const}}\right)$. Then, $y\left(\tau \right)$ can be approximated with a constrained expression $Y\left(\tau \right)$, where the free function is a linear combination of L Chebyshev polynomials, ${\mathcal{T}}_j$ for all $j=(N_{\mathrm{const}},\dots ,L+N_{\mathrm{const}}-1)$. Assuming point constraints, the projection functionals within the constrained expression take the form

$${\varrho }_j\left({\tau }_l\right)=y\left({\tau }_l\right)-\sum _{j=N_{\mathrm{const}}}^{L+N_{\mathrm{const}}-1}{\xi }_j{\mathcal{T}}_j\left({\tau }_l\right).$$

Hence, the approximation (constrained expression) takes the form

$$y\left(\tau \right)\approx Y\left(\tau \right)=\sum _{j=N_{\mathrm{const}}}^{L+N_{\mathrm{const}}-1}{\xi }_j{\mathcal{T}}_j\left(\tau \right)+\sum _{l=1}^{N_{\mathrm{const}}}{\Omega }_l\left(\tau \right)\left(y\left({\tau }_l\right)-\sum _{j=N_{\mathrm{const}}}^{L+N_{\mathrm{const}}-1}{\xi }_j{\mathcal{T}}_j\left({\tau }_l\right)\right),$$

(13)

where ${\Omega }_l$ is the switching function for the l-th constraint. Since $y\left(\tau \right)$ is piecewise smooth and bounded, it can be represented exactly by an infinite Chebyshev polynomial series,

$$y\left(\tau \right)=\sum _{j=0}^{\infty }{a}_j{\mathcal{T}}_j\left(\tau \right)=\sum _{j=0}^{N_{\mathrm{const}}-1}{a}_j{\mathcal{T}}_j\left(\tau \right)+\sum _{j=N_{\mathrm{const}}}^{L+N_{\mathrm{const}}-1}{a}_j{\mathcal{T}}_j\left(\tau \right)+\sum _{j=L+N_{\mathrm{const}}}^{\infty }{a}_j{\mathcal{T}}_j\left(\tau \right),$$

where $a_j$ for all $j=(0,\dots ,\infty )$ are the Chebyshev coefficients of the infinite series. The Euclidean norm, otherwise called the ${\mathrm{L}}_2$-norm, of the error between $y\left(\tau \right)$ and $Y\left(\tau \right)$ satisfies the following inequality:

$$\begin{array}{cc}\hfill e& ={\parallel y\left(\tau \right)-Y\left(\tau \right)\parallel }_2\hfill \\ \hfill & ={∥\sum _{j=0}^{\infty }{a}_j{\mathcal{T}}_j\left(\tau \right)-\left(\sum _{j=N_{\mathrm{const}}}^{L+N_{\mathrm{const}}-1}{\xi }_j{\mathcal{T}}_j\left(\tau \right)+\sum _{l=1}^{N_{\mathrm{const}}}{\Omega }_l\left(\tau \right)\left(y\left({\tau }_l\right)-\sum _{j=N_{\mathrm{const}}}^{L+N_{\mathrm{const}}-1}{\xi }_j{\mathcal{T}}_j\left({\tau }_l\right)\right)\right)∥}_2\hfill \\ \hfill & \le e_{\mathrm{T}}+e_{\mathrm{A}}+e_{\mathrm{C}},\hfill \end{array}$$

where $e_{\mathrm{T}}$ is the truncation error, $e_{\mathrm{A}}$ is the aliasing error, and $e_{\mathrm{C}}$ is what we call the constraint error. They are given as

$$e_{\mathrm{T}}={∥\sum _{j=L+N_{\mathrm{const}}}^{\infty }{a}_j{\mathcal{T}}_j\left(\tau \right)∥}_2,$$

(14)

$$e_{\mathrm{A}}={∥\sum _{j=N_{\mathrm{const}}}^{L+N_{\mathrm{const}}-1}\left(a_j-{\xi }_j\right){\mathcal{T}}_j\left(\tau \right)∥}_2,$$

and

$$e_{\mathrm{C}}={∥\sum _{j=0}^{N_{\mathrm{const}}-1}{a}_j{\mathcal{T}}_j\left(\tau \right)-\sum _{l=1}^{N_{\mathrm{const}}}{\Omega }_l\left(\tau \right)\left(y\left({\tau }_l\right)-\sum _{j=N_{\mathrm{const}}}^{L+N_{\mathrm{const}}-1}{\xi }_j{\mathcal{T}}_j\left({\tau }_l\right)\right)∥}_2.$$

The aliasing and constraint errors can only be calculated if the function being approximated is known. A critical conclusion about the magnitude of the truncation error can be made from Rule of Thumb 1, as presented by Boyd [32] (p. 32):

Rule of Thumb 1. 

$e_{\mathrm{A}}$ will be of the same order of magnitude as $e_{\mathrm{T}}$.

Furthermore, through testing, we found that the constraint error is of a similar magnitude to the aliasing error. Thus, if the truncation error is low, so too should be the aliasing and constraint errors.

Note that ${∥f∥}_2={\langle f,f\rangle }^{1/2}$ is the norm induced by the inner product

$$\langle f,g\rangle ={\int }_{-1}^{1}f\left(\tau \right)g\left(\tau \right)\mathrm{d}\tau .$$

(15)

Using the definition of the inner product in Equation (15), the orthogonal property of Chebyshev polynomials is

$$〈{\mathcal{T}}_m,{\mathcal{T}}_n〉={\int }_{-1}^1{\mathcal{T}}_m\left(\tau \right){\mathcal{T}}_n\left(\tau \right){\left(1-{\tau }^2\right)}^{-1/2}\mathrm{d}\tau =\frac{\pi }{2}{\delta }_{mn},$$

(16)

where ${\delta }_{mn}$ is the Kronecker delta function. Multiplying ${\left(1-{\tau }^2\right)}^{-1/4}$ by Equation (14) and applying the absolute homogeneity of norms gives

$$\begin{array}{cc}\hfill {\left(1-{\tau }^2\right)}^{-1/4}{e}_{\mathrm{T}}& ={\left(1-{\tau }^2\right)}^{-1/4}{∥\sum _{j=L+N_{\mathrm{const}}}^{\infty }{a}_j{\mathcal{T}}_j\left(\tau \right)∥}_2\hfill \\ \hfill & ={∥\sum _{j=L+N_{\mathrm{const}}}^{\infty }{a}_j{\mathcal{T}}_j\left(\tau \right){\left(1-{\tau }^2\right)}^{-1/4}∥}_2.\hfill \end{array}$$

(17)

Expanding the norm in Equation (17), substituting in Equation (15), and plugging in Equation (16) gives a new expression for $e_{\mathrm{T}}$,

$$\begin{array}{cc}\hfill e_{\mathrm{T}}& ={\left(1-{\tau }^2\right)}^{1/4}{∥\sum _{j=L+N_{\mathrm{const}}}^{\infty }{a}_j{\mathcal{T}}_j\left(\tau \right){\left(1-{\tau }^2\right)}^{-1/4}∥}_2\hfill \\ \hfill & ={\left(1-{\tau }^2\right)}^{1/4}{\left[\sum _{j=L+N_{\mathrm{const}}}^{\infty }{\int }_{-1}^1{a}_j^2{\mathcal{T}}_j^2\left(\tau \right){\left(1-{\tau }^2\right)}^{-1/2}\mathrm{d}\tau \right]}^{1/2}\hfill \\ \hfill & ={\left(1-{\tau }^2\right)}^{1/4}{\left[\sum _{j=L+N_{\mathrm{const}}}^{\infty }{a}_j^2{\int }_{-1}^1{\mathcal{T}}_j^2\left(\tau \right){\left(1-{\tau }^2\right)}^{-1/2}\mathrm{d}\tau \right]}^{1/2}\hfill \\ \hfill & ={\left(\frac{\pi }{2}\right)}^{1/2}{\left(1-{\tau }^2\right)}^{1/4}{\left[\sum _{j=L+N_{\mathrm{const}}}^{\infty }{a}_j^2\right]}^{1/2}.\hfill \end{array}$$

(18)

It has been shown that the Chebyshev series has super-geometric convergence when $y\left(\tau \right)$ is analytic in the neighborhood of $\tau \in [-1,+1]$ [32,33]. Hence, the Chebyshev coefficient values from the infinite expansion $a_j$ for all $j=(0,\dots ,\infty )$ decay as

$$\left|a_j\right|\sim \mathcal{O}\left(s{e}^{-\tilde{q}{j}^r}\right),$$

with constants $s,\tilde{q}>0$, and $r>1$. Following in the spirit of Lie et al. [27], we note that a slightly smaller value $q>0$ exists, such that

$$s{e}^{-\tilde{q}{j}^r}\le \left(s-\underset{j}{min}\left(s{e}^{-qj}-\left|a_j\right|\right)\right)e^{-qj}.$$

(19)

The exponential upper bound is more convenient because it allows the Chebyshev coefficients to be approximated with an exponential least-squares fit. The minimum in Equation (19) represents a translation that ensures every value of $a_j$ is below the bound. Since the solution of the problem being solved is often not known, the $\left|a_j\right|$ values cannot be computed to perform the regression, but the truncated coefficients from the constrained expression $\left|{\xi }_j\right|$ can. Based on Rule of Thumb 1, $a_j\approx {\xi }_j$ when the truncation error is low. Thus, simply replace $a_j$ with ${\xi }_j$ in Equation (19) to obtain an upper bound estimate for $a_j$ for all $j=(N_{\mathrm{const}},\dots ,L+N_{\mathrm{const}}-1)$. Extrapolation can then be used to obtain an upper bound for $j=(0,\dots ,\infty )$. As a result, in this work, the upper bound for $\left|a_j\right|$ is approximated by

$$ln\left(\left|a_j\right|\right)\approx ln\left(\left|{\xi }_j\right|\right)\le ln\left(s\right)-\delta -qj,$$

(20)

with $s,q>0$, $j=(N_{\mathrm{const}},\dots ,L+N_{\mathrm{const}}-1)$, and 

$$\delta =\underset{j}{min}\left(ln\left(s\right)-qj-ln\left(\left|{\xi }_j\right|\right)\right).$$

The s and q coefficients are obtained by performing an exponential least-squares fit on $\left|{\xi }_j\right|$,

$$ln\left(\left|{\xi }_j\right|\right)=ln\left(s\right)-qj.$$

(21)

Note that q represents the exponential decay of the Chebyshev coefficients. Furthermore, an upper bound of $e_{\mathrm{T}}$ can be derived by plugging Equation (20) into (18),

$$\begin{array}{cc}\hfill e_{\mathrm{T}}& \le {\left(\frac{\pi }{2}\right)}^{1/2}{\left(1-{\tau }^2\right)}^{1/4}{\left[\sum _{j=L+N_{\mathrm{const}}}^{\infty }{e}^{2\left(ln\left(s\right)-\delta -qj\right)}\right]}^{1/2}\hfill \\ \hfill & \le {\left(\frac{\pi }{2}\right)}^{1/2}{\left(1-{\tau }^2\right)}^{1/4}{\left[\sum _{j=L+N_{\mathrm{const}}}^{\infty }\left(s^2{e}^{-2\delta }\right)e^{-2qj}\right]}^{1/2}.\hfill \end{array}$$

(22)

The summation in Equation (22) is a geometric series with the common ratio $e^{-2q}$,

$$\sum _{j=L+N_{\mathrm{const}}}^{\infty }\left(s^2{e}^{-2\delta }\right)e^{-2qj}=\frac{\left(s^2{e}^{-2\delta }\right)e^{-2q\left(L+N_{\mathrm{const}}\right)}}{1-e^{-2q}}.$$

Thus, Equation (22) becomes

$$\begin{array}{cc}\hfill e_{\mathrm{T}}& \le {\left(\frac{\pi }{2}\right)}^{1/2}{\left(1-{\tau }^2\right)}^{1/4}{\left[\frac{\left(s^2{e}^{-2\delta }\right)e^{-2q\left(L+N_{\mathrm{const}}\right)}}{1-e^{-2q}}\right]}^{1/2}\hfill \\ \hfill & \le \frac{{\pi }^{1/2}{\left(1-{\tau }^2\right)}^{1/4}\left(s{e}^{-\delta }\right)e^{-q\left(L+N_{\mathrm{const}}\right)}}{{\left(2\left(1-e^{-2q}\right)\right)}^{1/2}}\hfill \\ \hfill & \le \frac{{\pi }^{1/2}s{e}^{\left(-\delta -q\left(L+N_{\mathrm{const}}\right)\right)}}{{\left(2\left(1-e^{-2q}\right)\right)}^{1/2}}={\hat{e}}_{\mathrm{T}}.\hfill \end{array}$$

(23)

From Equations (20) and (23), one can see that the coefficients $\left|{\xi }_j\right|$ and the approximate truncation error upper bound ${\hat{e}}_{\mathrm{T}}$ decrease at the same exponential rate q, a result similar to that seen in Liu et al. [27]. Consequently, the Chebyshev coefficients ${\xi }_j$ for all $j=(N_{\mathrm{const}},\dots ,L+N_{\mathrm{const}}-1)$ can be used to estimate the decay rate, q, of the truncation error as a function of the free function’s Chebyshev polynomial degree given in Equation (13).

Before moving on, the reader may have noticed that the ${\mathrm{L}}_2$-norm is used to derive the analytical expression for the truncation error (see Equation (14)). The ${\mathrm{L}}_2$-norm had to be used to derive the expression using the orthogonal property of Chebyshev polynomials (Equation (16)). On the other hand, the accuracy of the TFC is quantified by taking the ${\mathrm{L}}_{\infty }$-norm of the losses (see Equation (9)). In general, the ${\mathrm{L}}_2$-norm of the loss can also be used to quantify the accuracy of the TFC, but we chose to use the ${\mathrm{L}}_{\infty }$-norm in order to align with the only textbook on the TFC [1]. Although the ${\mathrm{L}}_2$-norm of the truncation error and the ${\mathrm{L}}_{\infty }$-norm of the loss are two separate measurements, for one to be used for hp-adaptive mesh refinement and the other used to be used as a stopping criterion, it is required that both correlate. When one decreases, so must the other, and vice versa. This correlation is demonstrated in Section 3.3 and Section 3.4.

3.2. Determining Function Smoothness

How to determine when to add basis functions to a constrained expression within a segment or split it up is a critical aspect of our mesh refinement algorithm. In the same vein as Liu et al. [27], this decision can be made based on the decay rate of the ${\xi }_j$ coefficients and the truncation error, i.e., the q value from Equations (21) and (23). The main distinction between our work and [27] is that our decay rate comes from an exponential fit on the truncated Chebyshev expansion coefficients within an approximating constrained expression, whereas the decay rate from [27] comes from an exponential fit on the truncated Legendre expansion coefficients computed from an approximating Lagrange polynomial. Similar to [27], the function is smooth enough to be approximated with a single polynomial if q is large. In other words, only basis functions need to be added to bring the error down to some desired tolerance. If q is sufficiently small, then the function is not smooth enough to be approximated with a single polynomial, and a piecewise polynomial should be employed to approximate it. The cutoff decay rate value $\overline{q}$, which decides when to add basis functions or split a segment, is a design choice determined by the engineer. When $q\ge \overline{q}$, basis functions should be added to the free function of the constrained expression to improve the approximation. On the other hand, when $q<\overline{q}$, the segment the constrained expression approximates should be divided to improve the approximation. In the next two subsections, error analysis studies are carried out on a normal TPBVP with a smooth solution and a hybrid TPBVP with a nonsmooth solution to demonstrate agreement between the decay rate of the ${\xi }_j$ coefficients and the truncation error’s maximum upper bound ${\hat{e}}_{\mathrm{T}}$, as well as determine how they correspond to the aliasing error, constraint error, actual error, and ${\mathrm{L}}_{\infty }$-norm of the loss.

3.3. Case 1 Error Analysis: TPBVP with a Smooth Solution

Consider the following TPBVP:

$$2{\ddot{y}}_1\left(t\right)-y_1\left(t\right)=4sin\left(2t\right),\mathrm{s}.\mathrm{t}.\left\{\begin{array}{c}y\left(0\right)=0\hfill \\ y\left(1\right)=0\hfill \end{array}\right.,$$

(24)

which has the analytical solution

$$y_1\left(t\right)=c_1{e}^{t/\sqrt{2}}+c_2{e}^{-t/\sqrt{2}}-\frac{4}{19}sin\left(3t\right),$$

where

$$c_1=-\frac{4e^{1/\sqrt{2}}sin\left(3\right)}{19\left(-e^{\sqrt{2}}+1\right)}$$

and

$$c_2=\frac{4e^{1/\sqrt{2}}sin\left(3\right)}{19\left(-e^{\sqrt{2}}+1\right)}.$$

It can be seen that $y_1\left(t\right)$ is a smooth function. Using the TFC procedure outlined in Section 2, Equation (24) can be solved by approximating $y_1$ as a constrained expression with the truncated Chebyshev coefficients ${\xi }_j$. Using the computed ${\xi }_j$ values via the TFC, a least-squares exponential fit of the form shown in Equation (21) can be constructed. Equation (20) can then be used to build an upper bound on the ${\xi }_j$ values. Subsequently, an upper bound on the truncation error ${\hat{e}}_{\mathrm{T}}$ can be computed, as shown in Equation (23). The aliasing error $e_{\mathrm{A}}$ and the constraint error $e_{\mathrm{C}}$ can also be computed because the true solution is known. Only the infinite Chebyshev expansion coefficients $a_j$ have to be computed, which we performed using the open-source MATLAB toolbox Chebfun [34]. Figure 2 shows how the actual error; two variants of ${\hat{e}}_{\mathrm{T}}$, $e_{\mathrm{A}}$, $e_{\mathrm{C}}$; two estimated error bounds (e = ${\hat{e}}_{\mathrm{T}}+e_{\mathrm{A}}+e_{\mathrm{C}}$); the ${\mathrm{L}}_{\infty }$-norm of the loss shown in Equation (6); and the final Chebyshev coefficient ${\xi }_{L+N_{\mathrm{const}}-1}$ changed as the number of Chebyshev basis functions L increased. The number of collocation points N was set equal to $L+3$. Note that the results in Figure 2 correspond to Rule of Thumb 2 from Boyd [32] (p. 51), which states the following:

Rule of Thumb 2. 

The truncation error is the same order of magnitude as the last coefficient retained in the truncation for a series with geometric convergence.

For $L\ge 15$, the ${\xi }_{L+N_{\mathrm{const}}-1}$ coefficients are so low in value that they are negligible. Hence, the accuracy of the solution will not improve by adding more than 15 basis functions to the Chebyshev polynomial. This can be observed by following the ${\parallel \mathbb{L}\parallel }_{\infty }$ and e curves in Figure 2. All coefficients with $j>L_R$, where $L_R=15$, are called the roundoff plateau [32] (pp. 30–31) because their values are less than the machine-level error, which is $1\times {10}^{-16}$. Since the inclusion of Chebyshev series terms with their coefficients on the roundoff plateau has no effect on the accuracy of the solution, they do not have to be included in the exponential regression. Not including them results in an improved upper bound (IUB) for the non-negligible ${\xi }_j$ coefficients, i.e., those that are off the roundoff plateau. Including the ${\xi }_j$ coefficients that are on the roundoff plateau in the exponential regression results in a conservative upper bound (CUB) for them. The resulting truncation errors from using a CUB or IUB on the ${\xi }_j$ coefficients are referred to as CUB ${\hat{e}}_{\mathrm{T}}$ and IUB ${\hat{e}}_{\mathrm{T}}$ in Figure 2. The CUB and IUB ${\hat{e}}_{\mathrm{T}}$ curves are practically identical until $L>L_R$. The CUB causes ${\hat{e}}_{\mathrm{T}}$ to level out, while the IUB causes ${\hat{e}}_{\mathrm{T}}$ to keep decreasing for larger values of L. One of the main takeaways from Figure 2 is that it validates the truncation error bound because the actual error is always below the estimated total error bounds with a CUB and IUB ${\hat{e}}_{\mathrm{T}}$. Figure 2 also shows that when ${\hat{e}}_{\mathrm{T}}$ is low, so too is ${\parallel \mathbb{L}\parallel }_{\infty }$. Lastly, both truncation errors follow ${\xi }_{L+N_{\mathrm{const}}-1}$, demonstrating that the decay rate of the ${\xi }_j$ coefficients is the same as the truncation error for a smooth analytic function.

Figure 3 aids in the analysis of the exponential regression performed on the ${\xi }_j$ values computed via the TFC for solving Equation (24). Subplot (a) shows the ${\xi }_j$ coefficients for $j=(2,\dots ,L)$ when L = 30, the exponential fit performed on all coefficients (Exp. Fit 1), the exponential fit performed on only the coefficients of the roundoff plateau (Exp. Fit 2), the CUB, and the IUB. From the subplot, one can clearly see that the IUB bounds the ${\xi }_j$ coefficients off the roundoff plateau tighter than the CUB. Furthermore, it is clear that geometric convergence is maintained until the roundoff plateau is reached. Subplot (b) shows the goodness of the exponential fit according to the coefficient of determination $R^2$ as L increases. $R^2$ can be expressed as

$$R^2=1-\frac{{\sum }_{j=N_{\mathrm{const}}}^{L+N_{\mathrm{const}}}{\left(\left|{\xi }_j\right|-f_j\right)}^2}{{\sum }_{i=N_{\mathrm{const}}}^{L+N_{\mathrm{const}}}{\left(\left|{\xi }_j\right|-\mathrm{mean}\left(\xi \right)\right)}^2},$$

where $f_j$ is the value of the fit for the j-th coefficient and

$$\mathrm{mean}\left(\xi \right)=\frac{1}{L+1}\sum _{i=N_{\mathrm{const}}}^{L+N_{\mathrm{const}}}\left|{\xi }_j\right|.$$

From subplot (a), one can see that the exponential fit is not very good when all ${\xi }_j$ coefficients are included in the regression. This is confirmed by subplot (b) because $R^2$ for Exp. Fit 1 moves further from 1 when more ${\xi }_j$ coefficients with $j>L_R$ are included in the regression. However, when only the ${\xi }_j$ coefficients off the roundoff plateau are fitted, $R^2\approx 1$ as j increases. Subplot (c) shows the decay rates q computed via Exp. Fit 1 and Exp. Fit 2 as L increases. One can see that all decay rates are greater than 1.5, which is emblematic of the $y_1$ function’s smoothness. The different decay rates are identical until $L=L_{\mathrm{RP}}=15$. The Exp. Fit 2 decay rates for $L>L_{\mathrm{RP}}$ are constant because the ${\xi }_j$ coefficients included in the fit are only for $j=2,\dots ,L_{\mathrm{RP}}$. One can see that the Exp. Fit 1 decay rates are greater than those for the Exp. Fit 2 directly after $L_{\mathrm{RP}}$ is reached but then start to decrease quickly as L is further increased. Hence, when $L=50$, the decay rate for Exp. Fit 1 could be close to 0. This means that a smooth function could have a very low decay rate if the round-off plateau is included in the exponential regression of the ${\xi }_j$ coefficients. Therefore, Exp. Fit 2 is preferred over Exp. Fit 1.

3.4. Case 2 Error Analysis: Hybrid TPBVP with a Nonsmooth Solution

Consider the hybrid TPBVP

$$\ddot{y}\left(t\right)=t^2+a,\mathrm{subject}\mathrm{to}:\left\{\begin{array}{c}y\left(0\right)=0\hfill \\ y\left(1\right)=0\hfill \end{array}\right.\mathrm{where}:\left\{\begin{array}{c}a=0\mathrm{for}t\le 0.5\hfill \\ a=1\mathrm{for}t>0.5\hfill \end{array}\right..$$

(25)

which has a discontinuous solution. We instead find a smoothed solution by transforming Equation (25) into

$${\ddot{y}}_2\left(t\right)=t^2+1\left(\frac{1}{2}-\frac{1}{2}tanh\left(\frac{\gamma -t}{\alpha }\right)\right),\mathrm{subject}\mathrm{to}:\left\{\begin{array}{c}y\left(0\right)=0\hfill \\ y\left(1\right)=0\hfill \end{array}\right.,$$

(26)

which has the analytical solution

$$y_2\left(t\right)=\frac{1}{12}\left(-3{\alpha }^2{\mathrm{Li}}_2\left(-e^{\frac{-2\left(\gamma -t\right)}{\alpha }}\right)+6\alpha ln\left(2\right)\left(\gamma -t\right)-3{\gamma }^2+6\gamma t+t^4\right)+c_{1}t+c2,$$

where

$$c_1=\frac{1}{4}{\alpha }^2{\mathrm{Li}}_2\left(-e^{\frac{-2\left(\gamma -1\right)}{\alpha }}\right)-\frac{1}{4}{\alpha }^2{\mathrm{Li}}_2\left(-e^{\frac{-2\gamma }{\alpha }}\right)-\frac{\gamma }{2}+\frac{\alpha ln\left(2\right)}{2}+\frac{11}{12},$$

$$c_2=\frac{1}{4}{\alpha }^2{\mathrm{Li}}_2\left(-e^{\frac{-2\gamma }{\alpha }}\right)-\frac{\gamma \alpha ln\left(2\right)}{2}+\frac{{\gamma }^2}{4},$$

and

$${\mathrm{Li}}_2\left(x\right)={\int }_2^x\frac{\mathrm{d}\tau }{ln\left(\tau \right)}.$$

The $\gamma$ variable represents the discontinuous point, which is set to $\gamma =0.5$. The $\alpha$ variable determines how smooth the region around $t=\gamma$ is. The closer $\alpha$ is to 0, the sharper the gradient at $t=\gamma$, and the closer the solution of Equation (26) to Equation (25). Thus, we set $\alpha =1.5\times {10}^{-3}$. An error analysis study, similar to that shown in Section 3.3, was then performed. The main difference from the previous study is that $y_2$ is nearly not smooth. This results in the ${\xi }_j$ coefficients, computed via the TFC, oscillating severely between local maximums and local minimums. Hence, in this study, Exp. Fit 2 refers to performing an exponential regression on only the local maximums of the ${\xi }_j$ coefficients. Exp. Fit 1 refers to performing an exponential regression on all ${\xi }_j$ coefficients, just like in Section 3.3. The resulting CUB and IUB are from using Equation (20) with Exp. Fit 1 and Exp. Fit 2, respectively.

Error curves of the TFC solutions to Equation (26) as L increases are shown in Figure 4. Compared with Figure 2, many more basis functions are required for the actual error to converge. Furthermore, the ${\mathrm{L}}_{\infty }$-norm of the loss is still decreasing and greater than the actual error. The reason for this is that the sharp gradient that replaces the discontinuity in Equation (25) appears for ${\ddot{y}}_2$, which contributes to the loss equation, while the actual error is only for the constrained expression of $y_2$. Another important observation from Figure 4 is that both truncation error bounds are well above the actual error, meaning they do not accurately represent the total error. Rule of Thumb 1, as described in Section 3.1, does not seem to apply. However, this makes sense because our selection of $\alpha =1.5\times {10}^{-3}$ creates such a sharp gradient that the solution has a difficult time being approximated by a global polynomial, i.e., it is nearly not smooth. However, the IUB ${\hat{e}}_{\mathrm{T}}$ is much closer to the actual error and the ${\mathrm{L}}_{\infty }$-norm of the loss curve than the CUB ${\hat{e}}_{\mathrm{T}}$. This is because the IUB ${\hat{e}}_{\mathrm{T}}$ is, as its name suggests, an improved upper bound, considering only the local maximums of the ${\xi }_j$ coefficients during the exponential regression, as shown in Figure 5.

Subplot (a) in Figure 5 shows the ${\xi }_j$ coefficients for $j=(2,\dots ,L)$ belonging to the TFC solution when $L=4000$, the exponential fit performed on all coefficients (Exp. Fit 1), the exponential fit performed on only the local maximums (Exp. Fit 2), the CUB, and the IUB. As can be clearly seen, the coefficients fluctuate significantly. This results from the Gibbs Phenomenon, which is the oscillatory behavior of an approximation around a jump discontinuity, or in our case, a sharp gradient nearly identical to a jump discontinuity. The oscillatory behavior of the coefficients causes their exponential fit to be poor, as seen in subplot (b), which shows the $R^2$ value as L increases. When an exponential fit of only the local maximum ${\xi }_j$ coefficients is performed, the $R^2$ values are much closer to 1, and the resulting upper bound is much less conservative while still bounding all coefficients. Comparing Figure 3a with Figure 5a, one can see that the local maximum ${\xi }_j$ coefficients for approximating $y_2$ yield algebraic convergence instead of geometric convergence. This is a sign that $y_2$ is very close to not being smooth, if not already. Indeed, geometric convergence is only a given for a truncated series of Chebyshev polynomials if the function they approximate is definitively continuous and smooth.

Since the local maximums of the ${\xi }_j$ coefficients for approximating $y_2$ yield algebraic convergence, Rule of Thumb 2 does not apply (which can be seen in Figure 4). Also, the IUB in subplot (a) has a lot of room between it and the local maximum ${\xi }_j$ coefficients as j increases. Hence, the translated exponential fit shown in Equation (20) is not the best representation for nonsmooth functions. Nonetheless, the translated exponential fit is still a sufficient upper bound, meaning the resulting IUB ${\hat{e}}_{\mathrm{T}}$ is still valid. Other than the global polynomial not approximating the solution well, another reason why Rule of Thumb 1 is not valid in this case is that the IUB ${\hat{e}}_{\mathrm{T}}$ is still a conservative upper bound, although not as conservative as the CUB ${\hat{e}}_{\mathrm{T}}$. Therefore, Exp. Fit 2 is better than Exp. Fit 1 because it leads to the IUB ${\hat{e}}_{\mathrm{T}}$. Figure 4 shows that the IUB ${\hat{e}}_{\mathrm{T}}$ is closer to the ${\mathrm{L}}_{\infty }$-norm of the loss and, as such, correlates better with how the TFC method determines the accuracy of a solution.

An indication that the approximated function is not smooth, other than the local maximums of the ${\xi }_j$ coefficients yielding algebraic convergence, is the decay rate of the exponential fits, as shown in subplot (c). The decay rate of the exponential fits as L increases are all very close to 0. Thus, since the solution to Equation (26) is very close to not being smooth (i.e., $\alpha =1.5\times {10}^{-3}$), the decay rate of the ${\xi }_j$ coefficients alone can be used to determine that the actual solution is not smooth, or very close to not being smooth. When this is the case, the segment of the domain the constrained expression approximates should be divided.

4. hp-Adaptive Mesh Refinement Algorithm

The proposed hp-adaptive algorithm for solving hypersensitive TPBVPs begins by constructing an initial mesh $M_{h=0}$ and formulating the unconstrained system of ODEs via the TFC. All meshes $M_h$ are composed of mesh segments $S_k=[t_k,t_{k+1}]$ for all $k=(1,\dots ,K)$, the number of basis functions for each constrained expression on each segment ${ }^{(h,k)}{L}_i$, and the number of collocation points on each segment of each mesh ${ }^{(h,k)}N$. Note that the subscript h refers to an iteration of the hp-adaptive algorithm. Hence, each iteration of the algorithm has its own mesh. Furthermore, the number of basis functions does not have to be the same among the constrained expressions on $S_k$ belonging to $M_h$. However, the number of collocation points for all constrained expressions on the same segment must be identical. Therefore, we express ${ }^{(h,k)}N$ as

$${ }^{(h,k)}N=\underset{i}{max}\left({ }^{(h,k)}{L}_i\right)+N^{+},$$

where $N^{+}$ is a user-defined parameter.

With $M_0$ and the unconstrained system of ODEs formed via the TFC, an iteration of the hp-adaptive mesh refinement algorithm begins by solving the algebraic system of equations representing the unconstrained system of ODEs on $M_0$ with the Gauss–Newton method or linear least squares. Afterward, the truncated Chebyshev series terms/coefficients on the roundoff plateau or the local minimums of significant oscillation are first eliminated for the decay rate and bounded truncation error computations of the constrained expressions. The reason for this was explained in Section 3.3 and Section 3.4, and the method for accomplishing this is explained in Section 4.1. The estimated truncation error bound and decay rates, as described in Section 3, for each constrained expression in each interval can then be computed. If a k segment’s maximum absolute loss on $M_h$, ${\parallel { }^{(h,k)}\mathbb{L}\parallel }_{\infty }$, is greater than some prescribed tolerance, ${ϵ}_{\mathrm{MR}}$, then ${ }^{(h+1,k)}{L}_i$ for all $i=(1,\dots ,n)$ are increased, decreased, or the mesh interval is divided into subintervals for the next mesh iteration (see Section 4.2 and Section 4.3). The computational efficiency of subsequent mesh iterations can be improved by combining neighboring pairwise segments that are “super smooth” or by combining all neighboring segments that contain the minimum number of basis functions in their constrained expressions, $L_{\min }$ (see Section 4.4). With the next mesh interval defined as $M_{h+1}$, the process can be repeated until a maximum number of mesh iterations is reached, $h=H$, or ${\parallel { }^{(h,k)}\mathbb{L}\parallel }_{\infty }\le {ϵ}_{\mathrm{MR}}$ for all $k=(1,\dots ,K)$. An outline of the process is given in Section 4.5.

4.1. Disregarding Unnecessary Truncated Chebyshev Series Terms for Bounded Truncation Error Computation

In Section 3.1, we showed that a bounded exponential least-squares approximation of the truncated Chebyshev coefficients is necessary to obtain an expression for the maximum bound of a constrained expression’s truncation error. If the bounded exponential least-squares approximation of the Chebyshev coefficients is bad, then so too is the computation for the truncation error bound of the constrained expression. By “bad”, we do not only mean that the bounds are not valid. Instead, we mean that the Chebyshev coefficients and actual truncation error are well below their respective bounds (i.e., too conservative). In the next subsection, it is mathematically shown that conservative bounds lead to overly large mesh sizes. Hence, conservative bounds are undesirable. Obviously, bounds that are not valid are also undesirable.

In Section 3.3 and Section 3.4, it was shown that performing an exponential least-squares fit on all ${\xi }_j$ coefficients can lead to an incorrect or highly conservative bound for ${\xi }_j$. However, it was also shown that performing an exponential least-squares fit without the ${\xi }_j$ values that are on the roundoff plateau and without including the local minimums of significant oscillation leads to much better bounds. Representing the set of indices corresponding to the coefficients not on the roundoff plateau and not local minimums of significant oscillation as $\varsigma$, the exponential least-squares fit equation for computing q and s can then be given as

$$ln\left(\left|{\xi }_{\varsigma }\right|\right)=ln\left(s\right)-q\varsigma .$$

To determine the j indices not included in $\varsigma$, we refer to the coefficients on the roundoff plateau as those that are less than double machine epsilon. Furthermore, coefficients are believed to be a local minimum of significant oscillation when they are smaller than 3 orders of magnitude from their neighboring coefficients.

4.2. Increasing or Decreasing the Number of Basis Functions

Suppose ${\parallel { }^{(h,k)}\mathbb{L}\parallel }_{\infty }>{ϵ}_{\mathrm{MR}}$ and $q\ge \overline{q}$. Then, all constrained expressions in $S_k$ are considered to be smooth. Every ${ }^{(h+1,k)}{L}_i$ can then be increased to reduce ${\parallel { }^{(h+1,k)}\mathbb{L}\parallel }_{\infty }$. Let ${ }^{(h,k)}{\hat{e}}_{{\mathrm{T}}_i}$ denote the constrained expression’s estimate for the upper truncation error bound over interval $S_k$, on mesh h, and for the i-th differentiation variable. Then, we have the relationship

$${ }^{(h,k)}{\hat{e}}_{{\mathrm{T}}_i}=\frac{{\pi }^{1/2}sexp\left(-\delta -q\left({ }^{(h,k)}{L}_i+{ }^{(h,k)}{N}_{{\mathrm{const}}_i}+1\right)\right)}{{\left(2\left(1-exp(-2q)\right)\right)}^{1/2}},$$

(27)

where ${ }^{(h,k)}{N}_{{\mathrm{const}}_i}$ is the number of constraints on the i-th differentiation variable in $S_k$ belonging to $M_h$. On the next mesh, $M_{h+1}$, it is desired to achieve ${ }^{(h+1,k)}{\hat{e}}_{{\mathrm{T}}_i}={\epsilon }_{\mathrm{T}}\approx 0$ because, as Section 3.3 shows, a small ${ }^{(h+1,k)}{\hat{e}}_{{\mathrm{T}}_i}$ corresponds to a small $\parallel { }^{(h+1,k)}{\mathcal{L}}_i{\parallel }_{\infty }$. There is a specific value of ${ }^{(h+1,k)}{L}_i$ that yields ${ }^{(h+1,k)}{\hat{e}}_{{\mathrm{T}}_i}={\epsilon }_{\mathrm{T}}$,

$${\epsilon }_{\mathrm{T}}=\frac{{\pi }^{1/2}sexp\left(-\delta -q\left({ }^{(h+1,k)}{L}_i+{ }^{(h+1,k)}{N}_{{\mathrm{const}}_i}+1\right)\right)}{2^{1/2}{\left(1-exp(-2q)\right)}^{1/2}}.$$

(28)

Dividing Equations (27) and (28), recognizing that ${ }^{(h,k)}{N}_{{\mathrm{const}}_i}={ }^{(h+1,k)}{N}_{{\mathrm{const}}_i}$ when the intervals are not changed, and solving for ${ }^{(h+1,k)}{L}_i$ gives

$${ }^{(h+1,k)}{L}_i=⌈{ }^{(h,k)}{L}_i+\frac{1}{q}ln\left(\frac{{ }^{(h,k)}{\hat{e}}_{{\mathrm{T}}_i}}{{\epsilon }_{\mathrm{T}}}\right)⌉,$$

(29)

where q is calculated as shown in Section 4.1. Note that $⌈·⌉$ rounds the $·$ inside it up to the next highest integer, which is necessary because the number of basis functions has to be an integer that strictly increases for a lower ${ }^{(h+1,k)}{\hat{e}}_{{\mathrm{T}}_i}$. In Equation (29), one can see that if ${ }^{(h,k)}{\hat{e}}_{{\mathrm{T}}_i}<{\epsilon }_{\mathrm{T}}$, then the number of basis functions decreases (i.e., ${ }^{(h+1,k)}{L}_i<{ }^{(h,k)}{L}_i$). This is ideal because it allows the number of basis functions for specific constrained expressions in a specific segment to decrease and still have $\parallel { }^{(h+1,k)}{\mathbb{L}\parallel }_{\infty }\le {ϵ}_{\mathrm{MR}}$. Allowing smoother constrained expressions to decrease their ${ }^{(h,k)}{L}_i$ values reduces computational complexity when obtaining the Jacobians and optimizing for the Chebyshev coefficients. Lastly, note that ${\epsilon }_{\mathrm{T}}$ is a user-defined value but should be as low as possible.

4.3. Dividing a Mesh Interval

Let $\parallel { }^{(h,k)}{\mathbb{L}\parallel }_{\infty }>{ϵ}_{\mathrm{MR}}$ and $q<\overline{q}$. Then, computing ${ }^{(h+1,k)}{L}_i$ with Equation (29) will be very large for some small ${\epsilon }_{\mathrm{T}}$. Large ${ }^{(h+1,k)}{L}_i$ values cause large inversion matrices in the Gauss–Newton and linear least-squares algorithms, which raises the condition number. This can cause both algorithms to become numerically unstable and fail. Instead of increasing the number of basis functions in a mesh interval, it can instead be divided. The number of subintervals, $V_k$, into which $S_k$ is divided should equal the predicted number of basis functions in the constrained expression for the next mesh in Equation (29) using $\overline{q}$,

$${ }^{(h,k)}{\overline{L}}_i={ }^{(h,k)}{L}_i+\frac{1}{\overline{q}}ln\left(\frac{{ }^{(h,k)}{\hat{e}}_{{\mathrm{T}}_i}}{{\epsilon }_{\mathrm{T}}}\right).$$

Thus, we have

$$V_k=\underset{i}{max}\left(⌈\frac{{ }^{(h,k)}{\overline{L}}_i}{{ }^{(h,k)}{L}_i}⌉\right).$$

The locations of the knot points for all $V_k$ intervals within $S_k$ are equidistant.

4.4. Combining Mesh Intervals

The prescribed decay rate $\overline{q}$ indicates whether a function is smooth ($q\ge \overline{q}$) or nonsmooth ($q<\overline{q}$). Here, a new prescribed decay rate $\stackrel{`}{q}>\overline{q}$ is introduced to determine whether a function is super-smooth ($q\ge \stackrel{`}{q}$). If two and only two adjacent mesh intervals have $q>\stackrel{`}{q}$ in all of their constrained expressions, then the intervals are combined. The number of basis functions on the new interval is equal to the sum of the basis functions in the intervals being combined. Only two neighboring mesh intervals are combined in a mesh iteration at a time to reduce the chance that multiple super-smooth segments are combined into one nonsmooth segment. Furthermore, neighboring pairwise super-smooth segments can only be combined in a mesh iteration if they were not divided in the same iteration.

Another way in which segments are combined is if adjacent segments have

$${\displaystyle \underset{i}{max}\left({ }^{(h,k)}{L}_i\right)=L_{\min },}$$

(30)

which means every constrained expression in a segment has the minimum number of basis functions. Hypersensitive TPBVPs have a solution that typically consists of constant values for long periods of time, followed or proceeded by sharp changes in its gradient. When the equality in Equation (30) holds true, the solution on the k segment is often constant. Thus, all neighboring segments where Equation (30) holds true are combined. We do not have to worry about combining a collection of super-smooth segments into a nonsmooth segment because the solution on the combined segments consists of constant values. Segments can only be combined if they were not divided in the same mesh iteration as well.

4.5. hp-Adaptive Mesh Refinement Algorithm Outline

For the reader’s convenience, a brief step-by-step procedure of the hp-adaptive mesh refinement algorithm is given as follows:

Step 1:

Set $h=0$ and provide an initial mesh $M_0$ that is composed of initial segments ${ }^{(0,k)}S$ and an initial number of basis functions for each constrained expression within each segment, ${ }^{(0,k)}{L}_i$. Furthermore, make sure to set the hyperparameters, which are listed in Table 1.

Step 2:

Formulate the unconstrained system of ODEs from the TPBVP by building the constrained expressions for the differentiation variables on $M_h$.

Step 3:

Solve the unconstrained system of ODEs on mesh $M_h$, as described in Section 2.

Step 4:

If ${\parallel }^{(h,k)}{\mathbb{L}\parallel }_{\infty }\le {ϵ}_{\mathrm{MR}}$ for all $k=(1,\dots ,K)$, then the TPBVP is solved with the user’s desired accuracy, and the mesh refinement is complete. If $h=H$, then the mesh refinement is also complete, even though the TPBVP may not be solved at the desired accuracy. Otherwise, for every $S_k$ where $k=(1,\dots ,K)$ on $M_h$:

• Proceed to the next segment if ${\parallel { }^{(h,k)}\mathbb{L}\parallel }_{\infty }\le {ϵ}_{\mathrm{MR}}$.
• Calculate $q_i$ and ${\hat{e}}_{T_i}$ for every $i=1,\dots ,n$ in the segment, as shown in Section 3.1. Make sure to disregard the terms from the Chebyshev polynomial within the constrained expressions that contain negligible coefficients on the roundoff plateau or are a local minimum of significant oscillation, as described in Section 4.1.
• If ${\displaystyle \underset{i}{max}\left(q_i\right)>\overline{q}}$, modify the segment for the next mesh iteration by either increasing/decreasing the number of basis functions in any of the segment’s constrained expressions (Section 4.2) or by dividing the segment (Section 4.3).

Step 5:

For neighboring segments that were not divided for the next mesh iteration, combine them for the next mesh iteration, as shown in Section 4.4).

Step 6:

Set $h=h+1$ and return to Step 2.

Table 1. Hyperparameters.

Symbol	Value	Description
${ϵ}_{\mathrm{GN}}$	$1\times {10}^{-15}$	Gauss–Newton error tolerance.
${ϵ}_{\mathrm{MR}}$	$1\times {10}^{-13}$	Mesh refinement error tolerance for an acceptable solution.
${\epsilon }_{\mathrm{T}}$	$1\times {10}^{-15}$	Desired truncation error for a constrained expression.
$p_{\max }$	40	Maximum number of Gauss–Newton iterations.
H	20	Maximum number of mesh refinement iterations.
$N^{+}$	3	Number of collocation points more than the number of basis functions.
$\overline{q}$	0.8	Cutoff decay rate that determines whether a function is smooth or not.
$\stackrel{`}{q}$	1.2	Cutoff decay rate that determines whether a function is super-smooth or just smooth
$L_{\min }$	3	Minimum number of basis functions in a constrained expression.

Table 1. Hyperparameters.

Symbol	Value	Description
${ϵ}_{\mathrm{GN}}$	$1\times {10}^{-15}$	Gauss–Newton error tolerance.
${ϵ}_{\mathrm{MR}}$	$1\times {10}^{-13}$	Mesh refinement error tolerance for an acceptable solution.
${\epsilon }_{\mathrm{T}}$	$1\times {10}^{-15}$	Desired truncation error for a constrained expression.
$p_{\max }$	40	Maximum number of Gauss–Newton iterations.
H	20	Maximum number of mesh refinement iterations.
$N^{+}$	3	Number of collocation points more than the number of basis functions.
$\overline{q}$	0.8	Cutoff decay rate that determines whether a function is smooth or not.
$\stackrel{`}{q}$	1.2	Cutoff decay rate that determines whether a function is super-smooth or just smooth
$L_{\min }$	3	Minimum number of basis functions in a constrained expression.

5. Results

We solved three hypersensitive TPBVPs with our proposed TFC hp-adaptive mesh refinement algorithm. Each TPBVP was derived by applying PMP to its respective OCP. A brief procedure for how to use PMP to formulate TPBVPs from OCPs is given in Appendix B. All programming was performed in MATLAB^® R2021b and run on an Intel Core i7-9700 CPU PC with 64 GB of RAM. The hyperparameters used for the algorithm are given in Table 1, except that ${ϵ}_{\mathrm{MR}}$ equaled $1^{-15}$ for the first test problem.

5.1. Linear Hypersensitive Problem

The first OCP we attempted to solve was

$$\begin{array}{cc}\hfill \underset{u}{min}& J=\frac{1}{2}{\int }_{t_0}^{t_f}\left(x^2+u^2\right)\mathrm{d}t\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& \dot{x}=-x+u\hfill \end{array}$$

with the boundary constraints

$$\left\{\begin{array}{c}x\left(t_0\right)=1.5\hfill \\ x\left(t_f\right)=1\hfill \end{array}\right..$$

(31)

The initial and final times were $t_0=0$ and $t_f$ = 10,000, respectively. After applying PMP, the dual TPBVP ODEs were

$$\left\{\begin{array}{c}\dot{x}=-x-\lambda \hfill \\ \dot{\lambda }=-x+\lambda \hfill \end{array}\right.,$$

(32)

where $\lambda$ is the costate variable that can be used to compute the control,

$$u=-\lambda .$$

As its name suggests, the TPBVP is subject to the same boundary conditions as the OCP (Equation (31)). The main reason we decided to solve this hypersensitive TPBVP was because its linearity allows a true analytical solution to be found quite easily:

$$x^{*}\left(t\right)=c_{1}exp\left(t/\sqrt{2}\right)+c_{2}exp\left(-t/\sqrt{2}\right)$$

and

$$u^{*}\left(t\right)={\dot{x}}^{*}\left(t\right)+x^{*}\left(t\right)$$

with

$$c_1=\frac{1}{exp\left(-t_f/\sqrt{2}\right)-exp\left(t_f/\sqrt{2}\right)}\left(1.5exp\left(-t_f/\sqrt{2}\right)-1\right)$$

and

$$c_2=\frac{1}{exp\left(-t_f/\sqrt{2}\right)-exp\left(t_f/\sqrt{2}\right)}\left(1-1.5exp\left(t_f/\sqrt{2}\right)\right).$$

To quantify the benefits of using our TFC-based hp-adaptive mesh refinement algorithm, we first solved this problem with the standard TFC procedure that uses global constrained expressions to approximate x and u with a hard-coded number of basis functions and collocation points. Multiple solutions were calculated, each having the same number of basis functions in the constrained expressions approximating x and u, but increased by one for each run. The number of collocation points for each solution was fixed at 1050. Figure 6 shows the ${\mathrm{L}}_{\infty }$-norm of the loss and actual error for the solutions. As can be seen, the TFC with global constrained expressions was very capable of solving this problem, but around 600 basis functions were required for the minimum loss and minimum actual error to be reached. With the hp-adaptive mesh refinement algorithm, this problem should be able to be solved with a much fewer number of total basis functions, and even with better accuracy.

The TFC-based hp-adaptive mesh refinement algorithm began with an initial mesh of only two knot points: one at $t_0$ and another at $t_f$, i.e., global constrained expressions. The number of basis functions in the x and $\lambda$ constrained expressions on the initial mesh was set to 50. The algorithm did not find a solution that satisfied ${\parallel \mathbb{L}\parallel }_{\infty }\le {ϵ}_{\mathrm{MR}}=1\times {10}^{-15}$, meaning it ran for the maximum number of iterations, but the solution it did find was still very accurate. Figure 7 shows the final solutions for x and u, each overlapping with the truth. The subplots in Figure 8 show how the loss decreased and the mesh knots changed over the mesh iterations. The final mesh consisted of five segments, i.e., six knots at $t_0=0$, $t_1=19.5$, $t_2=29.3$, $t_3=9970.7$, $t_4=9980.5$, and $t_5=t_f$ = 10,000. The ${\parallel \mathbb{L}\parallel }_{\infty }$ of the final solution was $9.3\times {10}^{-15}$. Summing the number of basis functions for all segments of the constrained expressions for x and $\lambda$ yielded 100 and 103, respectively.

5.2. Nonlinear Hypersensitive Problem

The nonlinear hypersensitive OCP we attempted to solve was

$$\begin{array}{cc}\hfill \underset{u}{min}& J=\frac{1}{2}{\int }_{t_0}^{t_f}\left(x^2+u^2\right)\mathrm{d}t\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& \dot{x}=-x^3+u\hfill \end{array}$$

with the boundary constraints

$$\left\{\begin{array}{c}x\left(t_0\right)=1.5\hfill \\ x\left(t_f\right)=1\hfill \end{array}\right..$$

(33)

As before, the initial and final times were $t_0=0$ and $t_f$ = 10,000. After applying PMP, the dual TPBVP ODEs were

$$\left\{\begin{array}{c}\dot{x}=-x^3-\lambda \hfill \\ \dot{\lambda }=-x+3x^2\lambda \hfill \end{array}\right.,$$

(34)

where the control is related to the costate by

$$u=-\lambda .$$

The TPBVP given by Equations (33) and (34) does not have a true analytical solution due to its nonlinearity. Due to this nonlinearity, an initial guess needed to be provided for the Gauss–Newton step of the TFC procedure. The initial guess we provided for x and $\lambda$ was simply zero over the entire time domain. Unlike for the linear hypersensitive problem, here, we do not provide a plot showing how ${\parallel \mathbb{L}\parallel }_{\infty }$ decreases when the number of basis functions is increased and global constrained expressions are used because ${\parallel \mathbb{L}\parallel }_{\infty }$ was greater than $1\times {10}^{-4}$ when 1000 basis functions were used.

Similar to the approach used to solve the previous problem, the TFC-based hp-adaptive mesh refinement algorithm began with an initial mesh of only one segment and with the number of basis functions in the x and $\lambda$ constrained expressions set to 50. The algorithm ran for the maximum number of mesh iterations, i.e., $h=H=20$. Thus, ${\parallel \mathbb{L}\parallel }_{\infty }\le {ϵ}_{\mathrm{MR}}=1\times {10}^{-13}$ was not satisfied by the final loss, even though it was still exceptionally low.

Table 2. ${\mathrm{L}}_{\infty }$-norm of the loss for different methods for the nonlinear hypersensitive problem.

TFC Split	TFC Global	bvp4c
$1.285\times {10}^{-13}$	$4.969\times {10}^{-1}$	$4.128\times {10}^{-4}$

gives the ${\mathrm{L}}_{\infty }$-norm of the loss for the TFC solution at the final mesh iteration (TFC Split), the TFC when only global constrained expressions were used with $L=300$ (TFC Global), and MATLAB’s bvp4c routine. Note that the bvp4c routine is a finite-difference implementation of the three-stage Lobatto IIIa formula [35,36] and can be regarded as an h method. The same initial guess was used for each mesh. Figure 9 shows the final solutions for x and u near the boundaries of the time domain. The proposed mesh refinement algorithm’s and bvp4c curves overlap, but it is clear that the proposed approach’s solutions are better based on its ${\mathrm{L}}_{\infty }$-norm of the loss being nine orders of magnitude lower. The subplots in Figure 10 show how the loss decreased in the proposed algorithm over its mesh iterations and how the mesh knots changed. The final mesh consisted of nine segments, one long segment in the center of the time domain, and several much smaller segments clustered near the boundaries to handle the gradients.

5.3. Mass Spring Problem

The last problem solved was a common mass spring system OCP,

$$\begin{array}{cc}\hfill \underset{u}{min}& J=\frac{1}{2}{\int }_{t_0}^{t_f}\left(x^2+u^2\right)\mathrm{d}t\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& {\dot{x}}_1=x_2\hfill \\ \hfill & {\dot{x}}_2=-x_1-x_1^3-u-1,\hfill \end{array}$$

with the boundary constraints

$$\left\{\begin{array}{c}{x}_1\left(t_0\right)=1\hfill \\ x_2\left(t_0\right)=0.75\hfill \\ x_1\left(t_f\right)=0\hfill \\ x_2\left(t_f\right)=0\hfill \end{array}\right..$$

(35)

The initial and final times were $t_0=0$ and $t_f$ = 10,000, respectively. After applying PMP, the dual TPBVP ODEs were

$$\left\{\begin{array}{c}{\dot{x}}_1=x_2\hfill \\ {\dot{x}}_2=-x_1-x_1^3-u-1\hfill \\ {\dot{\lambda }}_1=-x_1+{\lambda }_2\left(1+3x_1^2\right)\hfill \\ {\dot{\lambda }}_2=-x_2-{\lambda }_1\hfill \end{array}\right.,$$

(36)

where the control is related to the costate by

$$u=-\lambda .$$

Like the previous TPBVP solved, the TPBVP given by Equations (35) and (36) does not have a true analytical solution due to its nonlinearity. The initial mesh and initial guess of the solution were identical to what was used to solve the nonlinear hypersensitive problem.

Table 3. ${\mathrm{L}}_{\infty }$-norm of the loss for different methods for the mass spring problem.

TFC Split	TFC Global	bvp4c
$2.345\times {10}^{-12}$	$5.455\times {10}^{-1}$	$9.878\times {10}^{-4}$

shows the ${\mathrm{L}}_{\infty }$-norm of the loss for the proposed approach and the other approaches shown in Table 2. Once again, the proposed approach performed the best. Figure 11 shows the trajectories of the various solutions. The subplots in Figure 12 show how the loss decreased in the proposed algorithm over its mesh iterations and portray how the mesh knots changed. The final mesh consisted of 20 segments, and most of them were near the boundary, while one long segment was used to approximate the constant part of the solution, as shown in Figure 12b.

6. A Discussion on Computation Time

The main drawback of the proposed algorithm is its computation time when the problem is nonlinear. Figure 13 shows how the size of the mesh affects the most computationally intensive aspects of the proposed algorithm for the nonlinear hypersensitive problem (a) and mass spring problem (b). The steps that took the longest to complete were the numerical approximations of the Jacobian and the least-square computations performed within the Gauss–Newton algorithm. Hence, Figure 13 shows the total accumulated runtime for performing these calculations at each mesh iteration. The size of the Jacobian is related to the number of basis functions (columns) and the number of collocations (rows) being solved. As can clearly be seen, a larger mesh correlates to a longer runtime. The reason why all accumulated least-squares computations were so small was due to MATLAB’s efficient storage of sparse matrices. The numerical approximation of the Jacobian was time-consuming because it required a number of computations equal to the number of columns in the Jacobian, i.e., the derivatives of the loss with respect to all Chebyshev coefficients in all constrained expressions in all segments were needed. Hence, the proposed algorithm has a long runtime because of the need to numerically approximate the Jacobian at each iteration of the Gauss–Newton algorithm within each mesh iteration.

The best way to reduce the time needed to numerically compute the Jacobian would be to break apart the partial derivatives of the loss with respect to the unknown coefficients into two parts using the chain rule: the partial derivatives of the loss with respect to the differentiation variables and the partial derivatives of the constrained expressions with respect to the unknown coefficients. The partial derivatives of the constrained expressions with respect to the unknown coefficients could be computed by hand a priori. Thus, only the partial derivatives of the loss with respect to the differentiation variables would need to be computed numerically, which would drastically reduce the number of loss evaluations needed. Patterson and Rao [37] demonstrated a similar approach for the pseudospectral method when Lagrange polynomials are used to approximate the differentiation variables. We hope to do the same but for the TFC’s constrained expressions in future work.

7. Conclusions

An hp-adaptive mesh refinement algorithm has been developed for the TFC method that utilizes Chebyshev polynomial-free functions. The algorithm relies on the correlation between the ${\mathrm{L}}_{\infty }$-norm of the loss and the truncation error to determine the ideal number of basis functions in a constrained expression. Whether to add basis functions to constrained expressions within a segment or divide the segment is determined by the decay rates of the Chebyshev coefficients in the segment’s constrained expressions. Smooth segments have larger decay rates, while nonsmooth segments have small decay rates. The error is decreased by adding basis functions to the smooth segments and dividing the segments that are not smooth. To improve computational performance, the number of basis functions is decreased in constrained expressions on smooth segments that already satisfy the ${\mathrm{L}}_{\infty }$-norm of the loss tolerance well. Lastly, segments that are very smooth are combined to further improve computational performance.

The only problems solved in this work were TPBVPs generated by performing PMP on hypersensitive OCPs without inequality constraints. All problems were solved using the proposed approach with a maximum loss on the order of $1\times {10}^{-12}$ or better. In contrast, MATLAB’s bvp4c solved each nonlinear problem with a maximum loss on the order of $1\times {10}^{-4}$. Hypersensitive OCPs with inequality constraints will be solved in future work, which will require the continuation of regularization or smoothing parameters. Since the continuation process will be iterative, increased computational performance will be required. As the algorithm is currently constructed, it is too slow to be coupled with continuation due to the cost of numerically approximating the Jacobian. Hence, a major challenge of future work will be discovering new ways to approximate the Jacobian with fewer calculations, similar to what has already been accomplished with other collocation procedures (e.g., the pseudospectral method for optimal control).