Combinatorial Integral Approximation Decompositions for Mixed-Integer Optimal Control

Abstract

Solving mixed-integer nonlinear programs (MINLPs) is hard from both a theoretical and practical perspective. Decomposing the nonlinear and the integer part is promising from a computational point of view. In general, however, no bounds on the objective value gap can be established and iterative procedures with potentially many subproblems are necessary. The situation is different for mixed-integer optimal control problems with binary variables that switch over time. Here, a priori bounds were derived for a decomposition into one continuous nonlinear control problem and one mixed-integer linear program, the combinatorial integral approximation (CIA) problem. In this article, we generalize and extend the decomposition idea. First, we derive different decompositions and analyze the implied a priori bounds. Second, we propose several strategies to recombine promising candidate solutions for the binary control functions in the original problem. We present the extensions for ordinary differential equations-constrained problems. These extensions are transferable in a straightforward way, though, to recently suggested variants for certain partial differential equations, for algebraic equations, for additional combinatorial constraints, and for discrete time problems. We implemented all algorithms and subproblems in AMPL for a proof-of-concept study. Numerical results show the improvement compared to the standard CIA decomposition with respect to objective function value and compared to general-purpose MINLP solvers with respect to runtime.

1. Introduction

1.1. General Context

The goal of optimal control is to find control functions and state trajectories that are feasible and optimal. State trajectories are solutions of systems of differential equations for given control functions and boundary conditions. Feasibility refers to constraints on control functions and differential states; optimality refers to an objective functional of controls and states. Mixed-integer optimal control problems (MIOCPs) do have additional integrality constraints on some of the control functions. Hence, the differential equations of the underlying system depend on the value of integer control functions, or equivalently, one can switch instantly between several differential equations [1,2,3,4]. This problem class is ubiquitous in various application areas, e.g., water, gas, traffic, and supply chain networks [5,6,7,8,9,10,11], distributed autonomous systems [12], processes in chemical engineering that involve valves [13,14], cardiac assist devices [15], or the choice of gears in automotive control [16,17]. We choose problems from a web-based MIOCP benchmark collection [18] to evaluate our algorithms.

Several families of methods have been proposed for solving MIOCPs. This comprehends indirect or first-optimize-then-discretize approaches [19], dynamic programming or Hamilton–Jacobi–Bellman equations [20], switching time optimization [16,21,22,23,24,25], and direct or first-discretize-then-optimize approaches [3,26,27]. Surveys, references, and comparisons can be found in, e.g., [19,28,29,30].

1.2. Motivation

Our approach is based on previous ideas [27,31,32] to decompose the MIOCP into a relaxed continuous optimal control problem (OCP) and a mixed-integer linear program (MILP). In contrast to general MINLP decompositions (see [33] for recent results on error bounds for rounding approaches and [34] for a MINLP survey), the particular setting with dependent (states) and independent (controls) variables allows the derivation of a priori bounds [35]. As specified in Section 4, the difference between state trajectories $\mathbf{x}(·)$ and $\mathbf{y}(·)$, which are the unique solutions of the initial value problems for given control functions $\omega$ and $\alpha$, respectively, is bounded for a constant C and all $t\in \mathcal{T}$ by:

$$\begin{array}{c}\hfill \left|\left|\mathbf{x}\left(t\right)-\mathbf{y}\left(t\right)\right|\right|\le C\underset{t\in \mathcal{T}}{max}\left|\left|{\int }_{t_0}^t\alpha \left(\tau \right)-\omega \left(\tau \right)\mathrm{d}\tau \right|\right|.\end{array}$$

(1)

This motivates us to solve the relaxed problem (OCP) to obtain $\alpha (·)$, to calculate an integer control function $\omega$ in a second step such that the maximum on the right-hand side of (1) is as small as possible, and to obtain $\mathbf{y}(·)$ by solving the initial value problem (IVP). The continuity of constraint and objective functions implies “good” behavior with respect to feasibility and objective function value in the original MIOCP. The combinatorial integral approximation (CIA) problem in the second step can be formulated as an MILP [32] and can be solved either with generic MILP methods, with specifically tailored branch-and-bound methods [32], and in some cases even in linear time, with the sum-up rounding (SUR) method [27].

1.3. Review of the State of the Art

The idea of CIA decomposition has also been used in the context of hyperbolic partial differential equations [36,37], differential-algebraic equation systems [19], with additional combinatorial constraints [10,32], and based on discrete time formulations [10]. Recently, algorithms to solve CIA problems have been implemented into the software package pycombina [38].

In this publication, we generalize all of these approaches by presenting alternative ways to calculate $\mathit{\omega }$ based on $\mathbf{\alpha }$, using different MILPs. We formulate them for the case of initial value problems with ordinary differential equations, although they are applicable to all mentioned variants in a straightforward way.

Our results are also independent of the method that is applied to solve the relaxed problem (OCP). For the numerical results, we are going to use a first-discretize-then-optimize approach with Radau collocation. First-discretize refers (a) to approximating the control functions with parameterized basis functions, such as finitely many piecewise constant functions, and (b) to relaxing path and control constraints from the domain of a time horizon to finitely many time points. Then-optimize refers to solving the resulting finite dimensional optimization problem numerically to optimality. An overview of direct methods for continuous optimal control problems can be found in, e.g., [39,40]. For comparison, we also apply this approach directly to the MIOCP and solve the resulting mixed-integer nonlinear program (MINLP) with the general purpose solver Bonmin.

For a general problem class of MIOCPs, the integer gap depends linearly on the control discretization [41]. For many applications, this control discretization is not fixed, but can be refined. Thus, the integer gap can be driven to zero (usually at the expense of frequent switching). For practical reasons, good or even optimal solutions for a fixed control discretization are of interest, which is our main focus.

Recently, the MIOCP class has been increasingly studied in the context of combinatorial constraints that couple over time. For example, as part of the CIA decomposition, the switching-cost aware rounding problem (SCARP) has been proposed to solve it [42,43]. In addition, the CIA decomposition has been studied in the context of multibang and total variation regularization [44,45]. Switching constraints have been also investigated for mixed-integer partial differential equation optimal control problems [46,47]. Switching costs can also be included into the switching time optimization approach based on cardinality constraints [48].

1.4. Contributions

We derive different versions of the CIA problem, leading to multiple MILPs. Noting that the computational effort to solve one MILP is usually small compared to the solution of the relaxed nonlinear problem, and even smaller compared to the deterministic solution of the original MIOCP, we propose solving several approximation problems. These solutions are candidate solutions themselves, and can be recombined into new switching sequences. We derive theoretical a priori bounds for them, discuss computational (dis)advantages, and show numerically the improvement to existing decomposition approaches with respect to objective function value, and to general-purpose MINLP solvers with respect to runtime.

1.5. Outline of the Article

We define the considered MIOCP class and propose a general decomposition framework in Section 2. The algorithm consists of several MILP formulations, which are discussed in Section 3. We discuss theoretical properties in Section 4. In Section 5, we look at strategies that combine and improve existing binary control functions. In Section 6, we provide and discuss numerical results for the MIOCP benchmark library [18]. Finally, we conclude the article in Section 7, where we also summarize our findings.

2. Problem Class, Definitions, and Main Algorithm

We denote the considered time horizon by $\mathcal{T}:=[t_0,t_f]\subset \mathbb{R}$, and write “for a.a. $t\in \mathcal{T}$” for all $t\in \mathcal{T}$, except on a set of measure zero. Let $\left[n\right]:=\{1,\dots ,n\},{\left[n\right]}_0:=0\cup \left[n\right],$ for $n\in \mathbb{N}$. The null vector is written as $\mathbf{o}$. We are interested in the following class of mixed-integer optimal control problems.

Definition 1. 

(MIOCP) We refer to the following control problem (2) as (MIOCP).

$$\begin{array}{cc}\hfill \underset{\mathbf{x},\mathit{\omega }}{inf}& \mathsf{\Phi }\left(\mathbf{x}\left(t_f\right)\right)\hfill \end{array}$$

(2a)

$$\begin{array}{ccc}\hfill s.t.& \dot{\mathbf{x}}\left(t\right)={\mathit{f}}_0(t,\mathbf{x}\left(t\right))+\sum _{i=1}^{n_{\omega }}{\omega }_i\left(t\right){\mathit{f}}_i(t,\mathbf{x}\left(t\right)),\hfill & \hfill fora.a.t\in \mathcal{T},\end{array}$$

(2b)

$$\begin{array}{ccc}& \mathbf{x}\left(t_0\right)={\mathbf{x}}_0,\hfill & \hfill \end{array}$$

(2c)

$$\begin{array}{ccc}& 1=\sum _{i=1}^{n_{\omega }}{\omega }_i\left(t\right)\hfill & \hfill fora.a.t\in \mathcal{T},\end{array}$$

(2d)

$$\begin{array}{ccc}& \omega \left(t\right)\in {\{0,1\}}^{n_{\omega }}\hfill & \hfill fora.a.t\in \mathcal{T},\end{array}$$

(2e)

$$\begin{array}{ccc}& \mathbf{o}\le \mathbf{c}(t,\mathbf{x}(t\left)\right)\hfill & \hfill fora.a.t\in \mathcal{T}.\end{array}$$

(2f)

We minimize a Mayer term $\mathsf{\Phi }\in C^1({\mathbb{R}}^{n_x},\mathbb{R})$ over differential states $\mathbf{x}\in {\mathcal{W}}^{1,\infty }(\mathcal{T},{\mathbb{R}}^{n_x})$ for binary control functions $\mathit{\omega }\in L^{\infty }(\mathcal{T},{\{0,1\}}^{n_{\omega }})$. The system of ordinary differential equations (ODE), (2b), is written using partial outer convexification to model the switched system, using a one-hot encoding (1hot) constraint (2d), a drift term ${\mathit{f}}_0$, and $n_{\omega }\ge 2$ functions ${\mathit{f}}_i$, ref. [27], both out of $C^0({\mathbb{R}}^{n_x+1},{\mathbb{R}}^{n_x})$. We assume fixed initial values ${\mathbf{x}}_0\in {\mathbb{R}}^{n_x}$ for the differential states. The functions $\mathbf{c}\in C^1({\mathbb{R}}^{n_x},{\mathbb{R}}^{n_c})$ model nonlinear state inequalities.

In the following, we assume that (MIOCP) has an optimal solution, so that we can write “min” instead of “inf”. We refer to [19,29,49] for a discussion of the generality of (MIOCP) and extensions to cope with, e.g., Lagrange functionals, boundary and multi-point constraints, vanishing constraints, free final time, control values, and the like. Particularly interesting are continuous control functions $\mathbf{u}\in L^{\infty }(\mathcal{T},\mathcal{U})$ that often enter (2b) and (2f) in practical applications. From a theoretical point of view, in the interest of comparability, and for computational speed, it is convenient to consider the continuous controls $u(·)$ as fixed to the solution that was obtained by solving the continuous relaxation of (MIOCP) in our approach. It is also possible, though, and often makes sense in practice, to improve the objective function value by reoptimizing $\mathbf{u}$ when (MIOCP) is evaluated for fixed $\mathit{\omega }$. For notational convenience and without loss of generality, we omit the continuous controls $\mathbf{u}$ in the following. An evaluation of (MIOCP) for fixed $\mathit{\omega }$ is then the solution of an initial value problem. We stress that there are only $n_{\omega }$ possible solutions for the integer part of the problem due to the constraint (2d).

Our algorithm solves continuous relaxations of (MIOCP), defined as follows.

Definition 2. 

(OCP) We define (OCP) as the canonical relaxation of (MIOCP) with respect to (2e), where we substitute $\mathit{\omega }\in L^{\infty }(\mathcal{T},{\{0,1\}}^{n_{\omega }})$ for $\mathbf{\alpha }\in L^{\infty }(\mathcal{T},{[0,1]}^{n_{\omega }})$.

The problem (OCP) in function space can be solved by different approaches, as mentioned above. For using MILPs to approximate control functions, we map between function space and ${[0,1]}^{n_{\omega }\times M}$, using a time grid as follows.

Definition 3. 

(${\mathcal{G}}_{\omega }$, Δ, φ, ${\phi }^{-1}$, Ω, ${\mathsf{\Omega }}_M$) Let the ordered set ${\mathcal{G}}_{\omega }:=\{t_0<\dots <t_M=t_f\}$ denote a time grid with ${\Delta }_j:=t_{j+1}-t_j$ and ${\Delta }_{\max }:={max}_j{\Delta }_j$ for $j\in {[M-1]}_0$. We define the mapping:

$$\begin{array}{c}\hfill \phi :{[0,1]}^{n_{\omega }\times M}\to L^{\infty }(\mathcal{T},{[0,1]}^{n_{\omega }}),\mathbf{\alpha }=\phi \left(\mathbf{a}\right)\end{array}$$

using $n_{\omega }$ piecewise constant functions:

$$\begin{array}{cc}\hfill {\alpha }_i\left(t\right)& :=a_{i,j},i\in \left[n_{\omega }\right],t\in [t_j,t_{j+1}),j\in {[M-1]}_0,t_j\in {\mathcal{G}}_{\omega }.\hfill \end{array}$$

A mapping in reverse direction:

$$\begin{array}{c}\hfill {\phi }^{-1}:L^{\infty }(\mathcal{T},{[0,1]}^{n_{\omega }})\to {[0,1]}^{n_{\omega }\times M},\mathbf{a}={\phi }^{-1}\left(\mathbf{\alpha }\right)\end{array}$$

is defined by extracting integrals on the grid ${\mathcal{G}}_{\omega }$:

$$\begin{array}{c}\hfill a_{i,j}:=\frac{1}{{\Delta }_j}{\int }_{t_j}^{t_{j+1}}{\alpha }_i\left(\tau \right)\mathrm{d}\tau ,i\in \left[n_{\omega }\right],j\in {[M-1]}_0,t_j\in {\mathcal{G}}_{\omega }.\end{array}$$

We denote integrality and (2d) using the sets:

$$\begin{array}{cc}\hfill \mathsf{\Omega }& :=\left\{\mathit{\omega }\in L^{\infty }(\mathcal{T},{\{0,1\}}^{n_{\omega }}):1=\sum _{i=1}^{n_{\omega }}{\omega }_i\left(t\right)fora.a.t\in \mathcal{T}\right\},\hfill \\ \hfill {\mathsf{\Omega }}_M& :=\left\{\mathbf{w}\in {\{0,1\}}^{n_{\omega }\times M}:1=\sum _{i=1}^{n_{\omega }}{w}_{i,j}forj\in {[M-1]}_0\right\}.\hfill \end{array}$$

To scale control variables, we are going to need function evaluations and adjoint (dual) variables on the grid ${\mathcal{G}}_{\omega }$.

Definition 4 ($(\tilde{\mathbf{\lambda }},\tilde{\mathit{f}})$). 

Let ${\mathbf{x}}^{\ast }$ be the optimal solution of (OCP). The evaluated right-hand side function terms ${\left\{{\tilde{\mathit{f}}}_{i,j,k}\right\}}_{k\in \left[n_x\right]}$ from (2b) are defined as the entries of:

$$\begin{array}{c}\hfill {\mathbb{R}}^{n_x}\ni {\tilde{\mathit{f}}}_{i,j}:={\displaystyle \frac{1}{{\Delta }_j}}\underset{t_j}{\overset{t_{j+1}}{\int }}{\mathit{f}}_i(\tau ,{\mathbf{x}}^{\ast }\left(\tau \right))\mathrm{d}\tau ,i\in \left[n_{\omega }\right],t_j\in {\mathcal{G}}_{\omega }.\end{array}$$

We denote, by ${\tilde{\lambda }}_{j,k}\in \mathbb{R},t_j\in {\mathcal{G}}_{\omega },k\in \left[n_x\right],$ the discretized and evaluated dual variables of the ODE constraints (2b) in (OCP).

Our decomposition algorithm is based on the algorithmic choices of the sets $S^{\mathrm{CIA}}$ and $S^{\mathrm{REC}}$, which we define next.

Definition 5 

(${\mathit{S}}^{\mathbf{CIA}},{\mathit{S}}^{\mathbf{REC}}$). We introduce the set of CIA problems $S^{\mathrm{CIA}}$ as:

$$\begin{array}{cc}\hfill S^{\mathrm{CIA}}& :=\left\{\left(\mathrm{CIAmax}\right),\left(\mathrm{CIA}1\right),\left(\mathrm{CIAmaxB}\right),\left(\mathrm{CIA}1\mathrm{B}\right),\left(\lambda \mathrm{CIA}1\right),\left(\lambda \mathrm{CIA}1\mathrm{B}\right),\right.\hfill \\ \hfill & \left.\left(\mathrm{SCIAmax}\right),\left(\mathrm{SCIA}1\right),\left(\mathrm{SCIAmaxB}\right),\left(\mathrm{SCIA}1\mathrm{B}\right)\right\},\hfill \end{array}$$

where we define the specific CIA problems in the next section. For a subset ${\tilde{S}}^{\mathrm{CIA}}\subseteq S^{\mathrm{CIA}}$, we denote, with $n_{\mathrm{CIA}}:=\left|{\tilde{S}}^{\mathrm{CIA}}\right|$, the number of different CIA problem formulations. Let the elements of ${\tilde{S}}^{\mathrm{CIA}}$ be numbered by $1,\dots ,n_{\mathrm{CIA}}$. Let the set $S^{\mathrm{REC}}$ of recombination mappings $F^{\mathrm{rec}}\in S^{\mathrm{REC}}$ be defined via:

$$F^{\mathrm{rec}}:\underset{k\in \left[n_{\mathrm{CIA}}\right]}{\times }{\mathsf{\Omega }}_N\to {\mathsf{\Omega }}_N,F^{\mathrm{rec}}\left({\mathbf{w}}^1,\dots ,{\mathbf{w}}^{n_{\mathrm{CIA}}}\right)\mapsto {\mathbf{w}}^{\mathrm{rec}},$$

(3)

where ${\mathbf{w}}^k$ denotes the optimal solution of the problem ${\left(milp\right)}^k\in {\tilde{S}}^{\mathrm{CIA}}$.

We propose to use Algorithm 1 to approximate the solution of (MIOCP) efficiently with a priori bounds. Relaxing (MIOCP) to (OCP) results in state and control trajectories (line 1). We solve different MILPs to approximate the relaxed controls with binary ones in lines 2–3. Their performance is evaluated in line 4 by calculating their corresponding (feasible) state trajectories and objective values. In lines 6–7, we create new candidate binary controls in several recombination heuristics based on the existing binary controls, which we evaluate as well (line 8). As a final step, we select the best-performing binary control as the solution in line 10.

Algorithm 1: Decomposition of (MIOCP).

The main idea of Algorithm 1 is to decouple controls and states. We approximate the relaxed control function ${\mathbf{\alpha }}^{\ast }$ that is optimal for (OCP) with binary controls $\mathit{\omega }$, such that a good objective function value is obtained when (MIOCP) is evaluated for $\mathit{\omega }$. Which MILPs and recombination heuristics are used in the algorithm depends on the definition of the sets ${\tilde{S}}^{\mathrm{CIA}}$ and ${\tilde{S}}^{\mathrm{REC}}$, which we discuss in Section 3 and Section 5. A theoretical motivation and error bounds on the approximation quality are given in Section 4. Algorithm 1 is a generalization of the decomposition approach in [27,32], for which ${\tilde{S}}^{\mathrm{REC}}$ is empty and ${\tilde{S}}^{\mathrm{CIA}}$ contains only one CIA problem formulation.

3. Combinatorial Integral Approximation MILPs

In the following, we define the MILP formulations of CIA type for $S^{\mathrm{CIA}}$ in Algorithm 1. CIA, $\lambda$CIA, and SCIA refer to different scalings (Section 3.1), “1” and “max” to different norms $\parallel ·\parallel$ (Section 3.2), and the presence of “B” to a reversal of time (Section 3.3).

3.1. Combinatorial Integral Approximation and Scaled Variants

As part of the approximation step, we aim at finding binary control values that are close to the relaxed values with respect to the accumulated difference over all grid points. The following definition specifies the so-far-applied (CIA) problem together with two novel variants. We let the vector norm $\parallel ·\parallel$ be unspecified here, but discuss applicable norms later.

Definition 6 (${\theta }_{\mathrm{CIA}}^{\ast },{\theta }_{\mathrm{SCIA}}^{\ast },{\theta }_{\lambda \mathrm{CIA}}^{\ast }$, cf. Definition 4.17 in [50] ). 

Let ${\mathbf{a}}^{\ast }$ be the given optimal control solution of (OCP), and let the evaluated model function values $\tilde{\mathit{f}}$ and dual variables $\tilde{\mathbf{\lambda }}$ be given as introduced in Definition 4. Consider a vector norm $\parallel ·\parallel$. We introduce the following optimization problems:

$$\begin{array}{cc}\hfill {\theta }_{\mathrm{CIA}}^{\ast }& :=\underset{\mathbf{w}\in {\mathsf{\Omega }}_N}{min}\underset{j\in {[M-1]}_0}{max}\left|\left|\sum _{l\in \left[j\right]}({\mathbf{a}}_{·,l}^{\ast }-{\mathbf{w}}_{·,l}){\Delta }_l\right|\right|,\hfill \end{array}$$

(4)

$$\begin{array}{cc}\hfill {\theta }_{\mathrm{SCIA}}^{\ast }& :=\underset{\mathbf{w}\in {\mathsf{\Omega }}_N}{min}\underset{j\in {[M-1]}_0}{max}\left|\left|\sum _{l\in \left[j\right]}\sum _{i\in \left[n_{\omega }\right]}(a_{i,l}^{\ast }-w_{i,l}){\Delta }_l{\tilde{\mathit{f}}}_{i,l}\right|\right|,\hfill \end{array}$$

(5)

$$\begin{array}{cc}\hfill {\theta }_{\lambda \mathrm{CIA}}^{\ast }& :=\underset{\mathbf{w}\in {\mathsf{\Omega }}_N}{min}\underset{j\in {[M-1]}_0}{max}\left|\sum _{k\in \left[n_x\right]}{\tilde{\lambda }}_{j,k}\sum _{l\in \left[j\right]}\sum _{i\in \left[n_{\omega }\right]}(a_{i,l}^{\ast }-w_{i,l}){\Delta }_l{\tilde{\mathit{f}}}_{i,l,k}\right|.\hfill \end{array}$$

(6)

3.2. Norm Dependent MILP Formulation

By introducing the auxiliary variable $\theta$, we can reformulate (4) and (5) with the maximum norm.

Definition 7. 

(CIAmax, SCIAmax) Let ${\mathbf{a}}^{\ast }$ be the given optimal control solution of (OCP) and let the evaluated model function values $\tilde{\mathit{f}}$ be given as introduced in Definition 4. We define (CIAmax) as:

$$\begin{array}{cc}\hfill \underset{\theta \ge 0,\mathbf{w}\in {\mathsf{\Omega }}_M}{min}& \theta \hfill \end{array}$$

(7a)

$$\begin{array}{ccc}\hfill s.t.& \theta \ge \pm \sum _{j=0}^l(a_{i,j}^{\ast }-w_{i,j}){\Delta }_j,\hfill & \hfill fori\in \left[n_{\omega }\right],l\in {[M-1]}_0,\end{array}$$

(7b)

and (SCIAmax) as:

$$\begin{array}{cc}\hfill \underset{\theta \ge 0,\mathbf{w}\in {\mathsf{\Omega }}_M}{min}& \theta \hfill \end{array}$$

(8a)

$$\begin{array}{ccc}\hfill s.t.& \theta \ge \pm \sum _{j=0}^l\sum _{i=1}^{n_{\omega }}(a_{i,j}^{\ast }-w_{i,j}){\Delta }_j{\tilde{\mathit{f}}}_{i,j,k},\hfill & \hfill fork\in \left[n_x\right],l\in {[M-1]}_0.\end{array}$$

(8b)

We introduce the MILP analogue formulations for the Manhattan norm with auxiliary variables $s_{i,j}\ge 0,i\in \left[n_{\omega }\right],j\in {[M-1]}_0$. In this way, we specify the norm choices for Definition 6.

Definition 8. 

(CIA1, SCIA1, λCIA1) Consider ${\mathbf{a}}^{\ast }$ as the given optimal control solution of (OCP). Let the evaluated model function values $\tilde{\mathit{f}}$ and dual variables $\tilde{\mathbf{\lambda }}$ be given as introduced in Definition 4. Based on the auxiliary variables $s_{i,l}$ we define (CIA1) as:

$$\begin{array}{cc}\hfill \underset{\theta ,s_{i,l}\ge 0,\mathbf{w}\in {\mathsf{\Omega }}_M}{min}& \theta \hfill \end{array}$$

(9)

$$\begin{array}{ccc}\hfill s.t.\theta & \ge \sum _{i=1}^{n_{\omega }}{s}_{i,l},\hfill & \hfill forl\in {[M-1]}_0,\end{array}$$

(10)

$$\begin{array}{ccc}\hfill s_{i,l}& \ge \pm \sum _{j=0}^l(a_{i,j}^{\ast }-w_{i,j}){\Delta }_j,\hfill & \hfill fori\in \left[n_{\omega }\right],l\in {[M-1]}_0.\end{array}$$

(11)

With a different dimension for $s_{k,l}$, we define (SCIA1) as:

$$\begin{array}{cc}\hfill \underset{\theta ,s_{k,l}\ge 0,\mathbf{w}\in {\mathsf{\Omega }}_M}{min}& \theta \hfill \end{array}$$

(12a)

$$\begin{array}{ccc}\hfill s.t.\theta & \ge \sum _{k=1}^{n_x}{s}_{k,l},\hfill & \hfill forl\in {[M-1]}_0,\end{array}$$

(12b)

$$\begin{array}{ccc}\hfill s_{k,l}& \ge \pm \sum _{j=0}^l\sum _{i=1}^{n_{\omega }}(a_{i,j}^{\ast }-w_{i,j}){\Delta }_j{\tilde{\mathit{f}}}_{i,j,k},\hfill & \hfill fork\in \left[n_x\right],l\in {[M-1]}_0.\end{array}$$

(12c)

Finally, we define ($\lambda$CIA1)by modifying (12c) with the dual variables:

$$\begin{array}{cc}\hfill \underset{\theta ,s_{k,l}\ge 0,\mathbf{w}\in {\mathsf{\Omega }}_M}{min}& \theta \hfill \end{array}$$

(13a)

$$\begin{array}{ccc}\hfill s.t.\theta & \ge \sum _{k=1}^{n_x}{s}_{k,l},\hfill & \hfill forl\in {[M-1]}_0,\end{array}$$

(13b)

$$\begin{array}{ccc}\hfill s_{k,l}& \ge \pm {\lambda }_{l,k}·\sum _{j=0}^l\sum _{i=1}^{n_{\omega }}(a_{i,j}^{\ast }-w_{i,j}){\Delta }_j{\tilde{\mathit{f}}}_{i,j,k},\hfill & \hfill fork\in \left[n_x\right],l\in {[M-1]}_0.\end{array}$$

(13c)

Of course, also other norms, such as the Euclidean norm, can be used. We do not consider them here because of the resulting nonlinearity of the constraints.

3.3. Chronologically Ordered Constraints

We consider the possibility to modify the (CIA) problems by altering the chronological order in the constraints for the accumulated difference $\parallel {\sum }_{l\in \left[j\right]}({\mathbf{a}}_{·,l}^{\ast }-{\mathbf{w}}_{·,l}){\Delta }_l\parallel$ for $j\in {[M-1]}_0$. We may use an arbitrary ordering of time intervals, instead of starting from the first interval $j=0$. Here, we consider backward accumulation, starting from the interval with index $j=M-1$, i.e., $[t_{M-1},t_M]$:

$$\begin{array}{cc}\hfill \theta \ge \pm \sum _{j=l}^{M-1}(a_{i,j}^{\ast }-w_{i,j}){\Delta }_j,& fori\in \left[n_{\omega }\right],l\in {[M-1]}_0.\hfill \end{array}$$

(14)

Let the problem where (14) replaces (7b) in (CIAmax) be denoted by (CIAmaxB). The other introduced MILPs can be modified analogously with backward time accumulation and are named accordingly; e.g., (SCIA1B) refers to (SCIA1) with backward accumulation.

3.4. Combinatorial Constraints

One advantage of using an MILP for obtaining binary controls after the relaxation step, rather than using SUR, is the possibility to impose combinatorial constraints that couple over time. An example of such constraints is to limit the number of allowed switches ${\sigma }_{\max }\in \mathbb{N}$ between activated binary controls, which can be formulated as:

$${\sigma }_{\max }\ge \frac{1}{2}\sum _{i=1}^{n_{\omega }}\sum _{j=1}^{M-1}|w_{i,j}-w_{i,j-1}|.$$

(15)

We will omit this constraint class in the next section, but refer to [51,52] for a priori bounds and reformulations. Nevertheless, we are going to come back to these constraints in the numerical experiments section for also testing the different MILP approaches in this situation.

4. A Priori Bounds for CIA Decompositions

We revise results for the a priori bounds resulting from a CIA decomposition [41] and extend them to alternative decompositions that may be used in $S^{\mathrm{CIA}}$ in Algorithm 1. We stress here that Manns et al. [53] have recently presented a proof of improved regularity conditions. Nevertheless, we revise the theorem from [41] here, because it results in natural algorithmic extensions.

4.1. Combinatorial Integral Approximation

We recapitulate a variant of Grönwall’s Lemma from [41], which is needed for the main theorem.

Lemma 1 (A variant of Grönwall’s Lemma, see [41], Lemma 1). 

Let $z_1,z_2:\mathcal{T}\to \mathbb{R}$ be real-valued integrable functions and let $z_2$ also belong to $L^{\infty }(\mathcal{T},\mathbb{R})$. If, for a constant $L\ge 0$, the following holds:

$$z_1\left(t\right)\le z_2\left(t\right)+L{\int }_{t_0}^t{z}_1\left(\tau \right)\mathrm{d}\tau fora.a.t\in \mathcal{T},$$

then we have:

$$z_1\left(t\right)\le {\parallel z_2\parallel }_{\infty }{\mathrm{e}}^{L(t-t_0)}fora.a.t\in \mathcal{T}.$$

(16)

Proof. 

See [41], proof of Lemma 1.   □

In the following results, we analyze the evolution of two trajectories, $\mathbf{x}$ and $\mathbf{y}$, based on the same ODE system (2b) but driven by two different controls, $\mathbf{\alpha }$ and $\mathit{\omega }$. The following theorem gives a statement on the distance between the two trajectories depending on the distance of the controls.

Theorem 1. 

Consider $\mathbf{\alpha }$ and $\mathit{\omega }\in \mathsf{\Omega }$. We reuse the model functions ${\mathit{f}}_0,{\mathit{f}}_i:\mathcal{T}\times {\mathbb{R}}^{n_x}\to {\mathbb{R}}^{n_x}$ from Definition 2 for $i\in \left[n_{\omega }\right]$. Let $\mathbf{x}(·)$ and $\mathbf{y}(·)$ be the unique solutions of the IVP:

$$\begin{array}{cc}\hfill \dot{\mathbf{x}}\left(t\right)& ={\mathit{f}}_0(t,\mathbf{x}\left(t\right))+\sum _{i=1}^{n_{\omega }}{\alpha }_i\left(t\right){\mathit{f}}_i(t,\mathbf{x}\left(t\right)),\mathbf{x}\left(t_0\right)={\mathbf{x}}_0,\hfill \end{array}$$

(17a)

$$\begin{array}{cc}\hfill \dot{\mathbf{y}}\left(t\right)& ={\mathit{f}}_0(t,\mathbf{y}\left(t\right))+\sum _{i=1}^{n_{\omega }}{\omega }_i\left(t\right){\mathit{f}}_i(t,\mathbf{y}\left(t\right)),\mathbf{y}\left(t_0\right)={\mathbf{y}}_0,\hfill \end{array}$$

(17b)

where ${\mathbf{x}}_0,{\mathbf{y}}_0\in {\mathbb{R}}^{n_x}$. Assume that there are positive constants $L,C\in {\mathbb{R}}^{+},$ together with a vector norm $\left|\left|·\right|\right|$, such that, for a.a., $t\in \mathcal{T}$ holds:

$$\begin{array}{cc}\hfill \left|\left|{\mathit{f}}_i(t,\mathbf{x}\left(t\right))-{\mathit{f}}_i(t,\mathbf{y}\left(t\right))\right|\right|& \le L\left|\left|\mathbf{x}\left(t\right)-\mathbf{y}\left(t\right)\right|\right|,fori\in {\left[n_{\omega }\right]}_0,\hfill \end{array}$$

(17c)

$$\begin{array}{cc}\hfill \left|\left|\frac{\mathrm{d}}{\mathrm{d}t}{\mathit{f}}_i(t,\mathbf{x}\left(t\right))\right|\right|& \le C,fori\in \left[n_{\omega }\right].\hfill \end{array}$$

(17d)

Furthermore, let ${\mathit{f}}_i(·,\mathbf{x}(·)),i\in \left[n_{\omega }\right]$ be essentially bounded by $B\in {\mathbb{R}}^{+}$ on $\mathcal{T}$, and assume that for all $t\in \mathcal{T}$. It holds that:

$$\begin{array}{c}\hfill \left|\left|{\int }_{t_0}^t\alpha \left(\tau \right)-\omega \left(\tau \right)\mathrm{d}\tau \right|\right|\le \theta ,\end{array}$$

(17e)

with the constant $\theta \ge 0$. Then, for a.a. $t\in \mathcal{T}$ we also have:

$$\left|\left|\mathbf{x}\left(t\right)-\mathbf{y}\left(t\right)\right|\right|\le \left(\left|\left|{\mathbf{x}}_0-{\mathbf{y}}_0\right|\right|+\theta n_{\omega }\left(B+C(t-t_0)\right)\right){\mathrm{e}}^{L(t-t_0)}.$$

(17f)

Proof. 

This Theorem is an (equivalent) reformulation of Theorem 2 from [41]. The only differences are modified notations and the usage of ${\mathit{f}}_i,i\in {\left[n_{\omega }\right]}_0$ as differentiable mapping instead of $\mathit{A}$ in [41].    □

We first recognize that Theorem 1 is applicable for the CIA problem with any vector norm, which can be guessed from the equivalence of norms.

Corollary 1 (Approximation bounds via (CIA), cf. Corollary 5.2 in [50]). 

Consider the setting of Theorem 1; in particular, let the regularity assumptions on ${\mathit{f}}_i,i\in {\left[n_{\omega }\right]}_0,$ hold. Assume that $\mathbf{x}$ and ${\mathbf{y}}_{\mathrm{CIA}}$ are the solutions of the IVP (2b) and (2c), where $\mathbf{x}$ is based on a given relaxed control $\mathbf{a}$ and ${\mathbf{y}}_{\mathrm{CIA}}$ is based on ${\mathbf{w}}^{\ast }$, which is the optimal solution of (CIA$no$), $no\in \{max,1\}$, with objective value ${\theta }_{\mathrm{CIA}}^{\ast }$ from Definition 6. Then, the state approximation error is bounded for a.a. $t\in \mathcal{T}$ by:

$$\parallel \mathbf{x}\left(t\right)-{\mathbf{y}}_{\mathrm{CIA}}\left(t\right)\parallel \le {\theta }_{\mathrm{CIA}}^{\ast }(B+C(t-t_0)){\mathrm{e}}^{L(t-t_0)}.$$

(18)

Proof. 

This corollary is a direct result from Theorem 1 with ${\mathbf{x}}_0={\mathbf{y}}_0$. We note that ${\theta }_{\mathrm{CIA}}^{\ast }$ represents the norm of the accumulated control deviation, which appears in the proof of the Theorem and is bounded by $\theta n_{\omega }$ so that this replaced term is settled.    □

4.2. Scaled Combinatorial Integral Approximation

The proof of Theorem 1 contains a motivation for the (SCIA) problems as derived in the following result.

Corollary 2 (Approximation bounds via (SCIA), cf. Corollary 5.3 in [50]). 

Consider the setting of Theorem 1, and let ${\parallel ·\parallel }_{no}$ refer to the maximum or 1-norm, i.e., $no\in \{max,1\}$. Assume that $\mathbf{x}$ and ${\mathbf{y}}_{\mathrm{CIA}}$ are the solutions of the IVP (2b) and (2c), where $\mathbf{x}$ is based on a given $\mathbf{a}$, and ${\mathbf{y}}_{\mathrm{CIA}}$ is driven by ${\mathbf{w}}^{\ast }$, which is the optimal solution of (SCIA$no$). Then, for a.a. $t\in \mathcal{T}$, the state approximation error is bounded by:

$$\parallel \mathbf{x}\left(t\right)-{\mathbf{y}}_{\mathrm{SCIA}}\left(t\right)\parallel \le {\theta }_{\mathrm{SCIA}}^{\ast }{\mathrm{e}}^{L(t-t_0)}\le {\theta }_{\mathrm{CIA}}^{\ast }(B+C(t-t_0)){\mathrm{e}}^{L(t-t_0)},$$

(19)

where ${\theta }_{\mathrm{SCIA}}^{\ast }$ is the optimal objective value of (SCIA$no$), $no\in \{max,1\}$.

Proof. 

From the second and last inequalities in the proof of Theorem 2 from [41] and Corollary 1, it follows that, for a.a. $t\in \mathcal{T}$ and any $\mathit{\omega }\in \mathsf{\Omega }$:

$$∥{\int }_{t_0}^t\sum _{i=1}^{n_{\omega }}({\alpha }_i\left(\tau \right)-{\omega }_i\left(\tau \right)){\mathit{f}}_i\left(\mathbf{x}\left(\tau \right)\right)\mathrm{d}\tau ∥\le ∥{\int }_{t_0}^t\mathbf{\alpha }\left(\tau \right)-\mathit{\omega }\left(\tau \right)\mathrm{d}\tau ∥(B+C(t-t_0)).$$

Let ${\mathit{\omega }}^{\mathrm{CIA}}$ denote the control based on the optimal solution ${\mathbf{w}}^{\ast }$ of (CIA$no$), $no\in \{max,1\}$. We take the minimum in the above inequality and obtain:

$${\theta }_{\mathrm{SCIA}}^{\ast }\le ∥{\int }_{t_0}^t\sum _{i=1}^{n_{\omega }}({\alpha }_i\left(\tau \right)-{\omega }_i^{\mathrm{CIA}}\left(\tau \right)){\mathit{f}}_i\left(\mathbf{x}\left(\tau \right)\right)\mathrm{d}\tau ∥\le {\theta }_{\mathrm{CIA}}^{\ast }(B+C(t-t_0)).$$

   □

We conclude from Corollary 1 that the approximation and convergence results of the decomposition still hold if (SCIA$no$), $no\in \{max,1\}$, is used to construct the binary control. The approximation bound based on (SCIA$no$) is tighter than the existing (CIA$no$)-related bound. Hence, it is an obvious choice to consult these alternative binary controls for an approximation study. Ideally, the binary control constructed in this way will result in an improved state approximation and objective value for (MIOCP). However, we oppose this hope next.

Remark 1 (Construction by (SCIA) does not guarantee superior quality). 

The binary control based on the optimal solution of (SCIA$no$), $no\in \{max,1\}$ and used in the decomposition does not necessarily result in a state approximation or objective value that is superior to that obtained using (CIA$no$). It may hold that:

$$\parallel \mathbf{x}\left(t\right)-{\mathbf{y}}_{\mathrm{CIA}}\left(t\right)\parallel <\parallel \mathbf{x}\left(t\right)-{\mathbf{y}}_{\mathrm{SCIA}}\left(t\right)\parallel <{\theta }_{\mathrm{SCIA}}^{\ast },forsomet\in \mathcal{T},$$

where we use the notation from Corollaries 1 and 2. Because of a possible non-convex objective, the computed trajectories may lead to a superior objective value for the solution based on (CIA$no$), compared with that based on (SCIA$no$), even if the above inequality does not hold.

4.3. $\lambda$-Combinatorial Integral Approximation

The $\lambda$-approximation stems from the aforementioned theorem of approximating differential states, but with assessing the difference of the cost-to-go function. We define it in a more general MIOCP setting with a Bolza objective:

$$\mathsf{\Phi }\left(\mathbf{x}\left(t_f\right)\right)+\underset{t\in \mathcal{T}}{\int }L(\mathbf{x}\left(\tau \right),\omega \left(\tau \right))\mathrm{d}\tau ,$$

(20)

where $L\in C^1({\mathbb{R}}^{n_x+1}\times {\mathbb{R}}^{n_{\omega }},\mathbb{R})$.

Definition 9 (Cost-to-go function by Hamilton–Jacobi–Bellman). 

Let the cost-to-go function $J\in L^{\infty }({\mathbb{R}}^{n_x}\times \mathcal{T},\mathbb{R})$ with Bolza objective (20) be implicitly defined as:

$$\begin{array}{cc}\hfill J(\mathbf{x}\left(t_f\right),t_f)& =\mathsf{\Phi }\left(\mathbf{x}\left(t_f\right)\right),\hfill \\ \hfill -{\displaystyle \frac{\partial J}{\partial t}}(\mathbf{x}\left(t\right),t)& =\underset{\omega \in \mathsf{\Omega }}{min}L(\mathbf{x}\left(t\right),\omega \left(t\right))+{\displaystyle \frac{\partial J}{\partial x}}(\mathbf{x}\left(t\right),t)\left({\mathit{f}}_0(t,\mathbf{x}\left(t\right))+\sum _{i=1}^{n_{\omega }}{\omega }_i\left(t\right){\mathit{f}}_i(t,\mathbf{x}\left(t\right))\right).\hfill \end{array}$$

We recognize that $\frac{\partial J}{\partial x}$ can be interpreted as Lagrange multiplier or dual variables of the ODE constraints in (MIOCP). Therefore, we write, in short, $\lambda \left(t\right)$ instead of $\frac{\partial J}{\partial x}$. With the groundwork above, we are ready to deduce the corresponding bound.

Corollary 3 (Approximation bounds via ($\mathbf{\lambda }$CIA), cf. Corollary 5.4 in [50]). 

Consider the setting of Theorem 1. In particular, let the regularity assumptions (17d), (17c), and the essential boundedness of $\mathit{f}$ be true. Assume that $\mathbf{x}$ and ${\mathbf{y}}_{\lambda \mathrm{CIA}}$ are the solutions of the IVP (2b) and (2c), where $\mathbf{x}$ is based on a given $\mathbf{a}$, and ${\mathbf{y}}_{\lambda \mathrm{CIA}}$ is based on ${\mathbf{w}}^{\ast }$, which is the optimal solution of (λCIA1). Let J be the cost-to-go function as defined in Definition 9 for (MIOCP) and $\mathbf{\lambda }\left(t\right)$ be the adjoint vector at $t\in \mathcal{T}$ for the ODE system (2b). For a.a. $t\in \mathcal{T}$, it follows that:

$$|J(\mathbf{x}\left(t\right),t)-J({\mathbf{y}}_{\lambda \mathrm{CIA}}\left(t\right),t)|\le {\theta }_{\lambda \mathrm{CIA}}^{\ast }{\mathrm{e}}^{L(t-t_0)}+o\left(\parallel \mathbf{x}\left(t\right)-{\mathbf{y}}_{\lambda \mathrm{CIA}}{\left(t\right)\parallel }^2\right),$$

where o refers to Landau’s little-o notation.

Proof. 

Let us consider the difference of the cost-to-go functions by approximation with a partial first-order Taylor expansion around $J\left(\mathbf{x}\right(t),t)$. We perform the approximation with respect to the trajectories $\mathbf{x},{\mathbf{y}}_{\lambda \mathrm{CIA}}$. Thus, we apply Taylor’s theorem for $t\in \mathcal{T}$:

$$J({\mathbf{y}}_{\lambda \mathrm{CIA}}\left(t\right),t)-J(\mathbf{x}\left(t\right),t)={\displaystyle \frac{\partial J}{\partial \mathbf{x}}}(\mathbf{x}\left(t\right),t)\left({\mathbf{y}}_{\lambda \mathrm{CIA}}\left(t\right)-\mathbf{x}\left(t\right)\right)+o\left({\left|\left|{\mathbf{y}}_{\lambda \mathrm{CIA}}\left(t\right)-\mathbf{x}\left(t\right)\right|\right|}^2\right).$$

(21)

As pointed out above, the dual variables of the ODE constraint (2b) are equal to $\frac{\mathrm{d}J}{\mathrm{d}\mathbf{x}}(\mathbf{x}\left(t\right),t)$. We use the notation ${\parallel \mathbf{x}\left(t\right)\parallel }_{\lambda \left(t\right)}:=\left|{\sum }_{k\in \left[n_x\right]}{\lambda }_k\left(t\right)x_k\left(t\right)\right|$ for $t\in \mathcal{T}$, which defines a semi-norm. Next, for a.a. $t\in \mathcal{T}$, we transfer the proof of Theorem 1 to this notation and to (21):

$$\begin{array}{cc}\hfill |J({\mathbf{y}}_{\lambda \mathrm{CIA}}\left(t\right),t)-J(\mathbf{x}\left(t\right),t)|& \le \parallel {\mathbf{y}}_{\lambda \mathrm{CIA}}{-\mathbf{x}\parallel }_{\lambda \left(t\right)}+o\left({\left|\left|{\mathbf{y}}_{\lambda \mathrm{CIA}}-\mathbf{x}\right|\right|}^2\right)\hfill \\ & \le \dots (\mathrm{as}\mathrm{in}\mathrm{proof}\mathrm{of}\mathrm{Theorem}1)\hfill \\ & \le {\left|\left|{\mathbf{y}}_0-{\mathbf{x}}_0\right|\right|}_{\lambda \left(t\right)}+L{\int }_{t_0}^t{\left|\left|{\mathbf{y}}_{\lambda \mathrm{CIA}}\left(\tau \right)-\mathbf{x}\left(\tau \right)\mathrm{d}\tau \right|\right|}_{\lambda \left(t\right)}\hfill \\ & +{\left|\left|\sum _{i=1}^{n_{\omega }}{\int }_{t_0}^t({\mathit{\omega }}_i^{\lambda \mathrm{CIA}}\left(\tau \right)-{\mathbf{\alpha }}_i\left(\tau \right))·{\mathit{f}}_i(\tau ,\mathbf{x}\left(\tau \right))\mathrm{d}\tau \right|\right|}_{\lambda \left(t\right)}\hfill \\ & +o\left({\left|\left|{\mathbf{y}}_{\lambda \mathrm{CIA}}-\mathbf{x}\right|\right|}^2\right).\hfill \end{array}$$

The third summand of the last inequality is equal to the objective ${\theta }_{\lambda \mathrm{CIA}}^{\ast }$ that is to be minimized in ($\lambda$CIA1). Finally, we apply ${\mathbf{x}}_0={\mathbf{y}}_0$ and use the Grönwall Lemma 1 with the integrable functions:

$$z_1\left(t\right)={∥\mathbf{x}-{\mathbf{y}}_{\lambda \mathrm{CIA}}∥}_{\lambda \left(t\right)},z_2\left(t\right)={\left|\left|\sum _{i=1}^{n_{\omega }}{\int }_{t_0}^t({\mathit{\omega }}_i^{\lambda \mathrm{CIA}}\left(\tau \right)-{\mathbf{\alpha }}_i\left(\tau \right))·{\mathit{f}}_i(\tau ,\mathbf{x}\left(\tau \right))\mathrm{d}\tau \right|\right|}_{\lambda \left(t\right)},$$

so that the claim is proven.    □

Due to the first-order Taylor approximation, ($\lambda$CIA1) requires that the relaxed trajectory $\mathbf{x}$ can be well approximated by a trajectory $\mathbf{y}$ that is based on binary controls. On the other hand, if there is no such trajectory in a close neighborhood of $\mathbf{x}$, then ($\lambda$CIA1) may have an unintuitive binary control as its optimal solution.

4.4. Backwards Accumulating Constraints

If we adapt the MIOCP instance with fixed final states ${\mathbf{x}}_{\mathit{f}}\in {\mathbb{R}}^{n_x}$ and with a Lagrangian objective type, we can also apply this modified setting to Theorem 1. We express this issue in the following corollary.

Corollary 4 

(Approximation bounds via backward constraints). Consider the setting of Theorem 1. Let $\mathbf{x}$ and $\mathbf{y}$ be the state trajectory solutions of the terminal value problems (2b), with $\mathbf{x}\left(t_f\right)={\mathbf{x}}_f$ and $\mathbf{y}\left(t_f\right)={\mathbf{y}}_f$ for ${\mathbf{x}}_f,{\mathbf{y}}_f\in {\mathbb{R}}^{n_x}$. Assume that for all $t\in \mathcal{T},{\theta }_b\in {\mathbb{R}}^{+}$, it holds that:

$$∥{\int }_t^{t_f}\alpha \left(\tau \right)-\omega \left(\tau \right)\mathrm{d}\tau ∥\le {\theta }_b.$$

(22)

Then, for a.a., $t\in \mathcal{T}$ it also holds that:

$$\left|\left|\mathbf{x}\left(t\right)-\mathbf{y}\left(t\right)\right|\right|\le \left(\left|\left|{\mathbf{x}}_f-{\mathbf{y}}_f\right|\right|+{\theta }_b{n}_{\omega }\left(B+C(t_f-t)\right)\right){\mathrm{e}}^{L(t_f-t)}.$$

(23)

Proof. 

We apply the proof of Theorem 1 to the altered setting, in which we integrate over $[t,t_f]$ instead of integrating over $[t_0,t]$.

   □

With the assumption that $\parallel \mathbf{x}\left(t_f\right)-\mathbf{y}\left(t_f\right)\parallel$ is small, the backward (CIA) rounding problem approach from Section 3.3 is not only applicable for terminal constraint problems but is also appropriate for an (MIOCP) instance with a given initial value ${\mathbf{x}}_0$ and variable final-state values.

4.5. Connection to Decomposition Algorithm and Optimization Problem

As a last step in this chapter, the previous results are related to (MIOCP) and the decomposition Algorithm 1.

Remark 2 (Arbitrary close approximation of (OCP) solution). 

We could extend Algorithm 1 with an outer loop that checks if ${\mathsf{\Phi }}^{\mathrm{opt}}$ is sufficiently close to ${\mathsf{\Phi }}^{\mathrm{rel}}$, and if not, we would refine the grid ${\mathcal{G}}_{\omega }$. In [29], an arbitrary close approximation of the (OCP) solution has been deduced with this procedure and based on (CIA). With the assumption of $\mathsf{\Phi }(·),c(·)$ being continuous and that there exists a feasible trajectory $\mathbf{x}$ for (OCP), it follows that for any $ϵ>0$ there exists a grid ${\mathcal{G}}_{\omega }$, with grid size ${\Delta }_{\max }$, such that there is a feasible trajectory ${\left\{\mathbf{y}\left(t_j\right)\right\}}_{t_j\in {\mathcal{G}}_{\omega }}$ with:

$$\begin{array}{cc}\hfill |\mathsf{\Phi }\left(\mathbf{x}\left(t_f\right)\right)-\mathsf{\Phi }\left(\mathbf{y}\left(t_f\right)\right)|& \le ϵ,\hfill \\ \hfill \parallel \mathbf{c}(t_j,\mathbf{x}\left(t_j\right))-\mathbf{c}(t_j,\mathbf{y}\left(t_j\right))\parallel & \le ϵ.\hfill \end{array}$$

The proof uses the sum-up rounding scheme that derives the binary control approximation with ${\theta }_{\mathrm{CIA}}^{\ast }\le \mathrm{Const}\left(n_{\omega }\right){\Delta }_{\max }$ [35,41], where $\mathrm{Const}\left(n_{\omega }\right)$ is a constant depending on $n_{\omega }$. In case we extend Algorithm 1 with the refinement procedure, and if (CIA) or (SCIA) are chosen to be elements of $S^{\mathrm{CIA}}$, the same approximation result holds.

Corollary 5 (Solution accuracy of differential states of Algorithm 1). 

Let ${\theta }_{\mathrm{SCIA},\max }^{\ast },{\theta }_{\mathrm{SCIA},1}^{\ast }$ be the optimal objective values of (SCIAmax) and (SCIA1), respectively. Consider the setting of Theorem 1. In particular, let the regularity assumptions (17d), (17c), and essential boundedness of $\mathit{f}$ be true. Assume that $\mathbf{x}$ and $\mathbf{y}$ are the solutions of the IVP (2b) and (2c), where $\mathbf{x}$ is based on a given $\mathbf{a}$, and $\mathbf{y}$ is based on Algorithm 1. Let ${\mathcal{G}}_{\omega }$ be the applied grid. It follows for $t_i\in {\mathcal{G}}_{\omega }$ that:

$$\begin{array}{cc}\hfill {\left|\left|\mathbf{y}\left(t_i\right)-\mathbf{x}\left(t_i\right)\right|\right|}_j& \le \left({\left|\left|{\mathbf{y}}_{\mathbf{0}}-{\mathbf{x}}_{\mathbf{0}}\right|\right|}_j+{\theta }_{\mathrm{SCIA},j}^{\ast }\right)e^{L(t_i-t_0)},j\in \{\max ,1\}.\hfill \end{array}$$

(24)

Proof. 

The claim is a direct result of Corollary 2.    □

We have deliberately chosen the tightest bound from the previous corollaries, but could also use others. The received grid-specific rounding error bounds aim primarily at approximating the differential states. With ($\lambda$CIA) or Lipschitz continuity of the objective, there are also tools to discuss the rounding error of the objectives. The recombination heuristics work in the area of objective approximation and, hence, are to be discussed next.

5. Recombination Heuristics

We present several recombination heuristics that recombine different binary controls $\mathbf{w}$ to new candidate solutions with potentially smaller objective values. The general framework is open to apply different heuristics, such as genetic algorithms [54], that are not introduced in this article.

5.1. GreedyTime

Algorithm 2 is a routine for using the MILP solutions in a greedy pattern with the aim of constructing binary controls $\mathbf{w}$ that exhibit an improved objective value $\mathsf{\Phi }\left(\phi \right(\mathbf{w}\left)\right)$.   

Algorithm 2: GreedyTime heuristic for finding improved $\mathbf{w}$ variables.

In GreedyTime, we iterate over all intervals $j\in {[M-1]}_0$ in chronological order (line 1). On every interval, we check if there are MILP pairs $(m_1,m_2)$ that differ in their binary control vectors in line 2. We recombine for each of these pairs the $m_1$ solution with the binary control vector from $m_2$ at interval j to construct a temporary control solution ${\tilde{\mathbf{w}}}^{m_1}$ (line 3). Based on this construction, we evaluate the objective of this new control solution in line 4 and overwrite the binary control ${\mathbf{w}}^{m_1}$ with the recombined solution ${\tilde{\mathbf{w}}}^{m_1}$ if that latter results in an improved objective value (lines 5–6). We proceed in the same way with the second solution $m_2$ when the (same) pair $(m_2,m_1)$ appears in the inner loop.

A large number of calculated MILPs may result in a large number of pairs with unequal control solutions. In this case, it is advisable to only swap the ${\mathbf{w}}^{m_2}$ solution with the currently smallest objective value, instead of swapping and testing each variation for every ${\mathbf{w}}^{m_1}$.

If the control problem at hand involves no continuous controls $\mathbf{u}$, it is straightforward to evaluate the (MIOCP) in line 4 with the previously found and fixed $\mathbf{x}$ until grid interval j. This speeds up the process in problems with fine grids and numerous MILP solutions, because (MIOCP) needs to be solved iteratively. Moreover, if an MILP solution $m_1$ differs from two MILPs $m_2,m_3$ with identical binary control vectors ${\mathbf{w}}_{·,j}^{m_2}={\mathbf{w}}_{·,j}^{m_3}$, it is sufficient to test the recombination with only one of the two. We illustrate an example recombination step for the pairs (CIA, SCIA) and (SCIA, CIA) in Figure 1.

Remark 3 (Modifications of GreedyTime). 

1. 

We can also apply the outer loop in Algorithm 2 backward in time and name the backward version GreedyTimeBackward;

2. 

We may consider only singular arcs, instead of looping over all intervals, since the constructed binary controls are likely to be equal on bang–bang arcs. With singular arcs, we mean the intervals where $ϵ<a_{i,j}^{\ast }<1-ϵ$ holds for the optimal control solution ${\mathbf{a}}^{\ast }$ of (OCP), with a certain threshold $ϵ>0$;

3. 

Greedy-cost-to-go modification: Assume we have obtained the dual variables ${\tilde{\lambda }}_{j,k},j\in {[M-1]}_0,k\in \left[n_x\right],$ of the state equations of (OCP). Then, re-sort the intervals ${[M-1]}_0$ in descending order according to ${\sum }_{k\in \left[n_x\right]}\left|{\tilde{\lambda }}_{j,k}\right|,j\in {[M-1]}_0.$ In this way, we construct a new ordered grid ${\mathcal{G}}_{\omega }^{\lambda }$ to be iterated over in Algorithm 2.

5.2. Singular Arc Recombination

If ${\mathbf{a}}^{\ast }$ is (almost) binary on certain intervals, $\mathbf{w}$ should typically attain these binary values as well as an optimal solution of a CIA rounding problem—regardless of the MILP choice. To this end, we formalize singular arcs of ${\mathbf{a}}^{\ast }$ as sets of consecutive intervals on which the relaxed control takes values smaller than $ϵ$ or larger than $1-ϵ$. Here, $ϵ>0$ is a chosen small tolerance.

Definition 10 (Number of singular arcs ${\mathit{n}}_{\mathrm{sing}}$, singular arc interval sets ${\mathcal{J}}_{\mathbf{l}}^{\mathrm{sing}}$). 

Consider ${\mathbf{a}}^{\ast }$ as the optimal control solution of (OCP) and a small chosen tolerance $ϵ>0$. Let $k_0^{\mathrm{end}}:=-1$. We introduce the following singular arc interval index sets iteratively for $l\ge 1$:

$$\begin{array}{cc}\hfill k_l^{\mathrm{start}}& :=min\left\{j\in {[M-1]}_0\mid j>k_{l-1}^{\mathrm{end}}\wedge \exists i\in \left[n_{\omega }\right]:a_{i,j}^{\ast }\in [ϵ,1-ϵ]\right\},\hfill \\ \hfill k_l^{\mathrm{end}}& :=max\left\{j\in {[M-1]}_0\mid \forall r=k_l^{\mathrm{start}},\dots ,j\exists i\in \left[n_{\omega }\right]:a_{i,r}^{\ast }\in [ϵ,1-ϵ]\right\},\hfill \\ \hfill {\mathcal{J}}_l^{\mathrm{sing}}& =\left\{k_l^{\mathrm{start}},\dots ,k_l^{\mathrm{end}}\right\}.\hfill \end{array}$$

Let the number of singular arcs $n_{\mathrm{sing}}$ be defined as:

$$n_{\mathrm{sing}}:=arg\underset{l\in \mathbb{N}}{max}\left\{k_l^{\mathrm{end}}\right\}.$$

We aim to recombine singular arc realizations of the different MILP solutions from $S^{\mathrm{CIA}}$, which is performed in Algorithm 3.

We initialize the set of visited binary controls as empty in the algorithm and set the so-far best objective value ${\mathsf{\Phi }}^{\mathrm{rec}}$ to infinity (line 1). Next, the temporary binary control ${\mathbf{w}}^{\mathrm{tmp}}$ on the bang–bang arcs is set as equal to the rounded relaxed control (line 2). In line 3, we test every possible variation of the different MILP solutions on the singular arcs. In this way, we fill up the singular arcs of the temporary binary control ${\mathbf{w}}^{\mathrm{tmp}}$ (line 4). The constructed control ${\mathbf{w}}^{\mathrm{tmp}}$ is checked if it has already been visited (line 5), and if so, the algorithm jumps to the next iteration (line 6). Otherwise, we include ${\mathbf{w}}^{\mathrm{tmp}}$ in the set of visited controls (line 8), and we evaluate its objective value (line 9). When a recombination has an improved objective value than the so-far best control, it will be saved as the so-far best control (lines 10–12). We illustrate the algorithm in Figure 2.

Algorithm 3:Singular arc block heuristic for recombining binary controls ${\mathbf{w}}^m$, $\mathrm{m}\in S^{\mathrm{CIA}}$.

We have to take care of the number of possible variations of singular blocks and MILP solutions $|S^{\mathrm{CIA}}{|}^{n_{\mathrm{arc}}}$ to avoid a combinatorial explosion. Therefore, it is advisable to choose ${\tilde{S}}^{\mathrm{CIA}}$ in Algorithm 1 with a small number of MILPs. Only a few singular arcs result usually after solving (OCP). We may modify Algorithm 3 to be greedy, i.e., to apply the idea of GreedyTime on arcs instead of on single intervals, if there are more than four singular arcs.

The singular arc recombination yields an objective value that is at least as good as those previously constructed via the MILPs. However, there currently exists no framework for quantifying these possible improvements in terms of new rounding errors of the objective.

6. Computational Results

6.1. Software Implementation and Instances

We implemented Algorithm 1 in AMPL [55] using the code ampl_mintoc, which is a modeling framework to solve optimal control problems. It features different discretization schemes of ODEs, though we used only a Radau collocation from [39]. The tool is advantageous for our purpose, since it includes automatic differentiation, interfaces to MILP solvers, and its problem formulation stays close to mathematics. In addition, AMPL provides the dual variables $\mathbf{\lambda }$. Throughout the numerical study, we applied Gurobi 8.1 as the MILP solver and IPOPT 3.12.4 as the NLP solver, with default settings in each case, to solve the discretized (OCP). We assumed that the choice of the MILP solver has little influence on solution quality and verified this by testing also with CPLEX 12.9. We tested our algorithms also with CasADi and received similar results as with ampl_mintoc. All results were obtained on a workstation with four Intel i5-4210U CPUs (1.7 GHz) and 7.7 GB RAM.

We included MIOCPs from the benchmark collection site mintoc.de [18] in our numerical study, which we specify further in the following subsections. For these problems, we chose a differential states discretization with N intervals such that it was fine enough respect to the objective value. In this way, the objective value differs only to the fifth decimal place, with respect to a finer discretization for constant M. Afterwards, we varied M with fixed N in order to construct different instances. We refer to Appendix A for further details. Solving the binary approximation problem and then solving the MIOCP with fixed binary controls might result in infeasible solutions for problems involving path or terminal constraints. To this end, we relaxed these constraints and applied a merit function that penalizes constraint violation to be added to the objective with a sufficiently high penalty factor.

6.2. Scaled Combinatorial Integral Approximation

Our hypothesis is that the MILPs based on a scaled combinatorial integral approximation perform the best on instances where the binary control enters the control dependent right-hand side terms ${\mathit{f}}_i$ of the ODE in an affine way, i.e.,:

$$\dot{\mathbf{x}}\left(t\right)={\mathit{f}}_0(t,\mathbf{x}\left(t\right))+\sum _{i=1}^{n_{\omega }}{\omega }_i\left(t\right)c_i,fora.a.t\in \mathcal{T},$$

(25)

where $c_i\in \mathbb{R}$. On the other hand, if ${\mathit{f}}_i$ depends on $\mathbf{x}\left(t\right)$, it may change rapidly over time. To this end, this results in possibly inaccurate $\omega$ solutions, since we use only the discretized state trajectory $\mathbf{x}\left(t\right)$ value. The MIOCPs “Double tank (Multimode)” and “Lotka–Volterra (absolute fishing variant)” are identified as candidate problems with the above right-hand side structure and constructed their solutions for different discretizations and both with and without the combinatorial constraint (15); see Appendix A for details. The results are presented in Figure 3.

We chose to evaluate the MIOCP solutions according to the distance in the ${\parallel ·\parallel }_{\infty }$-norm of the differential state trajectories corresponding to either binary or relaxed controls. We argued in Section 4 that the CIA decomposition is built on this distance, and, particularly, the proximity of objective values and constraint satisfaction follows as derived in Section 4.5. The differential state trajectories based on the (SCIA1) and (SCIAmax) solutions are significantly closer to the relaxed solution compared with their CIA counterparts, as shown in the performance plot. There are hardly differences between ${\parallel ·\parallel }_1$- and ${\parallel ·\parallel }_{\infty }$-norm results, although a tendency can be detected of better-performing ${\parallel ·\parallel }_{\infty }$ variants.

We examined whether the (SCIA1) and (SCIAmax) are outperformed by (CIA1) or (CIAmax), if the state trajectory distance is measured in ${\parallel ·\parallel }_1$-norm, or if the objective value deviation to the relaxed solution is taken into account, but the SCIA variants remained the clear winners. Analogously, the result remains similar when comparing the algorithms solely on instances (not) including combinatorial constraints.

6.3. $\lambda$-Combinatorial Integral Approximation

We derived ($\lambda$CIA) as an approximation of the cost-to-go function difference to the relaxed solution. Since this approximation is linear, the standard (CIA) approach is more suitable on most of the (nonlinear) MIOCPs. Our hypothesis is that the situation is different when a regularization term enters the objective function, accounting for the cost of activating binary controls in the form of, e.g.,:

$$\mathsf{\Phi }\left(\mathbf{x}\left(t_f\right)\right)+\underset{t\in \mathcal{T}}{\int }\sum _{i=1}^{n_{\omega }}{\omega }_i\left(\tau \right)c_i\mathrm{d}\tau ,$$

where $c_i\in \mathbb{R}$. The problem “Quadrotor (binary variant)” includes a cost function, where the controls enter in the above form, so that we used it with different discretizations and both with and without the combinatorial constraint (15) for comparing the ($\lambda$CIA) solutions with the ones obtained via (CIA). We present the computation results in Figure 4.

In contrast to the previous section, we compared here the objective deviations from the relaxed solution in percentages, since the $\lambda$-combinatorial integral approximation aims directly at improving the objective values. However, we remark that the latter algorithm performs worse than (CIA1) and (CIAmax) if the distance to the relaxed solution is measured in differential state space. The performance plot shows that ($\lambda$CIA) provides solutions with improved objective values on some instances, but on many others it does not. Since ($\lambda$CIA) turned out to provide even weaker approximations of the relaxed solutions for other MIOCPs, as will be shown in Section 6.5, we do not recommend to use it in general as a single MILP approximation step. It serves, however, as beneficial candidate solution for recombination and might be useful for not-yet-explored problem classes.

6.4. Backwards Accumulating Constraints

Our hypothesis for the MILP variants based on backwards accumulated constraints is that they are beneficial if the MIOCP involves terminal equality constraints on the differential states. Because the deviation to the relaxed solution can become large, the standard (CIAmax) approach may construct a solution that does not satisfy this constraint. Nevertheless, the direct incorporation of terminal constraints into the MIOCP may lead to numerical difficulties already for the relaxed problem. To this end, we considered soft constraints, meaning that we introduce slack variables in order to penalize a deviation of the differential states from a desired terminal value. We identified the MIOCP “Lotka–Volterra (terminal constraint violation)” as a candidate problem and calculated its solutions for different discretizations and both with and without the combinatorial constraint (15). We illustrate the objective deviation from the relaxed solution in percentage of (CIA) and its backward variant solutions in Figure 5.

We chose the objective deviation as the performance measure for our comparison study because the objective accounts for a violation of the terminal constraints via a slack variable penalty term. The graphs of (CIAmaxB) and (CIA1B) indicate that their respective MIOCP solutions involve smaller objective values than their forward CIA counterparts. We observed that this result seems to be independent from the chosen norm, since the performance differences of (CIAmaxB) to (CIA1B) are neglectable.

6.5. Recombination Heuristics

We used the MILP solutions by (CIAmax), (CIA1), (SCIAmax), ($\lambda$CIA1), and (CIAmaxB) as a base for running the recombination heuristics on a set of 13 MIOCPs from the benchmark collection site mintoc.de with different discretizations (see Appendix A for details). The box plot in Figure 6 illustrates the numerical results with respect to objective deviation of each algorithm to the relaxed solution in percentages.

The boxes including median values of the SCIA and backwards approaches appear to be slightly larger and their mean values and their outliers a bit smaller, respectively, than the ones of the (CIA) MILPs. The numerical study revealed several instances where (SCIAmax) or (SCIA1) ran into a binary solution with active controls on some intervals with relaxed values close to zero. Under the assumption that the combinatorial approximation is conducted mainly on singular arcs, these cases might be called degenerated. We experienced underperforming objective values for SCIA in case of degenerated controls, which explains some of the underperforming instances. We conclude that (SCIA1), (SCIAmax), and (CIAmaxB) should be used with caution. For specific problem classes, as shown in the previous chapters, they can be very helpful. Here, we have not specifically selected the problems, and on this general problem class there is no guarantee that these algorithms provide any real improvement.

The solutions of ($\lambda$CIA1) clearly underperform, but we stress, as mentioned in Section 6.3, their importance for recombination. As a comparative calculation, we have also computed the solutions based on SUR and see that they provide similarly good, albeit somewhat worse, solutions compared with (CIA1) and (CIAmax). Note that depending on the selected algorithm, some instances resulted in a deviation of more than 100%, which can be explained by highly penalized infeasible solutions of path- or terminal-constrained problems.

The depicted recombination heuristics provide significantly better solutions in terms of the objective than the MILP-constructed binary solutions. The median values are reduced by a factor of about 2 (ArcRecombination) to a factor of 3 (GreedyTime) in comparison with (CIA). The other characteristics such as mean values, box borders, and outliers also reflect the improvements. Particularly noteworthy is the GreedyTime heuristic, which is robust against outliers as well as constructs on average solutions with small objective values. The ArcRecombination selects the solution of the MILP algorithms with the smallest objective value in the case of only one singular arc. Since many of the selected problems have only one singular arc, the box plot illustrates that this minimum over all MILPs can already provide a significant improvement.

6.6. Runtime Evaluation

Figure 7 shows exemplarily the relationship between runtime and objective function values for the Lotka–Volterra multimode problem with N = 12,000 and varying M. We compare (CIAmax) values both with GreedyTime and the solutions obtained by the MINLP solver Bonmin 1.8.6. For a fair comparison, we run Bonmin with its four different main algorithms, B-BB, B-OA, B-QG, and B-Hyb, and depicted the shortest runtime of these algorithms. Elapsed real time from AMPL represents runtime in our computations, since CPU time appeared to be very similar for our Bonmin calculations, and Gurobi, on the other hand, is known to be a multi-threaded solver. First of all, the illustration shows that the objective spread to the relaxed solution vanished with increasing M—regardless of the selected approach. Second, CIA was, for some instances, already quite close to Bonmin ($M=25,50$) in terms of objective quality, so GreedyTime cannot improve much. For other discretizations with a considerable gap between CIA and Bonmin solution, GreedyTime could close most of this gap while being two orders of magnitude faster than Bonmin.

The average runtime over all instances for (CIAmax) was about a few seconds and increased slightly for (SCIAmax); see Appendix A.2. Gurobi needed on average more than one minute for the MILPs with 1-norm and thus considerably more time. For instances involving a fine discretization, the runtime increased enormously, so we set a time limit of 30 min.

We remark that the greedy heuristics and the ArcRecombination are to be used cautiously, because an input of many MILPs leads to a high number of recombinations that have to be evaluated. ArcRecombination is relatively inexpensive and offers a solution that is at least as good as the best MILP. The algorithm is most beneficial in cases of several singular arcs, in contrast to most applied problem instances where there is only one arc. The greedy algorithm variants are quite expensive (run times of up to 15 min), yet provide solutions with objective function values very close to those of the relaxed problem.

A way to significantly reduce the computation time of the MILPs is to apply branch-and-bound [32] or to use SUR for constructing approximate solutions. These algorithms are implemented in the open-source software package pycombina (see https://github.com/adbuerger/pycombina) (accessed on 14 February 2022) [56] and might be adapted to the scaled MILP case as part of a future study. If the binary controls enter linearly into the dynamics, as in Equation (25), then the modification is straightforward, since only all differences $(a_i-w_i)$ have to be scaled with the factors $c_i$. Finally, run times of days, or even weeks when it comes to the MINLP solver, cast a positive light on the proposed decomposition algorithm including recombination.

7. Summary and Conclusions

We have extended the decomposition approach based on combinatorial integral approximation [32], using multiple MILP formulations and recombination heuristics in an outer loop. At the price of additional MILP solutions and MIOCP evaluations, we obtain an improvement of the objective function value for every fixed control discretization grid.

A numerical study with benchmark problems shows that the novel MILP solutions indeed improve the existing CIA solutions in terms of objective value on specific problem classes. We conclude that the CIA decomposition can be reasonably modified for certain MIOC subproblem classes. Furthermore, the computational results for a set of non-specific MIOCPs resulted in a substantial improvement of the CIA solution through recombination strategies.

The main added value of this study is a decomposition algorithm that works much faster than Bonmin, but still offers qualitatively similar solutions and performs on average better than the existing (CIA) approach. The framework is open to extensions, both on the MILP and on the post-processing level.

Additional work is necessary to incorporate other constraints, such as vanishing constraints, to derive further tailored MILP formulations for specific problem classes and to develop numerical algorithms that generalize SUR and/or branch-and-bound algorithms to the various MILP formulations.

Furthermore, it would be interesting to include and to compare the results of this study computationally with recent approaches for the CIA decomposition [43,44].