Optimal Darwinian Selection of Microorganisms with Internal Storage

Abstract

In this paper, we investigate the problem of species separation in minimal time. Droop model is considered to describe the evolution of two distinct populations of microorganisms that are in competition for the same resource in a photobioreactor. We focus on an optimal control problem (OCP) subject to a five-dimensional controlled system in which the control represents the dilution rate of the chemostat. The objective is to select the desired species in minimal-time and to synthesize an optimal feedback control. This is a very challenging issue, since we are are dealing with a ten-dimensional optimality system. We provide properties of optimal controls allowing the strain of interest to dominate the population. Our analysis is based on the Pontryagin Maximum Principle (PMP), along with a thorough study of singular arcs that is crucial in the synthesis of optimal controls. These theoretical results are also extensively illustrated and validated using a direct method in optimal control (via the Bocop software for numerically solving optimal control problems). The approach is illustrated with numerical examples with microalgae, reflecting the complexity of the optimal control structure and the richness of the dynamical behavior.

1. Introduction

The interaction between species coexisting in an ecosystem is complex and affected by external factors. Depending on their environment, some species will dominate, while others, less adapted, will progressively decline. This Darwinian pressure, when it can be manipulated [1], provides the opportunity of guiding the evolution of species of interest. This concept can be applied to artificial ecosystems to select individuals with a desired trait. Here, we focus on microalgae, unicellular photosynthetic microorganisms with promising potential for industrial applications [2,3]. The great biodiversity of microalgae opens the door for a large range of applications [4]. They are grown for their pigments, antioxidants or essential fatty acids [5], and, over the longer term, their efficient way of producing proteins, bricks for green chemistry, biofuel and CO${}_2$ mitigation [2,6,7]. To date, microalgae do not have the place they deserve in biotechnology (see, e.g., [8,9,10]) and many optimization steps must be carried out to improve the economic and environmental performances of these processes at a large scale [6,11]. Currently, only wild organisms sampled in nature are used on an industrial scale. One of the key challenges is to improve the productivity of these strains.

Species in agriculture have been improved after centuries of selection and hybridization. The objective of this work is to develop an alternative approach adapted to microorganisms to select, on a shorter time scale, more productive microalgae strains, by Darwinian pressure. The idea is based on the competitive exclusion principle in a continuous reactor [12] stating that the species which more efficiently uses the available resources will win the competition. The conditions for a bacterial or microalgal species to win the competition for a limiting substrate have been well established, and the outcome of the competition is known to depend on the minimum requested substrate to support a growth rate equal to the dilution rate [12,13].

Experimental works for maintaining a long-term selection process to favour individuals of interest have already been carried out [14,15]. Since the experiments can last several months or several years, these approaches are costly in time. There is a margin of improvement by applying optimal control theory [16] to enhance the selection process for N strains competing for the same resource (the control parameter being the dilution rate).

One main issue is to decrease the operating time when the species of interest starts to dominate. Several works addressed the question of improving the selection process in minimal time in the case of the chemostat system with Monod’s laws [17,18]. Microalgae are more complicated microorganisms better represented by the Droop model, taking into account the internal accumulation of the limiting nutrient [19,20]. Such a model for two strains in competition leads to a five-dimensional problem. The minimal time selection problem with this model is the main focus of the paper. It has been tackled in [21,22] after a simplification allowing for reducing the model dimension. This necessitated to oversimplify the initial dynamics which can play a role in the minimal time selection.

Optimal control [23] strategies ensuring the domination of the strain of interest are derived using the Pontryagin maximum principle [24]. Since the system is affine w.r.t. the control, we obtain various possible structures for an optimal control, namely, the concatenation of several bang arcs or of a bang arc with a singular arc of first order satisfying Legendre–Clebsch’s condition. The paper is structured as follows: in Section 2, we introduce the model and present the optimal control problem. We also prove the reachability of the target set. In Section 3, we make explicit the necessary conditions provided by the Pontryagin Maximum Principle and we introduce properties of the switching function. A thorough study of singular arcs is provided in Section 4 thanks to geometrical control theory. The paper is concluded with numerical simulations of optimal strategies using a direct method in Section 5.

2. The Optimal Control Problem (OCP)

2.1. Droop Model and Main Assumptions

We consider the Droop model [19]. This emblematic variable yield model represents the growth rate of microorganims which can intracellularly store nutrients. When two strains are in competition, it results in a five-dimensional system. The growth of each strain depends on the intracellular quota-storage (q-variable) of the limiting nutrient (s-variable). More precisely, when two species/strains, of biomass concentrations $x_1$ and $x_2$ are competing for one limiting nutrient s in the bioreactor, the Droop model reads as follows:

$$\left\{\begin{array}{cc}\dot{s}\hfill & =(s_{in}-s)D\left(t\right)-{\rho }_1\left(s\right)x_1-{\rho }_2\left(s\right)x_2,\hfill \\ {\dot{q}}_1\hfill & ={\rho }_1\left(s\right)-{\mu }_1\left(q_1\right)q_1,\hfill \\ {\dot{x}}_1\hfill & =\left[{\mu }_1\left(q_1\right)-D\left(t\right)\right]x_1,\hfill \\ {\dot{q}}_2\hfill & ={\rho }_2\left(s\right)-{\mu }_2\left(q_2\right)q_2,\hfill \\ {\dot{x}}_2\hfill & =\left[{\mu }_2\left(q_2\right)-D\left(t\right)\right]x_2,\hfill \end{array}\right.$$

(1)

where $q_i$ is the quota storage of the i-th species and $s_{in}$ is the input substrate concentration. The dilution rate $D(·)$ is a bounded non-negative control function such that $D\left(t\right)\in [0,D_{max}]$, where $D_{max}>0$ is the maximal admissible value of the dilution rate, above the maximum actual growth rates [22] (this will be made more precise in Section 2.3) of the two species as shown in Figure 1. In addition, for $i=1,2$, ${\rho }_i$ is a non-negative function representing the rate of substrate absorption, i.e., the uptake rate of the free nutrient s, and ${\mu }_i$ is also a non-negative function representing the growth rate of the i-th species (see [25]).

Following for instance [25], we suppose that the uptake rates ${\rho }_i\left(s\right)$ are expressed as,

$${\rho }_i\left(s\right)=\frac{{\rho }_{m_i}s}{K_i+s},$$

(2)

which corresponds to Michaelis–Menten’s kinetics. Here, the parameters $K_i$ and ${\rho }_{m_i}$ are positive, $i=1,2$. In addition, we assume that, for $i=1,2$, there is $k_i>0$ such that cell division does not occur if $k<k_i$. Concerning the Droop model, the kinetics ${\mu }_i$ are defined by

$$\left|\begin{array}{ccc}{\mu }_i\left(q_i\right)\hfill & =0,\hfill & 0\le q_i\le k_i,\hfill \\ {\mu }_i\left(q_i\right)\hfill & ={\mu }_{i\infty }\left(1-\frac{k_i}{q_i}\right),\hfill & k_i\le q_i.\hfill \end{array}\right.$$

(3)

In addition, for $i=1,2$, let $M_i$ stand for,

$$M_i:=\underset{s\in [0,s_{in}]}{sup}{\rho }_i\left(s\right)={\rho }_i\left(s_{in}\right)<{\rho }_{m_i},$$

and let ${\overline{q}}_1$,${\overline{q}}_2$ be such that ${\mu }_i\left({\overline{q}}_i\right){\overline{q}}_i=M_i$, $i=1,2$ (observe that ${\overline{q}}_1$, ${\overline{q}}_2$ are uniquely defined). Thus, one has,

$${\rho }_i\left(s_{in}\right)={\mu }_i\left({\overline{q}}_i\right){\overline{q}}_i.$$

System (1) satisfies the following invariance property.

Proposition 1.

For every $q_{m_1}\ge {\overline{q}}_1$, and for every $q_{m_2}\ge {\overline{q}}_2$, the set

$$\Omega :=(0,s_{in})\times [k_1,q_{m_1}]\times {\mathbb{R}}_{+}^{*}\times [k_2,q_{m_2}]\times {\mathbb{R}}_{+}^{*},$$

(4)

is forward invariant by (1).

Proof. 

First, observe that, for $i=1,2$, $x_i$ never vanishes whenever $x_i^0=x_i\left(0\right)>0$. Now, $(0,s_{in})$ is clearly invariant by the dynamics of $s(·)$ since $\dot{s}\ge 0$ (resp. $\dot{s}\le 0$) whenever $s=0$ (resp. $s=s_{in}$). Similarly, for $i=1,2$:

$$\begin{array}{ccc}{q}_i=k_i& \Rightarrow \hfill & {\dot{q}}_i={\rho }_i\left(s\right)>0,\hfill \\ q_i=q_{m_i}& \Rightarrow \hfill & {\dot{q}}_i\le M_i-{\mu }_i\left(q_{m_i}\right)q_{m_i}\le 0,\hfill \end{array}$$

where the last inequality follows from the choice of $M_i$ and the fact that $q_{m_i}\ge {\overline{q}}_i$, $i=1,2$. This ends the proof.    □

The parameter $q_{m_i}$ represents the maximum internal storage quota. Since $\Omega$ is invariant by (1) (Proposition 1), we notice that ${\overline{q}}_i$ stands for the effective maximum internal storage quota for $s\in (0,s_{in})$. Thus, in the sequel, we consider without loss of generality that $q_{m_i}={\overline{q}}_i$ for $i=1,2$. In the sequel, we also assume that ${\rho }_1$, ${\rho }_2$ fulfill the following hypothesis:

Assumption A1.

The affinity of species 1 for the substrate is higher than the one of species 2, i.e., $K_{s_2}>K_{s_1}$, or equivalently:

$$\frac{{\rho }_2^{″}}{{\rho }_2^{\prime }}>\frac{{\rho }_1^{″}}{{\rho }_1^{\prime }}.$$

(5)

From (2), we deduce that, for $s\ge 0$,

$${\rho }_2^{″}\left(s\right){\rho }_1^{\prime }\left(s\right)-{\rho }_1^{″}\left(s\right){\rho }_2^{\prime }\left(s\right)=\frac{2K_1{K}_2{\rho }_{m_1}{\rho }_{m_2}(K_2-K_1)}{{(K_1+s)}^3{(K_2+s)}^3}.$$

It means that species 1 with the lower $K_i$ absorbs nutrients slightly faster.

We are now in a position to formulate the OCP of interest.

2.2. Statement of the Optimal Control Problem (OCP)

In this work, we suppose that the first species (with biomass concentration $x_1$) is the one of interest. Our aim is to compute the best feeding strategy, that is, the optimal (dilution rate) control function $D(·)$, such that $x_1$ becomes predominant in the photobioreactor in minimal-time. This can be formulated and quantified in terms of the ratio between the two competing species. Intuitively, we wish to find an adequate control strategy (if possible optimal) $D(·)$ for which, at the end of the process, we have $\frac{x_1}{x_2}\gg 1$.

Firstly, the set of admissible controls is defined as,

$$\mathcal{D}:=\{D:[0,+\infty )\to [0,D_{max}];D(·)\in {\mathcal{L}}_{loc}^{\infty }\left({\mathbb{R}}_{+}\right)\},$$

where ${\mathcal{L}}_{loc}^{\infty }\left({\mathbb{R}}_{+}\right)$ is the space of locally integrable functions on every compact on ${\mathbb{R}}_{+}$ and $D_{max}>0$ is the maximum pump feeding capacity. In practice, $D_{max}$ is designed above the maximum growth rates of the coexisting species (see Section 2.3).

To handle the selection process between the two species, let us define a subset $\mathcal{T}$ of $\Omega$ as,

$$\mathcal{T}:=\{X:=(s,q_1,x_1,q_2,x_2)\in \Omega ;x_2\le \epsilon x_1\}.$$

We choose the parameter $\epsilon >0$ such that $\epsilon \ll 1$ in such a way to quantify the contamination rate of the interesting strain $x_1$. Whenever a trajectory reaches the target set $\mathcal{T}$, this means that the biomass of the first species is significantly greater than the other one when reaching the target $\mathcal{T}$ at the terminal time (if possible).

Objective 1.

The optimal control problem (OCP) can then be stated: determine a dilution-based control strategy $D(·)$ in such a way that trajectories of (1) starting from an initial condition within the set $\Omega$ reach the target set $\mathcal{T}$ in minimal-time, i.e.,

$$\underset{D\in \mathcal{D}}{inf}{t}_f^D\mathrm{s}.\mathrm{t}.X\left(t_f^D\right)\in \mathcal{T}\mathrm{and}{X}^0\in \Omega ,$$

(6)

where $X(·)$ is the unique solution of (1) associated with $D(·)\in \mathcal{D}$ such that $X\left(0\right)=X^0\in \Omega$ and $t_f^D\in (0,+\infty ]$ is the first entry time of $X(·)$ into the target set. In the sequel, we will use the simpler notation $t_f$ instead of $t_f^D$.

In other words, for every positive initial conditions $X^0=(s^0,q_1^0,x_1^0,q_2^0,x_2^0)$ such that $q_i^0>k_i$, we are seeking an admissible control strategy $D=D_{X_0}\in \mathcal{D}$, steering the trajectory $X\left(t\right)$ of the system (1) from $X^0$ to the target set $\mathcal{T}$ in minimal-time, for a fixed $D_{max}$ (Figure 1) and a given contamination rate $\epsilon \ll 1$. Note that, if one is able to synthesize such an optimal control for every $X^0\in \Omega$, then one is able to construct an optimal feedback control over $\Omega$ as $X_0\mapsto D_{X_0}\left(0\right)$. Such an optimal control problem falls into the class of minimal-time control problems governed by a mono-input affine controlled system, for which the synthesis of an optimal feedback control, thanks to geometric control theory, is a crucial (but also delicate) issue. In particular, handling the high dimension of the Droop model in competition and its resulting optimality system is challenging. Note also that the linearity of the problem w.r.t. D (in contrast, for instance, with strictly convex cost functionals) leads to technicalities because singular arcs usually occur in this setting, see Section 4.

2.3. Basic Properties

We now introduce the so-called actual growth rates. These key functions will have an important role in the optimal separation strategy. For that, let us firstly start with the following observations:

•

The mapping ${\rho }_i:[0,s_{in}]\to [0,M_i]$ is one-to-one with $M_i<{\rho }_{m_i}$;

•

For every $q_i\in [k_i,{\overline{q}}_i]$, one has $q_i{\mu }_i\left(q_i\right)\in [0,M_i]$ and $q_i\mapsto q_i{\mu }_i\left(q_i\right)$ is one-to-one from $[k_i,{\overline{q}}_i]$ into $[0,M_i]$;

•

It follows that the composition ${\delta }_i:[k_i,{\overline{q}}_i]\to [0,s_{in}]$ such that

$${\delta }_i\left(q_i\right):={\rho }_i^{-1}\left(q_i{\mu }_i\left(q_i\right)\right)=\frac{K_i{q}_i{\mu }_i\left(q_i\right)}{{\rho }_{m_i}-q_i{\mu }_i\left(q_i\right)},q_i\in [k_i,{\overline{q}}_i],$$

is well-defined and is also one-to-one;

•

Hence, the mapping ${\delta }_i^{-1}:[0,s_{in}]\to [k_i,{\overline{q}}_i]$ such that

$${\delta }_i^{-1}\left(s\right):=\frac{k_i{K}_i{\mu }_{i\infty }+({\rho }_{m_i}+{\mu }_{i\infty }{k}_i)s}{{\mu }_{i\infty }(K_i+s)},s\in [0,s_{in}],$$

is well-defined over $[0,s_{in})$ with values in $[k_i,{\overline{q}}_i]$ and is one-to-one.

From these observations, one can immediately check that the mappings ${\delta }_i$, $i=1,2$ and ${\delta }_i^{-1}$ are increasing.

Indeed, for $i=1$, one can write ${\delta }_1^{-1}\left(s\right)=c_1+\frac{c_2}{s+c_3}$ with $c_1,c_3>0$ and

$$c_2:={\delta }_i^{-1}\left(s\right)=C+K_{s_1}\left[\frac{k_{k_1}{\mu }_{1\infty }}{k_1{\mu }_{1\infty }+{\rho }_{m_1}}-1\right]<0,$$

which implies that ${\delta }_i^{-1}$ is increasing. The actual growth rate of species i is then defined as the mapping ${\mu }_i\circ {\delta }_i^{-1}$,

$${\mu }_i\left({\delta }_i^{-1}\left(s\right)\right)=\frac{{\rho }_{m_i}{\mu }_{i\infty }s}{k_i{K}_i{\mu }_{i\infty }+({\rho }_{m_i}+{\mu }_{i\infty }{k}_i)s},s\in [0,s_{in}],i=1,2.$$

The resulting generic functions are illustrated in Figure 1: Let us now define,

$$\Delta \left(s\right):={\mu }_1\left({\delta }_1^{-1}\left(s\right)\right)-{\mu }_2\left({\delta }_2^{-1}\left(s\right)\right),s\in [0,s_{in}].$$

(7)

Throughout the paper, we suppose that $\Delta$ satisfies the following assumption.

Assumption A2.

There is a unique $\widehat{s}\in (0,s_{in})$ such that $\Delta \left(s\right)>0$ for every $s\in (0,\widehat{s})$ and $\Delta \left(s\right)<0$ for every $s\in (\widehat{s},s_{in})$. In addition, Δ has a unique maximum $s_c\in [0,\widehat{s}]$.

Taking into account that $\Delta \left(\widehat{s}\right)=0$, the inequalities satisfied by $\Delta$ according to Assumption A2 can also be written:

$$\Delta \left(s\right)(s-\widehat{s})<0,s\in (0,s_{in})\setminus \left\{\widehat{s}\right\}.$$

If we assume that s is regulated to $s^{*}\left(t\right)=s_c$ using an appropriate control D, we notice that the q-variables are regulated to some unique $q_{ic}\in [k_i,q_{mi}]$, for $i=1,2$. The unique point $(s_c,q_{1c},q_{2c})$, where $s_c\in [0,s_{in}]$, plays a crucial role in the optimal control strategy of (OCP) as discussed in Section 5. Finally, $D_{max}$ (the maximum dilution rate) is assumed to be large enough in order to drive competition between the two species. More precisely, we assume that $D_{max}$ satisfies the hypothesis:

Assumption A3.

The maximal value of the dilution rate $D_{max}$ satisfies

$$\forall s\in [0,s_{in}],D_{max}>max\left({\mu }_1\left({\delta }_1^{-1}\left(s\right)\right),{\mu }_2\left({\delta }_2^{-1}\left(s\right)\right)\right).$$

These assumptions will ensure reachability (as detailed in the next section) of the target $\mathcal{T}$, and establishes a generic framework where both species may win the competition for sufficiently large time (considering for instance various constant control parameters D that favor species 1 or 2). Thus, under these considerations, we ensure the well-posedness of the optimal control problem of interest. In this general framework, the objective is then to determine the optimal D that steers the trajectories in minimal-time to the desired target.

2.4. Reachability of the Target

Our next aim is to show that the target is reachable from every initial condition. First, let us recall that the set

$$\mathcal{M}:=\{X\in \Omega ;x_1{q}_1+x_2{q}_2+s=s_{in}\}$$

(8)

is an invariant and attractive manifold for (1) for a given persistently exciting control (i.e., an admissible control function $D(·)$ such that ${\int }_0^{+\infty }D\left(t\right)dt=+\infty$).

Proposition 2.

For every initial condition $X^0\in \Omega$, there exists an admissible control $D(·)$ and a time $t_e\ge 0$ such that $X\left(t_e\right)\in \mathcal{T}$, where $X(·)$ is the unique solution of (1), starting from $X^0$, associated with $D(·)$.

Proof. 

Let $s^{†}\in (0,\widehat{s})$ and $X^0\in \Omega$. Without any loss of generality, we may assume that $s\left(0\right)=s^{†}$. Indeed, observe that $s=s^{†}$ is not a steady-state of $\dot{s}$ whenever $D=0$ over ${\mathbb{R}}_{+}$ or $D=D_{max}$ over ${\mathbb{R}}_{+}$. Thus, if we apply $D=0$ (in that case $\dot{s}<0$) or $D=D_{max}$ (in that case $\dot{s}>0$), then $s\left(t\right)=s^{†}$ is reached in a finite horizon. Consider the feedback control function,

$$D^{†}(x_1,x_2):=\frac{{\rho }_1\left(s^{†}\right)x_1+{\rho }_2\left(s^{†}\right)x_2}{s_{in}-s^{†}},$$

in such a way that the unique solution of (1) associated with this control satisfies $s\left(t\right)=s^{†}$ for every time $t\ge 0$ (Cauchy–Lipschitz’s Theorem). We claim that there exists $t_1\ge 0$ large enough such that $D^{†}(x_1\left(t\right),x_2\left(t\right))\in [0,D_{max}]$ for every time $t\ge t_1$. Indeed, when $t\to +\infty$, taking into account (8), it follows:

$$D^{†}(x_1\left(t\right),x_2\left(t\right))\sim \overline{D}\left(t\right):=\frac{{\rho }_1\left(s^{†}\right)x_1\left(t\right)+{\rho }_2\left(s^{†}\right)x_2\left(t\right)}{x_1\left(t\right)q_1\left(t\right)+x_2\left(t\right)q_2\left(t\right)}.$$

Now, when $t\to +\infty$, it is easy to see that $q_1\left(t\right)$ converges to $\frac{{\rho }_1\left(s^{†}\right)}{{\mu }_1^{\infty }}+k_1$, since $q_1$ satisfies the linear ODE $\dot{q_1}+{\mu }_1^{\infty }{q}_1={\rho }_1\left(s^{†}\right)+{\mu }_1^{\infty }{k}_1$). At steady-state, we thus have

$${\rho }_1\left(s^{†}\right)=q_1{\mu }_1\left(q_1\right).$$

In conclusion, when $t\to +\infty$,

$$\overline{D}\left(t\right)\sim \frac{q_1{\mu }_1(q_1\left(\left(t\right)\right)x_1\left(t\right)+q_2\left(t\right){\mu }_2\left(q_2\left(t\right)\right)x_2\left(t\right)}{x_1\left(t\right)q_1\left(t\right)+x_2\left(t\right)q_2\left(t\right)},$$

or, equivalently, using that, at steady state, $s^{†}={\rho }_1^{-1}\left(q_1{\mu }_1\left(q_1\right)\right)$ that is, $q_1={\delta }_i^{-1}\left(s^{†}\right)$, we end up with

$$\overline{D}\left(t\right)\sim \frac{{\mu }_1\left({\delta }_1^{-1}\left(s^{†}\right)\right)q_1\left(t\right)x_1\left(t\right)+{\mu }_2\left({\delta }_2^{-1}\left(s^{†}\right)\right)q_2\left(t\right)x_2\left(t\right)}{x_1\left(t\right)q_1\left(t\right)+x_2\left(t\right)q_2\left(t\right)}.$$

Thanks to Assumption A3, this last expression is upper bounded by $D_{max}$ for t large enough, which proves our claim. Finally, posit $y_i:=ln\left(x_i\right)$ and observe that

$${\dot{y}}_1-{\dot{y}}_2=\Delta \left(s^{†}\right)>0.$$

It follows that $y_1\left(t\right)-y_2\left(t\right)\to +\infty$ when $t\to +\infty$, which implies that ${lim}_{t\to +\infty }\frac{x_2\left(t\right)}{x_1\left(t\right)}=0$. This ends the proof.    □

2.5. Motivation of Studying the OCP

Thanks to Proposition 2, the target set is reachable from any initial condition, thus the existence of an optimal control of (6) is standard (namely because the dynamics is affine w.r.t. the control): it is an application of the Fillipov Theorem, see, e.g., [26,27,28]. Considering such a control as in the proof of Proposition 2 then indeed allows the system to let the species of interest dominate the reactor, but this process can be long (see, for instance, Example 1). Another possible strategy is to use a constant control D. Following [12], depending on the value of D, species 1 may win the competition, i.e.,

$$\underset{t\to +\infty }{lim}{x}_1\left(t\right)>0;\underset{t\to +\infty }{lim}{x}_2\left(t\right)=0.$$

In that case, this (simple) strategy indeed allows for reaching the target. However, this convergence is asymptotic and depends on the value of D as in the competitive exclusion principle. Roughly speaking, if $D>{\mu }_1\left({\tilde{q}}_1\right)$ (where ${\tilde{q}}_1$ and $s^{†}$ are such that ${\tilde{q}}_1:={\delta }_1^{-1}\left(s^{†}\right)$ and $({\mu }_1\circ {\delta }_1^{-1})\left(s^{†}\right)=({\mu }_2\circ {\delta }_2^{-1})\left(s^{†}\right)$), then species 2 wins the competition, whereas, if $D<{\mu }_1\left({\tilde{q}}_1\right)$, species 1 wins the competition. We refer to [12] for more details about the asymptotic behavior of (1) for a constant control D. Thus, the target set may not always be reachable with a constant control D. Our objective in this paper is precisely to propose a methodology to compute a control strategy to reach the target set $\mathcal{T}$ faster, playing on the control $D(·)$ as illustrated in Figure 2.

Example 1.

Let us consider the following parameters: ${\rho }_{1m}=0.8$, ${\rho }_{2m}=0.95$, $K_{s_1}=1$, $K_{s_2}=1.4$, $k_1=1.1$, $k_2=1.4$, ${\mu }_{1\infty }=1.8$, ${\mu }_{2\infty }=1.7$, $s_{in}=10$. The contamination rate is fixed to $\epsilon =0.05$. The initial conditions are given by: $s^0=2$, $q_i^0=2.5$ and $x_i^0=1$. As illustrated in Figure 2, the target $\mathcal{T}$ is reached after $t_f=46.774$ days using the control $D\left(t\right)=D_{opt}$, while $x_1$ dominates the culture after $t_f=62.68$ days using the constant control $D=0.48$. Let us also point out that, using an arbitrary constant control $D\in [0,D_{max}]$, we are not even sure that $x_1$ wins the competition (trajectories do not reach the target in that case).

3. Necessary Conditions on Optimal Controls

We start this section by generalizing previously obtained results [22,29] characterizing optimal solutions of (6). For that, we apply the PMP which allows for obtaining necessary conditions satisfied by optimal controls of (6). We denote by $X=(s,q_1,x_1,q_2,x_2)$ and $\lambda =({\lambda }_s,{\lambda }_{q_1},{\lambda }_{x_1},{\lambda }_{q_2},{\lambda }_{x_2})$, respectively, the state and adjoint variables (also called co-state or covector). The Hamiltonian associated with the optimal control problem

$$H=H(s,q_1,x_1,q_2,x_2,{\lambda }_s,{\lambda }_{q_1},{\lambda }_{x_1},{\lambda }_{q_2},{\lambda }_{x_2},{\lambda }^0,D),$$

is given by

$$\begin{array}{cc}\hfill H=& {\lambda }^0-\left({\rho }_1\left(s\right)x_1+{\rho }_2\left(s\right)x_2\right){\lambda }_s+({\rho }_1\left(s\right)-{\mu }_1\left(q_1\right)q_1){\lambda }_{q_1}+{\mu }_1\left(q_1\right)x_1{\lambda }_{x_1}\\ & +({\rho }_2\left(s\right)-{\mu }_2\left(q_2\right)q_2){\lambda }_{q_2}+{\mu }_2\left(q_2\right)x_2{\lambda }_{x_2}+D[\left(s_{in}-s\right){\lambda }_s-x_1{\lambda }_{x_1}-x_2{\lambda }_{x_2}].\end{array}$$

Let $X^0\in \Omega \backslash \mathcal{T}$ and let $\left(X\right(·),u(·\left)\right)$ be an optimal pair such that $X(·)$ reaches the set $\mathcal{T}$ in a time $t_f\ge 0$. Thanks to the PMP, there exist an absolutely-continuous map $\lambda :[0,t_f]\to {\mathbb{R}}^5$ and ${\lambda }^0\le 0$ such that:

• The pair $(\lambda (·),{\lambda }^0)$ is non-trivial, i.e., $(\lambda (·),{\lambda }^0)\ne (0,0)$.
• The covector satisfies

  $$\dot{\lambda }\left(t\right)=-{\nabla }_{X}H(X\left(t\right),\lambda \left(t\right),{\lambda }^0,D\left(t\right))\mathrm{a}.\mathrm{e}.t\in [0,t_f].$$

  (9)
• The Hamiltonian maximization condition writes

  $$D\left(t\right)\in \arg \underset{\xi \in [0,D_{max}]}{max}H(X\left(t\right),\lambda \left(t\right),{\lambda }^0,\xi )\mathrm{a}.\mathrm{e}.t\in [0,t_f].$$

  (10)
• At the terminal time, the transversality condition writes:

  $$\lambda \left(t_f\right)\in -N_{\mathcal{T}}\left(X\left(t_f\right)\right).$$

  (11)

Here, $N_{\mathcal{T}}\left(X\right)=\{p\in {\mathbb{R}}^5;\forall Y\in \mathcal{T},p·(Y-X)\le 0\}$ stands for the normal cone to $\mathcal{T}$ at some point $X\in \mathcal{T}$, see [28]. The adjoint Equation (9) is equivalent to:

$$\left\{\begin{array}{cc}{\dot{\lambda }}_s=& \left({\rho }_1^{\prime }\left(s\right)x_1+{\rho }_2^{\prime }\left(s\right)x_2\right){\lambda }_s-{\rho }_1^{\prime }\left(s\right){\lambda }_{q_1}-{\rho }_2^{\prime }\left(s\right){\lambda }_{q_2}+D{\lambda }_s,\hfill \\ {\dot{\lambda }}_{q_1}=& {\mu }_{1\infty }{\lambda }_{q_1}-{\mu }_1^{\prime }\left(q_1\right)x_1{\lambda }_{x_1},\hfill \\ {\dot{\lambda }}_{x_1}=& {\rho }_1\left(s\right){\lambda }_s-{\mu }_1\left(q_1\right){\lambda }_{x_1}+D{\lambda }_{x_1},\hfill \\ {\dot{\lambda }}_{q_2}=& {\mu }_{2\infty }{\lambda }_{q_2}-{\mu }_2^{\prime }\left(q_2\right)x_2{\lambda }_{x_2},\hfill \\ {\dot{\lambda }}_{x_2}=& {\rho }_2\left(s\right){\lambda }_s-{\mu }_2\left(q_2\right){\lambda }_{x_2}+D{\lambda }_{x_2}.\hfill \end{array}\right.$$

(12)

We define an extremal as a quadruplet $(X(·),\lambda (·),{\lambda }^0,D(·))$ such that $(\lambda (·),{\lambda }^0)$ is non zero and such that (1) and (9)–(11) is verified. Whenever ${\lambda }^0=0$, we say that the extremal is abnormal; if ${\lambda }^0\ne 0$, then we say that the extremal is normal. Because (6) is autonomous (i.e., the system does not depend explicitly on time), the Hamiltonian computed along an extremal is constant. In addition, since the terminal time is not fixed, we classically obtain, following optimal control theory, that $H=0$.

The transversality condition is crucial for obtaining properties on optimal controls by reasoning backward in time from the terminal time $t=t_f$. We shall next extend earlier results [22] by taking into account explicitly the fact that $\mathcal{T}$ is a half-space of ${\mathbb{R}}^5$ and exploiting that $X\left(t_f\right)$ belongs to the set $E:=\{X\in {\mathbb{R}}^5;x_2-\epsilon x_1=0\}$ (the boundary of the target set). Then, condition (11) can be transformed more explicitly as follows. At $X\left(t_f\right)$, the normal cone to $\mathcal{T}$ writes

$$N_{\mathcal{T}}\left(X\left(t_f\right)\right)={\mathbb{R}}_{+}(0,0,-\epsilon ,0,1).$$

Therefore, inclusion (11) is then equivalent to

$${\lambda }_s\left(t_f\right)={\lambda }_{q_1}\left(t_f\right)={\lambda }_{q_2}\left(t_f\right)=0,$$

(13)

together with

$${\lambda }_{x_1}\left(t_f\right)+\epsilon {\lambda }_{x_2}\left(t_f\right)=0,$$

(14)

and the inequalities ${\lambda }_{x_1}\left(t_f\right)\ge 0$ and ${\lambda }_{x_2}\left(t_f\right)\le 0$. Actually, one has ${\lambda }_{x_1}\left(t_f\right)>0$ and ${\lambda }_{x_2}\left(t_f\right)<0$. Suppose indeed that ${\lambda }_{x_1}\left(t_f\right)=0$. Then, (14) would imply ${\lambda }_{x_2}\left(t_f\right)=0$, and, since (9) is linear w.r.t. $\lambda$, we would obtain $\lambda \equiv 0$ over $[0,t_f]$. Using the constancy of H, we would also obtain ${\lambda }^0=0$ contradicting the PMP. We can then conclude that

$${\lambda }_{x_1}\left(t_f\right)>0\mathrm{and}{\lambda }_{x_2}\left(t_f\right)<0.$$

(15)

Throughout the paper, we suppose that only normal extremals occur, i.e., ${\lambda }^0<0$, and, without any loss of generality, we may assume that ${\lambda }^0=-1$ (up to a renormalization of the necessary conditions that are linear w.r.t. $\lambda$).

Remark 1.

Abnormal extremals are not generic. They correspond to the optimal path reaching the target set in some particular subset of the target set and are such that ${\lambda }^0=0$ (and thus $\lambda (·)\ne 0$). From the conservation of H, this implies that ${\mu }_1\left(q_1\left(t_f\right)\right)={\mu }_2\left(q_2\left(t_f\right)\right)$ in such a way that $\lambda \left(t_f\right)$ is not uniquely defined in contrast with the normal case (see below). At such a singular point, the value function (the minimal time as a function of $X^0$) is also non-differentiable.

Going back to the normal case, i.e., ${\lambda }^0=-1$, the covector $\lambda$ at $t=t_f$ can be completely determined (thanks to the conservation of H) as follows:

$$\begin{array}{cc}\hfill & {\lambda }_s\left(t_f\right)={\lambda }_{q_1}\left(t_f\right)={\lambda }_{q_2}\left(t_f\right)=0,\hfill \\ \hfill & {\lambda }_{x_1}\left(t_f\right)=\frac{1}{x_1\left(t_f\right)({\mu }_1\left(q_1\left(t_f\right)\right)-{\mu }_2\left(q_2\left(t_f\right)\right))}>0,\hfill \\ \hfill & {\lambda }_{x_2}\left(t_f\right)=-\frac{{\lambda }_{x_1}\left(t_f\right)}{\epsilon }<0.\hfill \end{array}$$

(16)

Notice that the quantity ${\mu }_1\left(q_1\left(t_f\right)\right)-{\mu }_2\left(q_2\left(t_f\right)\right))$ is non-zero along a normal extremal. As a consequence, the transversality condition (11) coupled with the conservation of H are equivalent to (16). The computation of $\lambda$ at $t=t_f$ is useful to integrate the state adjoint system backward in time from the target set.

We now wish to exploit the Hamiltonian condition (10). It is of particular interest to introduce the switching function, which allows us to determine the optimal control D according to the sign of the switching function. For that, let us denote by $\tilde{\Phi }$ the switching function:

$$\tilde{\Phi }:=-(x_1{\lambda }_{x_1}+x_2{\lambda }_{x_2})+(s_{in}-s){\lambda }_s,$$

associated with the control function $D(·)$.

$$\begin{array}{ccc}\tilde{\Phi }\left(t\right)>0& \Rightarrow \hfill & D\left(t\right)=D_{max},\hfill \\ \tilde{\Phi }\left(t\right)<0& \Rightarrow \hfill & D\left(t\right)=0,\hfill \end{array}$$

(17)

for a.e. $t\in [0,t_f]$. In the case where $\tilde{\Phi }>0$, resp. $\tilde{\Phi }<0$ over a time interval $[t_1,t_2]$, we say that the optimal control u is of bang type (denoted by $B_{+}$, resp. $B_{-}$). When the control $D(·)$ is non-constant in every neighborhood of a time $t_c\in (0,t_c)$, we say that $t_c$ is a switching time, and one must have $\tilde{\Phi }\left(t_c\right)=0$. Next, when the switching function $\tilde{\Phi }$ vanishes over a time-interval $[t_1,t_2]$, we state that a singular arc occurs. In this case, the corresponding trajectory is singular over $[t_1,t_2]$, and such an arc will be denoted by $\mathcal{S}$. Singular arcs are essential to optimize the time to steer an initial condition to the target set. Now, we are ready to state some main features of the switching function $\tilde{\Phi }$, and then investigate properties of the singular paths.

Lemma 1.

(i) 

The function $\tilde{\Phi }$ is continuously differentiable over $[0,t_f]$ and, moreover,

$$\dot{\tilde{\Phi }}=(s_{in}-s)({\rho }_1^{\prime }\left(s\right)[x_1{\lambda }_s-{\lambda }_{q_1}]+{\rho }_2^{\prime }\left(s\right)[x_2{\lambda }_s-{\lambda }_{q_2}]).$$

(18)

(ii) 

At the terminal time $t=t_f$, it holds:

$$\tilde{\Phi }\left(t_f\right)=\dot{\tilde{\Phi }}\left(t_f\right)=0.$$

(19)

Proof. 

By differentiating $\tilde{\Phi }$ w.r.t. t, we find that

$$\dot{\tilde{\Phi }}=-\dot{s}{\lambda }_s+(s_{in}-s){\dot{\lambda }}_s-[{\dot{x}}_1{\lambda }_{x_1}+{\dot{x}}_2{\lambda }_{x_2}].$$

Using (1) and (9), we obtain (18), which proves (i). For proving (ii), note that $x\left(t_f\right)\in E$ and $\lambda \left(t_f\right)\in E^{\perp }$, thus $\tilde{\Phi }\left(t_f\right)=0$. Using (13), we also obtain $\dot{\tilde{\Phi }}\left(t_f\right)=0$, which ends the proof.    □

Remark 2.

Following the formalism of geometric control theory, $\dot{\tilde{\Phi }}$ never involves D explicitly, but D is present in the expression of ${\tilde{\Phi }}^{\left(2k\right)}$, $k\ge 1$, see [27]. If $k\ge 1$ is the first integer for which the control is present in the expression of ${\tilde{\Phi }}^{\left(2k\right)}$, we usually say that the singular arc is of order k.

At this step, an optimal control is a concatenation of bang and singular arcs:

$${\mathcal{B}}_{\pm },{\mathcal{B}}_{\pm }{\mathcal{B}}_{\mp },{\mathcal{B}}_{\pm }\mathcal{S},{\mathcal{B}}_{\pm }{\mathcal{B}}_{\mp }\mathcal{S},{\mathcal{B}}_{\pm }{\mathcal{B}}_{\mp }{\mathcal{SB}}_{\pm },\dots$$

with possibly infinitely many crossing times (in particular, if there is a singular arc of order 2 [27]). The occurrence and properties of singular arcs as well as the various (possible) structure for an optimal control of (6) will be precisely the matter of the next section. The goal is to reduce (if possible) the number of possible structures for an optimal control.

4. Singular Arcs and Insights into Optimal Solutions

4.1. Legendre–Clsebsch’s Necessary Condition and Computation of the Singular Control

The analysis of singular arcs requires to compute $\ddot{\tilde{\Phi }}$. Indeed, the Hamiltonian condition does not give any information about an optimal control during a singular phase. Thanks to the next computations, we shall also be able to deduce an expression of the singular control during a singular arc.

Doing so, let us define ${\theta }_1,{\theta }_2:[0,T]\to \mathbb{R}$ as

$$\begin{array}{cc}\hfill {\theta }_1& :={\rho }_1^{\prime }\left(s\right)[x_1{\lambda }_s-{\lambda }_{q_1}]+{\rho }_2^{\prime }\left(s\right)[x_2{\lambda }_s-{\lambda }_{q_2}],\hfill \\ \hfill {\theta }_2& :={\rho }_1^{″}\left(s\right)[x_1{\lambda }_s-{\lambda }_{q_1}]+{\rho }_2^{″}\left(s\right)[x_2{\lambda }_s-{\lambda }_{q_2}].\hfill \end{array}$$

Hereafter, to simplify the layout, we do not write the dependency of ${\theta }_i$, s, $x_i$, ${\lambda }_s$, ${\lambda }_{q_i}$ w.r.t. the time. To shorten the notation, we also do not write explicitly the dependency of certain functions w.r.t. some variables. Using (12)–(18), one can write

$$\dot{\tilde{\Phi }}=(s_{in}-s){\theta }_1\mathrm{and}{\dot{\lambda }}_s={\theta }_1+D{\lambda }_s.$$

(20)

Lemma 2.

(i) 

The derivative of ${\theta }_1$ can be expressed as:

$$\begin{array}{cc}\hfill {\dot{\theta }}_1=& {\theta }_2\dot{s}+[{\rho }_1^{\prime }{x}_1+{\rho }_2^{\prime }{x}_2]{\theta }_1+{\lambda }_s[x_1{\mu }_1{\rho }_1^{\prime }+x_2{\mu }_2{\rho }_2^{\prime }]\hfill \\ \hfill & -{\rho }_1^{\prime }[{\mu }_{1\infty }{\lambda }_{q_1}-{\mu }_1^{\prime }\left(q_1\right)x_1{\lambda }_{x_1}]-{\rho }_2^{\prime }[{\mu }_{2\infty }{\lambda }_{q_2}-{\mu }_2^{\prime }\left(q_2\right)x_2{\lambda }_{x_2}].\hfill \end{array}$$

(21)

(ii) 

The second derivative of $\tilde{\Phi }$ fulfills the equality:

$$\begin{array}{cc}\hfill \ddot{\tilde{\Phi }}=& [-{\theta }_1+(s_{in}-s){\theta }_2]\dot{s}+(s_{in}-s)[{\rho }_1^{\prime }{x}_1+{\rho }_2^{\prime }{x}_2]{\theta }_1+(s_{in}-s){\lambda }_s[x_1{\mu }_1{\rho }_1^{\prime }+x_2{\mu }_2{\rho }_2^{\prime }]\hfill \\ \hfill & -(s_{in}-s){\rho }_1^{\prime }[{\mu }_{1\infty }{\lambda }_{q_1}-{\mu }_1^{\prime }\left(q_1\right)x_1{\lambda }_{x_1}]-(s_{in}-s){\rho }_2^{\prime }[{\mu }_{2\infty }{\lambda }_{q_2}-{\mu }_2^{\prime }\left(q_2\right)x_2{\lambda }_{x_2}].\hfill \end{array}$$

(22)

Proof. 

By differentiating ${\theta }_1$ w.r.t. t, we have

$${\dot{\theta }}_1={\theta }_2\dot{s}+{\rho }_1^{\prime }[{\dot{x}}_1{\lambda }_s+x_1{\dot{\lambda }}_s-{\dot{\lambda }}_{q_1}]+{\rho }_2^{\prime }[{\dot{x}}_2{\lambda }_s+x_2{\dot{\lambda }}_s-{\dot{\lambda }}_{q_2}].$$

Using (20) and (12), we obtain (21). Using that $\ddot{\tilde{\Phi }}=-\dot{s}{\theta }_1+(s_{in}-s){\dot{\theta }}_1$, we obtain (22).    □

Note that these computations have been verified thanks to a symbolic computation software. The next step is to establish whether Legendre–Clebsch’s condition is verified or not along a singular arc. Recall that this condition is necessary for optimality and that it can be stated as follows (see, e.g., [27,30,31]).

Theorem 1

(Legendre–Clebsch’s condition [27]). Let $I=[t_1,t_2]$ be such that the trajectory is singular over $[t_1,t_2]$. Then, one has:

$${\ddot{\tilde{\Phi }}}_{{|}_D}\ge 0,$$

(23)

which is fulfilled over $I=[t_1,t_2]$.

Using the expression of the derivative $\ddot{\tilde{\Phi }}$ given in (22), we provide in the next lemma the expression of the second derivative, ${\ddot{\tilde{\Phi }}}_{{|}_D}$.

Lemma 3.

Let $I=[t_1,t_2]$ be such that the trajectory is singular over $[t_1,t_2]$. Then, one has:

$${\ddot{\tilde{\Phi }}}_{{|}_D}={(s_{in}-s)}^2{\theta }_2=(x_1{\lambda }_s-{\lambda }_{q_1})\frac{({\rho }_1^{″}{\rho }_2^{\prime }-{\rho }_2^{″}{\rho }_1^{\prime })}{{\rho }_2^{\prime }}.$$

(24)

Proof. 

In (22), the only term involving the control D is related to $\dot{s}$. We obtain (24) using that ${\theta }_1\equiv 0$ over $[t_1,t_2]$.    □

Proposition 3.

Along a singular arc that occurs over a time interval $[t_c,t_f]$, it holds that:

$${\lambda }_s=0,$$

over $[t_c,t_f]$ and Legendre–Clebsch’s condition (with a strict inequality) is fulfilled over $[t_c,t_f)$.

Proof. 

Because the trajectory is singular over $[t_c,t_f]$, one has ${\theta }_1\equiv 0$ over this interval; thus, ${\lambda }_s(·)$ satisfies:

$${\dot{\lambda }}_s=D{\lambda }_s,{\lambda }_s\left(t_f\right)=0.$$

It follows that ${\lambda }_s\equiv 0$ over $[t_c,t_f]$. In a left neighborhood of $t=t_f$, one has from (12) ${\dot{\lambda }}_{q_1}<0$; thus, since ${\lambda }_{q_1}$ vanishes at $t=t_f$, we necessarily have ${\lambda }_{q_1}>0$ in a left neighborhood of $t_f$. Because ${\lambda }_s$ is zero over $[t_c,t_f]$, we deduce that $x_1{\lambda }_s-{\lambda }_{q_1}=-{\lambda }_{q_1}<0$ over $[t_c,t_f)$ (at $t=t_f$, ${\lambda }_{q_1}$ vanishes at $t_f$, as well as ${\ddot{\tilde{\Phi }}}_{{|}_D}$). Over $[t_c,t_f]$, we note that

$${\dot{\lambda }}_{x_1}={\lambda }_{x_1}(D-{\mu }_1\left(q_1\right)),$$

hence ${\lambda }_{x_1}$ does not vanish over $[t_c,t_f]$. Suppose that ${\lambda }_{q_1}$ vanishes over $[t_c,t_f]$ at a time $t^{\prime }\in [t_c,t_f]$. Then, one must have ${\dot{\lambda }}_{q_1}\left(t^{\prime }\right)\ge 0$ since ${\lambda }_{q_1}>0$ over $(t^{\prime },t_f)$. However, at $t=t^{\prime }$, the adjoint equation implies that

$${\dot{\lambda }}_{q_1}\left(t^{\prime }\right)=-{\mu }_1^{\prime }\left(q_1\left(t^{\prime }\right)\right)x_1\left(t^{\prime }\right){\lambda }_{x_1}\left(t^{\prime }\right)<0$$

because $q_1\left(t^{\prime }\right)>k_1$, ${\mu }_1^{\prime }>0$ over $(k_1,+\infty )$, $x_1>0$, and ${\lambda }_{x_1}>0$ over $[t_c,t_f]$. This is a contradiction and thus ${\lambda }_{q_1}$ does not vanish over $t_c,t_f$. Assumption A1 implies that ${\rho }_1^{″}{\rho }_2^{\prime }-{\rho }_2^{″}{\rho }_1^{\prime }<0$, thus ${\ddot{\tilde{\Phi }}}_{{|}_D}>0$ over $[t_c,t_f)$ as desired. We can then conclude that Legendre–Clebsch’s condition (with a strict inequality) is fulfilled over the whole interval $[t_c,t_f)$.    □

A consequence of the previous proposition is that, when a singular arc occurs over some time interval $[t_c,t_f]$, then it is of order 1. Based on this proposition, we shall only consider singular arcs of first order in the remaining of the paper. If Legendre–Clebsch’s condition holds true, the singular arc is said to be of turnpike type [26]. The expression defining the singular control can then be derived using (22). Next, let $\varsigma (X,\lambda )$ be defined by:

$$\begin{array}{cc}\hfill \varsigma (X,\lambda ):=& {\theta }_2[{\rho }_1{x}_1+{\rho }_2{x}_2]-{\lambda }_s(x_1{\mu }_1{\rho }_1^{\prime }+x_2{\mu }_2{\rho }_2^{\prime })-{\rho }_1^{\prime }[{\mu }_{1\infty }{\lambda }_{q_1}-{\mu }_1^{\prime }\left(q_1\right)x_1{\lambda }_{x_1}]\hfill \\ \hfill & -{\rho }_2^{\prime }[{\mu }_{2\infty }{\lambda }_{q_2}-{\mu }_2^{\prime }\left(q_2\right)x_2{\lambda }_{x_2}].\hfill \end{array}$$

(25)

We now give an expression of the singular control as a feedback of the state and covector.

Proposition 4.

Suppose that an extremal is singular over $[t_1,t_2]$ and that (23) is verified over $[t_1,t_2]$ with a strict inequality. Then, the singular control $D_s$ is given by

$$D_s(X,\lambda ):=\frac{\varsigma (X,\lambda )}{(s_{in}-s){\theta }_2},$$

(26)

where we recall that ${\theta }_2={\rho }_1^{″}\left(s\right)[x_1{\lambda }_s-{\lambda }_{q_1}]+{\rho }_2^{″}\left(s\right)[x_2{\lambda }_s-{\lambda }_{q_2}]$ and ς is given by (25).

Proof. 

This expression follows from (22) in which $\dot{s}$ is replaced by $(s_{in}-s)D-{\rho }_1{x}_1-{\rho }_2{x}_2$ and ${\theta }_1\equiv 0$ (since $\dot{\tilde{\Phi }}=\tilde{\Phi }=0$).    □

Corollary 1.

If the singular arc occurs over some time interval $[t_c,t_f)$, expression (26) simplifies into

$$D_s(X,\lambda ):=\frac{\tilde{\varsigma }(X,\lambda )}{(s_{in}-s){\theta }_2},$$

(27)

where $\tilde{\varsigma }$ is given by

$$\begin{array}{cc}\hfill \tilde{\varsigma }(X,\lambda ):=& {\theta }_2[{\rho }_1{x}_1+{\rho }_2{x}_2]-{\rho }_1^{\prime }[{\mu }_{1\infty }{\lambda }_{q_1}-{\mu }_1^{\prime }\left(q_1\right)x_1{\lambda }_{x_1}]-{\rho }_2^{\prime }[{\mu }_{2\infty }{\lambda }_{q_2}-{\mu }_2^{\prime }\left(q_2\right)x_2{\lambda }_{x_2}],\hfill \end{array}$$

and, in this case, ${\theta }_2$ simplifies also into ${\theta }_2=-{\rho }_1^{″}{\lambda }_{q_1}-{\rho }_2^{″}{\lambda }_{q_2}$ because ${\lambda }_s\equiv 0$.

Remark 3.

(i) 

Note that Legendre–Clebsch’s condition (with a strict inequality) is equivalent to ${\theta }_2>0$ (and that this condition is always verified over $[t_c,t_f)$ with $t_c$ close to $t_f$.

(ii) 

In view of the general expression giving the singular control, see (26), there is no guarantee a priori that the singular control $D_s$ is always with values in $[0,D_{max}]$, i.e., that the singular arc is always admissible (even if Legendre–Clebsch’s condition is verified). This can bring additional difficulties; however, we may discard this point by choosing $D_{max}$ large enough.

(iii) 

Notice that (27) is at least active at $t=t_f$ in the case where a singular arc $D_s$ steers the model trajectories towards the target $\mathcal{T}$, since the transversality conditions ensure that ${\lambda }_s\left(t_f\right)=0$.

4.2. About the Occurrence of a Terminal Singular Arc at the Terminal Time

The aim of this section is to discuss the possibility of having a singular arc over some time interval $[t_f-\tau ,t_f]$ (with $\tau >0$) and the structure of optimal controls. Our main questioning is as follows:

Does any optimal trajectory contain a singular arc over some time interval $[t_f-\tau ,t_f]$?

To analyze this point, let us summarize properties of the switching function at $t=t_f$ (that are consequences of transversality conditions associated with the codimension 1 target):

• The switching function and its derivative vanish at $t=t_f$:

  $$\dot{\tilde{\Phi }}\left(t_f\right)=\tilde{\Phi }\left(t_f\right)=0.$$

  (28)
• The second derivative of the switching function satisfies:

  $${\ddot{\tilde{\Phi }}}_{{|}_D}\left(t_f\right)=0.$$

  (29)
• In addition, Legendre–Clebsch’s condition (23) is always satisfied along a singular arc defined in a left neighborhood of the terminal time $t=t_f$.

The necessary conditions (28) and (29) are a very good indication for the occurrence of a singular arc and are thus strong arguments to answer positively to the above question. Thus, we could now wonder whether or not conditions (28) and (29) are sufficient to ensure the occurrence of a singular arc in some time interval $[t_f-\tau ,t_f]$. It appears that this question is complex and falls into the setting of geometric optimal control theory. As far as we know, such conditions are not equivalent to the occurrence of a singular arc over some time interval $[t_f-\tau ,t_f]$ (this may depend, in particular, on the initial condition). It is, however, worth mentioning that these conditions (in particular (28)) are commonly used numerically to implement a singular arc in shooting methods [32]. In our context of Droop model, it is very interesting to notice that singular arcs are the cornerstone of the optimal control, in particular for a large set of initial conditions that are biologically meaningful (typically for heterogeneous cultures where $x_1^0/x_2^0\approx 1$). However, the answer to the above question is not always true and depends on the initial condition (as it has been confirmed using direct optimization methods, see Section 5). Indeed, as illustrated in Example 2—Section 5, when the initial conditions $x_i^0$ are taken very close to the target ($x_2\left(t_f\right)/x_1\left(t_f\right)=\epsilon )$, and ${\mu }_1\left(q_1^0\right)-{\mu }_2\left(q_2^0\right)>0$, the singular arc does not appear or appears marginally at $t=t_f$ to satisfy the transversality conditions.

Recall that $\tilde{\Phi }\left(t_f\right)=\dot{\tilde{\Phi }}\left(t_f\right)=0$. Hence, the sign of $\tilde{\Phi }$ depends on $\ddot{\tilde{\Phi }}\left(t_f\right)$ that is computed in the next lemma.

Lemma 4.

At the terminal time, one has

$${\mu }_1\left(q_1\left(t_f\right)\right)-{\mu }_2\left(q_2\left(t_f\right)\right)>0.$$

(30)

In addition, the second derivative of the switching function exists at $t=t_f$ and:

$$\ddot{\tilde{\Phi }}\left(t_f\right)=\frac{[s_{in}-s\left(t_f\right)][{\rho }_2^{\prime }\left(s\left(t_f\right)\right){\mu }_2^{\prime }\left(q_2\left(t_f\right)\right)-{\rho }_1^{\prime }\left(s\left(t_f\right)\right){\mu }_1^{\prime }\left(q_1\left(t_f\right)\right)]}{{\mu }_2\left(q_2\left(t_f\right)\right)-{\mu }_1\left(q_1\left(t_f\right)\right)}.$$

(31)

Proof. 

Inequality (30) follows from the expression of $\lambda \left(t_f\right)$ and the transversality condition (16). The expression of $\ddot{\tilde{\Phi }}\left(t_f\right)$ in (31) follows from (22).    □

We can now define the function:

$$t\mapsto \xi \left(t\right):={\rho }_2^{\prime }\left(s\left(t\right)\right){\mu }_2^{\prime }\left(q_2\left(t\right)\right)-{\rho }_1^{\prime }\left(s\left(t\right)\right){\mu }_1^{\prime }\left(q_1\left(t\right)\right).$$

From the previous lemma, we deduce the behavior of an optimal path near the terminal time:

• First, the target set can only be reached at some point $X\left(t_f\right)$ such that (30) is fulfilled.
• In addition, if $\xi \left(t_f\right)\ne 0$, then, in a left neighborhood of $t=t_f$, the optimal control $D(·)$ is of bang type and satisfies

  $$D\left(t\right)=\mathrm{sign}\left(\xi \right(t\left)\right).$$
• If a singular arc occurs in a left neighborhood of $t=t_f$, then one must have $\xi \left(t_f\right)=0$, i.e., a singular arc reaches the target in the subset of $\mathcal{T}$ defined as:

  $$\begin{array}{cc}\hfill {\mathcal{T}}^{\prime }:=\{X=(s,q_1,x_1,q_2,x_2)\in \mathcal{T}& ;{\mu }_1\left(q_1\right)-{\mu }_2\left(q_2\right)>0\mathrm{and}\hfill \\ & {\rho }_2^{\prime }\left(s\right){\mu }_2^{\prime }\left(q_2\right)-{\rho }_1^{\prime }\left(s\right){\mu }_1^{\prime }\left(q_1\right)=0\}.\hfill \end{array}$$

4.3. Toward an Optimal Synthesis Characterizing the Optimal Solutions

Reducing the number of switching times is in general non-tractable for nonlinear optimal control problems governed by a system in dimension greater than three. Nevertheless, thanks to the properties of singular arcs, we obtained previously and of the switching function at $t=t_f$, we can expect a limited number of possible structures for an optimal control as we formulate in the next conjecture.

Conjecture 1.

Every initial condition in $\Omega$ is steered optimally to the target set via a control D that has a finite number of switching times. In addition, for almost every initial condition, an optimal control presents the following structure:

$$B_{\pm }{B}_{\mp }\mathcal{S}.$$

For a large set of initial conditions in some subset ${\Omega }^{†}\subset \Omega$ (far from the target), there is a single bang arc and a terminal singular arc, whereas, for some initial conditions close to the target set, no singular arc occurs (i.e., $\mathcal{S}$ is of zero duration).

This conjecture has been verified numerically for a large number of initial conditions (see Section 5). Our argumentation to confirm this conjecture is as follows.

• From the PMP, we have seen that, for every initial condition in $\Omega$, an optimal control is a concatenation of bang arcs ${\mathcal{B}}_{\pm }$ and singular arcs $\mathcal{S}$.
  Moreover, since the switching function $\tilde{\Phi }$ satisfies the strong requirements $\dot{\tilde{\Phi }}\left(t_f\right)=\tilde{\Phi }\left(t_f\right)=0$, ${\ddot{\tilde{\Phi }}}_{{|}_D}\left(t_f\right)=0$
  (from the transversality conditions) as well as Legendre–Clebsch’s condition, we conjecture that, for almost all initial conditions, an optimal control is singular in a left neighborhood of the terminal time. This implies in particular that the number of switchings is finite since we proved that any terminal singular arc is of first order. In addition, the number of switchings is minimal in general (apart when chattering occurs, see [27], which is not the case here). Thus, an optimal control should be of type $B_{\pm }{B}_{\mp }\mathcal{S}$ with one or two bang arcs before the terminal singular arc.
  As we have seen, we also must have $\xi \left(t_f\right)=0$, which only involves state variables at the terminal time. This surprising condition mixing the first derivative of the basic functions in the Droop model, with the s variable on one side and the $q_i$ variables on the other side, is, however, hard to interpret biologically.
• For some marginal—-but admissible—initial conditions outside of ${\Omega }^{†}$, the structure of an optimal control of (6) may be of bang type for almost all $t\in [0,t_f)$, or $[0,t_f-\tau ]$, with very small $\tau >0$ (see, e.g., Example 2 in the next section). This is the case when typically $x_1^0\gg x_2^0$, i.e., the initial condition is very close to the target set $\mathcal{T}$, with in addition ${\mu }_1\left(q_1^0\right)-{\mu }_2\left(q_2^0\right)>0$. Thus, the requirement ${\mu }_1\left(q_1\left(t_f\right)\right)-{\mu }_2\left(q_2\left(t_f\right)\right)>0$ is easily satisfied. Thus, in this particular situation, it comes as no surprise that the fastest path to reach the target $\mathcal{T}$ is the one exploiting the fact that ${\dot{x}}_1\left(0\right)\gg {\dot{x}}_2\left(0\right)$ (since $x_1^0\gg x_2^0$) along with $D\left(t\right)=0$, since it maximizes ${\dot{x}}_1\left(t\right)$ (we recall that ${\dot{x}}_i=({\mu }_i\left(q_i\right)-D\left(t\right))x_i$).

It is worth noticing that, when no singular arc occurs, the strategy mainly consists of “pushing” $x_1$ and $x_2$ as quickly as possible towards the target $\mathcal{T}$ using $D=0$, when the initial conditions $x_1^0$ and $x_2^0$ are very close to $\mathcal{T}$. Nevertheless, this strategy may not be the optimal one whenever $q_1^0$ and $q_2^0$ are “far” from satisfying ${\mu }_1\left(q_1\left(t_f\right)\right)-{\mu }_2\left(q_2\left(t_f\right)\right)>0$. This is typically the case illustrated in Example 3 in the next section.

For an optimal control of type $B_{\pm }\mathcal{S}$, the occurrence of a singular arc is related to the so-called turnpike phenomenon that we now explain in this framework. For a large subset of initial conditions ${\mathcal{S}}^{†}\subset \Omega$ that are biologically the most relevant, the structure of the optimal control is bang-singular$B_{\pm }\mathcal{S}$. The singular arc is the control $D_s$ given in (27) that reaches the target $\mathcal{T}$. Moreover, this singular phase coincides with optimal trajectories $(s\left(t\right),q_1\left(t\right),q_2\left(t\right))$ that stay most of the time close to the critical point $(s_c,q_{1c},q_{2c})$ defined in Section 2.3 (related to the actual growth rates and the function $\Delta \left(s\right)={\mu }_1\left({\delta }_1^{-1}\left(s\right)\right)-{\mu }_2\left({\delta }_2^{-1}\left(s\right)\right)$). Observe, for instance, the trajectories s, $q_1$ and $q_2$ in Figure 3a.We also believe that the concatenation of bang arcs before the major singular phase exclusively aims at moving $(s^0,q_1^0,q_2^0)$ towards $(s_c,q_{1c},q_{2c})$. Then, the singular arc $D_s$ takes over at a switching-time instant $t=t_s$ and ensures that the associated singular trajectory, denoted $(s_s\left(t\right),q_{1s}\left(t\right),q_{2s}\left(t\right))$, satisfies the so-called turnpike inequality (see, e.g., [33]),

$$\begin{array}{cc}\hfill & \parallel s_s\left(t\right)-s_c\parallel +\parallel q_{1s}\left(t\right)-q_{1c}\parallel +\parallel q_{2s}\left(t\right)-q_{2c}\parallel \le a_1\left(e^{-a_{2}t}+e^{a_2(t-t_f)}\right),\hfill \\ & a_1,a_2>0,\mathrm{for}\mathrm{all}t\in [t_s,t_f].\hfill \end{array}$$

(32)

This is, for instance, the case for optimal controls illustrated in Figure 4 (of type ${\mathcal{B}}_{-}\mathcal{S}$) and in Figure 5a (of type ${\mathcal{B}}_{-}{\mathcal{B}}_{+}\mathcal{S}$). The inequality (32) usually holds when the time interval $[0,t_f]$ is not excessively short [33,34,35], which is the generic case in the Droop model (1) associated with (6). Indeed, in practice, the most significant biological experiments aim to separate species and select $x_1$ starting from an homogeneous culture (a well-balanced initial culture with $x_1^0/x_2^0\approx 1$) or even from $x_2^0\gg x_1^0$ with the challenging issue of selecting the minority species ($x_1$), which is not naturally promoted. In these cases, Droop’s kinetics ensure that the minimum selection time $t_f$ cannot be excessively short and therefore singular arcs as well as the turnpike-type behavior appear systematically in the optimal strategy of (6).

5. Direct Optimization and Numerical Results

In this section, a direct-optimization approach is performed in order to solve (6) and illustrate the different cases discussed in Section 4.3. The numerical direct methods that we use throughout this paper are implemented in Bocop [36] (an optimal control toolbox), and they have the characteristic to transform the studied (6) into a nonlinear programming problem (NLP) in finite-dimension [37], through the discretization step of the control and the state variables [38]. Numerical results are organized as follows:

• An optimal control of type $B_{-}\mathcal{S}$ is developed throughout Example 1.
• An optimal control of type $B_{-}$ is developed throughout Example 2.
• An optimal control of type $B_{-}{B}_{+}\mathcal{S}$ is developed throughout Example 3.

In all the numerical examples, we consider the model parameters given in

Table 1. The model parameters used in Section 5.

	${\mathit{\rho }}_{{\mathit{m}}_{\mathit{i}}}$	${\mathit{K}}_{\mathit{i}}$	${\mathit{\mu }}_{\mathit{i}\mathit{\infty }}$	${\mathit{k}}_{\mathit{i}}$
Species/strain 1	7	0.3	1.7	1.75
Species/strain 2	8	0.6	1.8	1.80

, with the settings in

Table 2. Model and OCP settings. The contamination rate $\epsilon$ characterizes the target set $\mathcal{T}.$.

${\mathit{s}}_{\mathbf{in}}$	Control D	Contamination Rate $\mathit{\epsilon }$
6	$[0,D_{max}=1.5]$	0.08

.

Assumption 3 is verified when $D_{max}=1.5$, namely because $D_{max}$ is precisely chosen above the maximum actual growth rates of the species. The contamination rate in all the examples is fixed to a significantly small value, $\epsilon =0.08$.

In Bocop, the state variables (and even the time, in minimal-time OCPs) of the Droop model (1) are discretized with a Lobatto scheme based on Runge–Kutta methods of type Lobatto-IIIC of order 6, which uses an implicit trapezoidal rule. The main settings used in Bocop are given in

Table 3. Bocop settings used in Section 5.

Discretization Method	Lobatto IIIC (Implicit, 4-Stage, Order 6)
Time steps	130
NLP tolerance	<10${}^{-14}$

.

Example 2.

In the first example, we consider the Droop model resulting from the parameters in Table 1 and Figure 7, associated with the settings in Table 2 and Table 3, and the initial conditions given in

Table 4. The initial conditions used in Example 2.

${\mathit{s}}^0$	${\mathit{q}}_1^0$	${\mathit{x}}_1^0$	${\mathit{q}}_2^0$	${\mathit{x}}_2^0$
4	1.9	0.3	1.9	0.3

.

In this example, we have a well-balanced initial culture since $x_1^0/x_2^0=1$ (see Section 4.3).

The direct optimization method allows us to determine the optimal control D, given in Figure 4, that steers the model trajectories towards $\mathcal{T}$ (with $\epsilon =0.08$) in minimal-time $t_f=18.3526$ days.

We check and analyze the evolution of the switching function $\tilde{\Phi }$, its derivatives, and the co-state of the substrate s (Figure 8) in order to characterize the switching instant $t_s\in (0,t_f)$. This time-instant coincides with ${\lambda }_s=0$ (since the singular arc is the one reaching the target $\mathcal{T}$), $\tilde{\Phi }=\dot{\tilde{\Phi }}=0$ (thus activating $D_s$, according to the PMP). We also notice that the condition $\ddot{\tilde{\Phi }}\left(t_f\right)=0$ is also satisfied. The optimal state and co-state trajectories are depicted in Figure 3, where we notice that $s\left(t\right)$, $q_1\left(t\right)$ and $q_2\left(t\right)$ evolve around the static critical point $(s_c,q_{1c},q_{2c})$ for almost all $t\in [t_s,t_f]$, see the turnpike-like property discussed in Section 4.3.

The optimal control strategy aims to maximize the difference between the actual growth rates (the function $\Delta \left(s\right)$) as illustrated in Figure 9. The initial arc bang(0) drives $s^0$ towards $s_c$ (we recall that $\dot{s}=(s_{in}-s)D-{\rho }_1\left(s\right)x_1-{\rho }_2\left(x_2\right)$, $s^0=4$, $s_{in}=6$, and, $s_c=0.092$). The quantity ${\mu }_1\left(q_1\left(t\right)\right)-{\mu }_2\left(q_2\left(t\right)\right)$ is maximized, with a delayed-dynamics, as a consequence of maximizing $\Delta \left(s\right)$. At the final time of $t_f=18.3526$ days, we have ${\mu }_1\left(q_1\left(t_f\right)\right)-{\mu }_2\left(q_2\left(t_f\right)\right)>0$. Finally, we check in Figure 6 that, at $t=t_f$, we have

$$\frac{{\rho }_2^{\prime }\left(s\right)}{{\rho }_1^{\prime }\left(s\right)}\left(t_f\right)=\frac{{\mu }_1^{\prime }\left(q_1\right)}{{\mu }_2^{\prime }\left(q_2\right)}\left(t_f\right).$$

The behavior of the optimal control and optimal trajectories described in Example 2 is definitively the most compelling one (with bang(0)-singular or bang($D_{max}$)-singular arcs) from a biological standpoint, since it is the one that systematically appears when the final time is not extremely short. Indeed, in practice, initial conditions start more commonly sufficiently “far” from the target $\mathcal{T}$, leading to a final time that allows the singular arc and the turnpike-like behaviors to hold.

Example 3.

Now, let us consider the initial conditions in

Table 5. The initial conditions used in Example 3.

${\mathit{s}}^0$	${\mathit{q}}_1^0$	${\mathit{x}}_1^0$	${\mathit{q}}_2^0$	${\mathit{x}}_2^0$
1	2.5	1.2	2	0.1

.

It is worth noticing that the initial conditions in Example 3 are intuitively favourable for reaching the target $\mathcal{T}$ in a very short time, since ${\mu }_1\left(q_1^0\right)-{\mu }_2\left(q_2^0\right)>0$ and $x_2^0/x_1^0=0.083$, while $\epsilon =0.08$ (very close to the target). The optimal control in this case is mainly a bang(0) over time, as illustrated in Figure 10 (see Section 4.3 for more details). The optimal state trajectories and co-state trajectories (that satisfy the transversality conditions) are illustrated in Figure 5.

Example 4.

In the last example, let us consider the initial conditions in

Table 6. The initial conditions used in Example 4.

${\mathit{s}}^0$	${\mathit{q}}_1^0$	${\mathit{x}}_1^0$	${\mathit{q}}_2^0$	${\mathit{x}}_2^0$
5	1.8	2.2	5	0.18

.

In this situation, we notice that initial conditions are still favourable for reaching the target $\mathcal{T}$ in a very short time because $x_2^0/x_1^0=0.0818$ (with $\epsilon =0.08$, so $x_i^0$ are very close to the target). However, we also note that ${\mu }_1\left(q_1\left(0\right)\right)-{\mu }_2\left(q_2\left(0\right)\right)<0$, while ${\mu }_1\left(q_1\left(t_f\right)\right)-{\mu }_2\left(q_2\left(t_f\right)\right)$ should be positive at $t=t_f$ (see Section 3). Thus, the issue of minimal-time separation is slightly more complex than Example 3, and we obtain a structure for the optimal control of bang(0)-bang(1)-singular type, as illustrated in Figure 11. The corresponding model trajectories and optimal co-states are provided in Figure 12.

6. Conclusions

The minimal-time OCP for the competition between two microbial species accumulating nutrients is a key issue. Progressing along this problem will definitely help for experiments that nowadays last more than 6 months [14,15]. However, the competition described by the Droop model turns out to be significantly more complicated than for the Monod model in dimension 2 [17]. We have improved the preliminary results about this problem to be found in [22,29] in order to provide an optimal synthesis depending on the initial condition. In particular, we applied the PMP, we discussed the structure of the optimal control, and we identified the singular arc steering optimal paths to the desired target set in a minimal amount of time. This study also highlights the turnpike behavior [33] although an exact verification in our case is not possible in our setting since the problem is affine w.r.t. entries in contrast with [35] that, in general, requires coercive hypotheses on the Hamiltonian w.r.t. entries. As usual in optimal control problems that are affine in the control, the study of singular arcs via geometric methods is a crucial issue.

This study also raised a mathematical (open) question outside the scope of this paper on the existence or not of a terminal singular arc whenever the switching function, its derivative, and second derivative w.r.t. the input vanish on a terminal manifold of codimension 1 (see, e.g., [39]).

Future work will focus on the determination of closed loop (sub-optimal) controllers to be applied for bioreactor control subject to uncertainties that are inherently present in biological systems. Finally, the proposed strategy must now be tested experimentally to assess the gain in experimental time it can offer.