Sustainable Optimal Control for Switched Pollution-Control Problem with Random Duration

Abstract

Considering the uncertainty of game duration and periodic seasonal fluctuation, an n-player switched pollution-control differential game is modeled to investigate a sustainable and adaptive strategy for players. Based on the randomness of game duration, two scenarios are considered in this study. In the first case, the game duration is a random variable, $T_f$, described by the shifted exponential distribution. In the second case, we assumed that players’ equipment is heterogeneous, and the i-th player’s equipment failure time, $T_{fi}$, is described according to the shifted exponential distribution. The game continues until a player’s equipment breaks down. Thus, the game duration is defined as $T_f=min\{T_{f1},\dots ,T_{fn}\}$. To achieve the goal of sustainable development, an environmentally sustainable strategy and its corresponding condition are defined. By using Pontryagin’s maximum principle, a unique control solution is obtained in the form of a hybrid limit cycle, the state variable converges to a stable hybrid limit cycle, and the total payoff of all players increases and then converges. The results indicate that the environmentally sustainable strategy in the n-player pollution-control cooperative differential game with switches and random duration is a unique strategy that not only ensures profit growth but also considers environmental protection.

1. Introduction

Practical ecological, economic, and engineering problems comprise switching phenomena [1,2,3]. All systems involving logical decision-making and continuous (smooth) dynamics, such as robot systems [4], chemical control processes [5], etc., can be transformed into hybrid systems with multiple regimes of dynamics.

Therefore, the hybrid dynamic system has gained considerable a research interest in the environmental, economic, and engineering fields. In addition, changed systems, in the form of time-driven and state-driven switches, are increasingly common. The up-to-date contributions to those fields are [6,7,8,9,10]. In [9,11], an optimal solution in the form of a hybrid limit cycle (HLC) was introduced as the best possible candidate for the infinite-horizon optimization problem. However, the results were only about the optimal control of a single agent and did not explore the optimal control strategy of multiple players.

In addition, in contrast to the general pollution-control problems with deterministic terminal time [12,13] or with infinite horizon [10,14], the randomness of the game duration cannot be ignored, because the game may end abruptly. The reasons behind this can be an equipment break-down, an economic failure. or a natural disaster, among many others [15]. In [15,16], differential pollution-control games with random duration were thoroughly analyzed. However, the impact of seasonal fluctuation on the system was not considered.

Thus, by combining the two aforementioned research directions, an n-player cooperative differential game was explored for pollution control. The game involves infinite time-driven switches and encompasses random game duration.

The contributions of this paper are summarized as follows:

• A novel model is proposed to address challenges within the context of an n-player cooperative differential game for pollution control with time-driven switches and random duration. The time-driven switches within system are denoted by a periodic piecewise-constant function. Taking into account the randomness of game duration and the players’ equipment warranty period, the finite-horizon optimal control problem is reformulated as an infinite-horizon optimal control problem, in which the game duration is modeled considering two scenarios based on shifted exponential distributions. The proposed model introduces innovative concepts and refines previously established methodologies, aiming to enhance its adaptability to real-world scenarios and yield more practical outcomes.
• In addition, this study proves the sustainability and uniqueness of the environmentally sustainable solution upon the proposed model.
• Solving the optimal pollution-control problem with time-driven switches and random duration: We employed Pontryagin’s maximum principle and thoroughly analyzed the adjoint variable dynamics to derive the shifted equilibrium value, resulting in a unique environmentally sustainable solution in the form of an HLC, which is environmentally friendly and guarantees the profit of all players.

The obtained results can be applied to the optimization problems of switched systems with periodic switching signals, such as the formation control problem [17] and capital investment problem [8]. Additionally, they can provide valuable insights on how agents can effectively adapt to a rapidly changing and evolving environment, allowing them to reap the benefits.

The remainder of this paper is structured as follows. In Section 2, the model of the n-player pollution control differential game with time-driven switches and random duration is formulated. In addition, considering different types of game durations, two case studies were conducted. Section 3 presents a discussion of the cooperative game with identical shifted exponential distribution and provides a the definition of the environmently sustainable control and determines the equilibrium value of adjoint variable that is used to derive the unique solution. In Section 4, we discuss a cooperative game with different shifted exponential distributions and obtain the corresponding equilibrium value and unique solution. Section 5 provides an illustration of a pollution-control differential game involving two players, and optimal solutions for both scenarios are demonstrated numerically.

2. The Problem Statement

All notations used through the paper are summarized in Appendix A,

Table A1. Notations.

Symbols	Descriptions
$T_f$	the game duration
$T_{fi}$	the i-th player’s equipment failure time
T	the period time
$\tau$	the switching ratio
z	the stock of pollution within a fixed natural reservoir (e.g., a lake)
$v_i$	the emissions rate of player i
$b_i$	the maximal admissible emissions rate of player i
${\xi }_i$	the fraction of the emitted pollutants accumulated in the reservoir from each player (e.g., factory)
$\delta (t)$	the self-cleaning rate of the reservoir in time t
$C_i(z)$	the damage cost of player i
$R_i(v_i)$	the revenue functional of player i
$L(v_i,z)$	the instantaneous payoff of player i
$a_i$	the positive constant used to transform the player i’s emissions flow to the profit flow
$q_i$	the positive constant, corresponding to the tax that player i must bear
$J_i(z_0,v_1,v_2,\dots ,v_n)$	the integral payoff of player i
$J_{co}(z_0,v_1,v_2,\dots ,v_n)$	the joint payoff
$\theta$	the shift parameter of the exponential distribution from the initiation of game
$\lambda$	the parameter of distribution
${\theta }_i$	the shifted parameter of player i
${\lambda }_i$	the distribution parameter of player i

.

In this study, we consider the optimal lake-pollution-control game model based on [15,16,18]. The game involves n players (factories). Each player i manages his/her emissions policy toward the lake, such that the dynamic of the pollution stock is governed by the linear differential equation with the following initial condition:

$$\dot{z}=\sum _{i=1}^n{\xi }_i{v}_i-\delta (t)z,z(0)=z_0,$$

(1)

where z is the stock of pollution within a fixed natural reservoir (e.g., a lake), $v_i\in [0,b_i],i=\overline{1,n}$ denotes the emissions rate, $b_i$ is the maximal admissible emissions rate of each player, ${\xi }_i\in (0,1)$ is the fraction of the emitted pollutants accumulated in the reservoir from each player (e.g., factory), and $\delta (t)$ is the self-cleaning rate of the reservoir. Furthermore, we have $z_0\ge 0$ and the state $z(t)$ is non-negative for all $t\ge 0$.

The self-cleaning rate for lakes is widely acknowledged to vary throughout the year. This could be due to the impact of various factors, including temperature and light fluctuations, during specific periods (e.g., over the span of a year). Considering the impact of external seasonal changes on the lake, it is reasonable to postulate that the self-cleaning rate of the lake is not a fixed constant; however, it varies as a function of time. Thus, we further assumed that the self-cleaning rate of reservoir $\delta (t)$ is represented by a periodic piecewise-constant function, which defined as the following mathematical expression:

$$\delta (t):=\left\{\begin{array}{cc}{\delta }_1>0,\hfill & t\in [kT,(k+\tau )T],\hfill \\ {\delta }_2>0,\hfill & t\in [(k+\tau )T,(k+1)T].\hfill \end{array}\right.$$

(2)

The whole time duration $D=[0,T_f]$ is divided into equal periods of length T, each of which is subdivided into two parts: $[kT,(k+\tau )T]$ and $[(k+\tau )T,(k+1)T]$, where $\tau \in (0,1)$ is the switching ratio and $k\in {\mathbb{N}}_0$. When the system is in the first subperiod, $\delta (t)={\delta }_1$, whereas in the second subperiod, $\delta (t)={\delta }_2$ and ${\delta }_1\ne {\delta }_2$.

Note that the production is usually assumed to be linearly related to the emissionss. Therefore, the revenue function can be expressed in terms of emissionss [18]. The revenue function of each player $R_i(v_i)$ is strictly concave. The marginal revenue decreases with the increasing emissions rate of each player $v_i(t)\in [0,b_i]$. However, zero emissions (production) is unprofitable. Each player incurs a damage cost, $C_i(z)$, for mitigating their emissions at moment t, and this cost function is increasing and convex. Thus, a quadratic revenue functional $R_i(v_i)=a_i{v}_i(b_i-v_i/2)$ [19] and a linear cost functional form $C_i(z)=q_{i}z$ are commonly derived to represent the instantaneous payoff of player i as $L(v_i,z)=R_i(v_i)-C_i(z)$.

Then, the general form of the integral payoff for player i is as follows:

$$J_i(z_0,v_1,v_2,\dots ,v_n)={\int }_0^{T_f}[a_i{v}_i(b_i-v_i/2)-q_{i}z]dt,$$

(3)

where $a_i>0$ is a positive constant used to transform the emissions flow to the profit flow. Coefficient $q_i>0$ is a positive constant, corresponding to the tax that a player must bear (e.g., an ecotax).

Considering the randomness of the game duration, we assumed that the terminal time of game $T_f$ is a random variable. After simplifying the integral payoff [20], the expected integral payoff of player i is obtained as follows:

$$\begin{array}{cc}\hfill J_i(z_0,v_1,v_2,\dots ,v_n)& =E({\int }_0^{T_f}[a_i{v}_i(b_i-v_i/2)-q_{i}z]dt)\hfill \\ \hfill & ={\int }_0^{\infty }{\int }_0^s[a_i{v}_i(b_i-v_i/2)-q_{i}z]dtdF(s)\hfill \\ \hfill & ={\int }_0^{\infty }(1-F(t))[a_i{v}_i(b_i-v_i/2)-q_{i}z]dt,\hfill \end{array}$$

(4)

where $F(t)$ is the cumulative distribution function of $T_f$.

3. Cooperative Game with Identical Shifted Exponential Distribution

If all players share common pollution-control equipment (e.g., filters), we assume that the game duration is a realization of the random variable $T_f$ (the time of equipment failure is same for all players and coincides with the end of the game).

Assume that at the beginning of the game, all players use a new pollution-control equipment, and this equipment comes with a warranty period. The warranty period refers to a specified time frame after the sale of a product or service, during which the manufacturer or supplier assures free repairs or replacements. During this period, if the product experiences any quality issues or malfunctions, the consumer can avail free repairs/replacement services. Hence, before the warranty period, there is no risk of equipment breakdown, while after the warranty period, the equipment is subject to the risk of potential damage. The game terminates when the equipment breaks down.

Therefore, we consider a cooperative game for pollution control, wherein the randomness of game duration and consideration of the warranty period are represented by a shifted exponential distribution.

For the mathematical definition of random variable $T_f$, we applied a shifted exponential distribution, which is given by

$$F(t)=\left\{\begin{array}{cc}1-e^{\lambda (\theta -t)},\hfill & t>\theta ,\hfill \\ 0,\hfill & t\in [0,\theta ],\hfill \end{array}\right.$$

(5)

where $\theta >0$ is the shift parameter of the exponential distribution from the initiation of game 0, which represents the equipment warranty period. The equipment does not encounter the risk of failure before the warranty period, and after this period, $\theta$, the failure rate is constant over time. In addition, $\lambda >0$ is the parameter of distribution and $E(T_f)=\theta +\frac{1}{\lambda }$.

By substituting (5) into (4), the whole time horizon is split into $[0,+\infty ]$ as $[0,\theta ]\cup (\theta ,\infty ]$, according to Bellman’s optimality principle, and the overall payoff functional, (4), can be rewritten as a sum:

$$\begin{array}{cc}\hfill J_i(z_0,v_1,v_2,\dots ,v_n)=& {\int }_0^{\theta }[a_i{v}_i(b_i-v_i/2)-q_{i}z]dt\hfill \\ \hfill & +{\int }_{\theta }^{\infty }{e}^{\lambda (\theta -t)}[a_i{v}_i(b_i-v_i/2)-q_{i}z]dt.\hfill \end{array}$$

(6)

All players (factories) act together to maximize their joint payoff:

$$\begin{array}{cc}\hfill J_{co}(z_0,v_1,v_2,\dots ,v_n)& =\sum _{i=1}^n{J}_i(z_0,v_1,v_2,\dots ,v_n)\hfill \\ \hfill & =J_{co1}(z_0,v_1,v_2,\dots ,v_n)+J_{co2}(z_{\theta },v_1,v_2,\dots ,v_n)\hfill \\ \hfill & =\sum _{i=1}^n{\int }_0^{\theta }[a_i{v}_i(b_i-v_i/2)-q_{i}z]dt+\hfill \\ \hfill & \sum _{i=1}^n{\int }_{\theta }^{\infty }{e}^{\lambda (\theta -t)}[a_i{v}_i(b_i-v_i/2)-q_{i}z]dt.\hfill \end{array}$$

(7)

By using Pontryagin’s maximum principle, the cooperative solution is obtained as a result of the joint optimization problem. We solved this optimization problem separately. First, the second interval $(\theta ,\infty ]$ is considered with respect to players’ joint payoff $J_{co2}(z_{\theta },v_1,v_2,\dots ,v_n)={\sum }_{i=1}^n{\int }_{\theta }^{\infty }{e}^{\lambda (\theta -t)}[a_i{v}_i(b_i-v_i/2)-q_{i}z]dt$, and then the first interval $[0,\theta ]$ is considered with respect to players’ joint payoff $J_{co1}={\sum }_{i=1}^n{\int }_0^{\theta }[a_i{v}_i(b_i-v_i/2)-q_{i}z]dt$.

3.1. Second Interval—$(\theta ,\infty ]$

The payoff functional of the second interval $(\theta ,\infty ]$ is

$$J_{co2}(z_0,v_1,v_2,\dots ,v_n)=\sum _{i=1}^n{\int }_{\theta }^{\infty }{e}^{\lambda (\theta -t)}[a_i{v}_i(b_i-v_i/2)-q_{i}z]dt.$$

(8)

Further, the Hamiltonian is given by

$$H_{co2}(t)=e^{\lambda (\theta -t)}\sum _{i=1}^n[a_i{v}_i(b_i-v_i/2)-q_{i}z]+{\psi }_2(\sum _{i=1}^n{\xi }_i{v}_i-\delta (t)z).$$

(9)

Then, we have the canonical system:

$$\begin{array}{cc}\hfill & \dot{z}=\sum _{i=1}^n{\xi }_i{v}_i-\delta (t)z,\hfill \\ \hfill & {\dot{\psi }}_2=e^{\lambda (\theta -t)}\sum _{i=1}^n{q}_i+\delta (t){\psi }_2.\hfill \end{array}$$

(10)

Let ${\tilde{\psi }}_2=e^{-\lambda (\theta -t)}{\psi }_2$. Then, we have

$${\dot{\tilde{\psi }}}_2=\sum _{i=1}^n{q}_i+(\delta (t)+\lambda ){\tilde{\psi }}_2.$$

(11)

Based on the first order derivative of the Hamiltonian, the optimal control is obtained as follows:

$${\tilde{v}}_i^{*}=\left\{\begin{array}{cc}{b}_i,\hfill & {\tilde{\psi }}_2>0,\hfill \\ b_i+\frac{{\xi }_i}{a_i}{\tilde{\psi }}_2,\hfill & {\tilde{\psi }}_2\in [-\frac{a_i{b}_i}{{\xi }_i},0]\hfill \\ 0,\hfill & {\tilde{\psi }}_2<-\frac{a_i{b}_i}{{\xi }_i}.\hfill \end{array}\right.$$

(12)

For the payoff functional (8), which is defined in infinite horizon $(\theta ,\infty ]$ with infinite time-driven switches, players should consider a sustainable development. Therefore, this study aimed to find a sustainable decision-making pattern so that players can find an optimal compromise between profit and penalty (e.g., ecotax).

Definition 1. 

The optimal control, $v_i^{*}$, is environmentally sustainable if it does not take on boundary values, except at isolated instances of time, i.e., ${\tilde{\psi }}_2\in [-\frac{a_i{b}_i}{{\xi }_i},0]$, for all $t\ge \theta$.

This definition is based on the long-term economic interests of all players. For $v_i^{*}>b_i$, the control (the rate of emissions) of player i remains at its maximum value. Evidently, this is not profitable because player i bears high ecotax. In another situation $v_i^{*}<b_i$, the revenue of player i cannot exceed the cost; therefore, the player must halt the production after a certain time interval to allow the stock of pollution to decrease to a lower level. Hence, both of these situations are not acceptable for sustainable production [21].

The following theorem and its proof are close to the result presented in [11]. The main difference is that the condition for the adjoint variable was not set at the initial moment but at moment $\theta$. In addition, this value depends on the interval to which $\theta$ belongs.

Theorem 1. 

The solution to (10) satisfying $z(0)=z_0$ and ${\psi }_2(\theta )={\psi }_{hlc}$ with

$${\psi }_{hlc}=\left\{\begin{array}{cc}L(1-e^{p_{2}T(\tau -1)})e^{p_{1}m}-\frac{q}{p_1},\hfill & m\in [0,\tau T],\hfill \\ L(1-e^{p_1\tau T})e^{p_2(m-T)}-\frac{q}{p_2},\hfill & m\in [\tau T,T]\hfill \end{array}\right.$$

(13)

is the unique optimal solution to (1)–(7). Here, $q={\displaystyle \sum _{i=1}^n}{q}_i,p_1={\delta }_1+\lambda ,p_2={\delta }_2+\lambda ,m=\theta -\lfloor \frac{\theta }{T}\rfloor T,L=\frac{q(p_1-p_2)}{p_1{p}_2(e^{p_{2}T(\tau -1)}-e^{p_1\tau T})}.$

Proof. 

To obtain a periodic solution, the following equation is first solved:

$${\tilde{\psi }}_2(\theta )={\tilde{\psi }}_2(\theta +T).$$

(14)

Let $k_1=\lfloor \frac{\theta }{T}\rfloor$; then, $\theta \in [k_{1}T,(k_1+1)T]$. On interval $[k_{1}T,(k_1+1)T]$, the solution to (11) has the form

$${\tilde{\psi }}_2(t)=\left\{\begin{array}{cc}{c}_1{e}^{p_{1}t}-\frac{q}{p_1},\hfill & t\in [k_{1}T,k_{1}T+\tau T],\hfill \\ c_2{e}^{p_{2}t}-\frac{q}{p_2},\hfill & t\in [k_{1}T+\tau T,(k_1+1)T].\hfill \end{array}\right.$$

If $\theta \in [k_{1}T,k_{1}T+\tau T]$ or $m\in [0,\tau T]$, then ${\tilde{\psi }}_2(\theta )=c_1{e}^{p_1\theta }-\frac{q}{p_1}$, which is the same thing, solving (14), we have

$${\tilde{\psi }}_2(\theta )=L(1-e^{p_{2}T(\tau -1)})e^{p_{1}m}-\frac{q}{p_1}.$$

If $\theta \in [k_{1}T+\tau T,(k_1+1)T]$, or, which is the same thing, $m\in [\tau T,T]$, then ${\tilde{\psi }}_2(\theta )=c_2{e}^{p_1\theta }-\frac{q}{p_2}$. Solving (14) we have

$${\tilde{\psi }}_2(\theta )=L(1-e^{p_1\tau T})e^{p_2(m-T)}-\frac{q}{p_2}.$$

Then, the solution to (10) satisfying ${\psi }_2(\theta )={\tilde{\psi }}_2(\theta )={\psi }_{hlc}$ is given by

$$\begin{array}{c}\hfill {\tilde{\psi }}_2^{*}(t)=\left\{\begin{array}{c}\left({\psi }_{hlc}+\frac{q}{p_1}\right){\mathrm{e}}^{p_1(t-\theta )}-\frac{q}{p_1},t\in [\theta ,(k_1+\tau )T],\hfill \\ \left({\psi }_{hlc}+\frac{q}{p_1}\right){\mathrm{e}}^{p_1(t-kT-\theta )}-\frac{q}{p_1},t\in [(k_1+k)T,(k_1+k+\tau )T],k\in {\mathbb{N}}_{+},\hfill \\ \left(\left({\psi }_{hlc}+\frac{q}{p_1}\right){\mathrm{e}}^{p_1((k_1+\tau )T-\theta )}-\frac{q}{p_1}+\frac{q}{p_2}\right){\mathrm{e}}^{p_2(t-(k_1+\tau )T-kT)}-\frac{q}{p_2},\hfill \\ t\in [(k_1+\tau +k)T,(k_1+1+k)T],k\in {\mathbb{N}}_0,\hfill \end{array}\right.\end{array}$$

(15)

if $\theta \in [k_{1}T,k_{1}T+\tau T]$ and

$${\tilde{\psi }}_2^{*}(t)=\left\{\begin{array}{c}\left({\psi }_{hlc}+\frac{q}{p_2}\right){\mathrm{e}}^{p_2(t-\theta )}-\frac{q}{p_2},t\in [\theta ,(k_1+1)T],\hfill \\ \left({\psi }_{hlc}+\frac{q}{p_2}\right){\mathrm{e}}^{p_2(t-kT-\theta )}-\frac{q}{p_2},t\in [(k_1+k+\tau )T,(k_1+k+1)T],k\in {\mathbb{N}}_{+},\hfill \\ \left(\left({\psi }_{hlc}+\frac{q}{p_2}\right){\mathrm{e}}^{p_2((k_1+1)T-\theta )}-\frac{q}{p_2}+\frac{q}{p_1}\right){\mathrm{e}}^{p_1(t-(k_1+1)T-kT)}-\frac{q}{p_1},\hfill \\ t\in [(k_1+k)T,(k_1+\tau +k)T],k\in {\mathbb{N}}_0,\hfill \end{array}\right.$$

(16)

if $\theta \in [k_{1}T+\tau T,(k_1+1)T]$. As observed, ${\tilde{\psi }}_2^{*}(t)={\tilde{\psi }}_2^{*}(t+kT)$ and ${\tilde{\psi }}_2^{*}(t)\in [min(-\frac{q}{p_1},-\frac{q}{p_2}),max(-\frac{q}{p_1},-\frac{q}{p_2})]$; therefore, this solution is periodic and bounded.

The boundedness of ${\tilde{\psi }}_2^{*}(t)$ and the state $z(t)$ guarantees the fulfillment of the transversality condition

$$\underset{t\to +\infty }{lim}inf{e}^{\lambda (\theta -t)}{\tilde{\psi }}_2^{*}(t)(z(t)-z^{*}(t))\ge 0,$$

(17)

for any admissible solution $z(t)$.

Thus, by considering the concavity of the Hamiltonian, we can conclude that $v^{*}$ is the optimal control, where

$$v_i^{*}=\left\{\begin{array}{cc}{b}_i,\hfill & {\tilde{\psi }}_2^{*}(t)>0,\hfill \\ b_i+\frac{{\xi }_i}{a_i}{\tilde{\psi }}_2^{*}(t),\hfill & {\tilde{\psi }}_2^{*}(t)\in [-\frac{a_i{b}_i}{{\xi }_i},0],\hfill \\ 0,\hfill & {\tilde{\psi }}_2^{*}(t)<-\frac{a_i{b}_i}{{\xi }_i}.\hfill \end{array}\right.$$

(18)

The uniqueness of the obtained optimal solution follows from the concavity of the Hamiltonian. Note that the Hamiltonian is a concave function with respect to the state z and strictly concave with respect to the control $v_i$, thus following the uniqueness of the obtained optimal solution. This completes the proof of the theorem. □

Lemma 1. 

If ${\tilde{\psi }}_2(\theta )\ne {\psi }_{hlc}$, then $\underset{t\to +\infty }{lim}\parallel {\tilde{\psi }}_2(t)\parallel =\infty$.

Proof. 

When ${\tilde{\psi }}_2(\theta )$ deviates from the equilibrium initial value ${\psi }_{hlc}$, let ${\tilde{\psi }}_2(\theta )={\psi }_{hlc}+c$. Then, from the moment $\theta$, the difference of each period can be obtained:

$${\tilde{\psi }}_2(\theta +kT)-{\tilde{\psi }}_2(\theta +(k-1)T)=c{e}^{kT(p_2(1-\tau )+p_1\tau )}.$$

For $k\in {\mathbb{N}}_{+}$ and $kT(p_2(1-\tau )+p_1\tau )>0$, the value difference $c{e}^{kT(p_2(1-\tau )+p_1\tau )}$ varies monotonically. Thus, the value of ${\tilde{\psi }}_2$ diverges with time. This implies that when $t\to +\infty$, if $c>0$, ${\tilde{\psi }}_2(t)$ approaches $+\infty$, and if $c<0$, ${\tilde{\psi }}_2(t)$ approaches $-\infty$. □

Lemma 2. 

Optimal control $v_i^{*}$ is environmentally sustainable when the following inequalities are satisfied:

$$\left\{\begin{array}{cc}({\psi }_{hlc}+\frac{q}{p_1})e^{p_1((k_1+\tau )T-\theta )}-\frac{q}{p_1}\ge -\frac{a_i{b}_i}{{\xi }_i},\hfill & \theta \in [k_{1}T,(k_1+\tau )T],\hfill \\ (({\psi }_{hlc}+\frac{q}{p_2})e^{p_2((k_1+1)T-\theta )}-\frac{q}{p_2}+\frac{q}{p_1})e^{p_1(1-\tau )T}-\frac{q}{p_1}\ge -\frac{a_i{b}_i}{{\xi }_i},\hfill & \theta \in [(k_1+\tau )T,(k_1+1)T],\hfill \end{array}\right.$$

(19)

if ${\delta }_1>{\delta }_2$, $k_1=\lfloor \frac{\theta }{T}\rfloor$, and

$$\left\{\begin{array}{cc}(({\psi }_{hlc}+\frac{q}{p_1})e^{p_1((k_1+\tau )T-\theta )}-\frac{q}{p_1}+\frac{q}{p_2})e^{p_2(1-\tau )T}-\frac{q}{p_2}\ge -\frac{a_i{b}_i}{{\xi }_i},\hfill & \theta \in [k_{1}T,(k_1+\tau )T],\hfill \\ ({\psi }_{hlc}+\frac{q}{p_2})e^{p_2((k_1+1)T-\theta )}-\frac{q}{p_2}\ge -\frac{a_i{b}_i}{{\xi }_i},\hfill & \theta \in [(k_1+\tau )T,(k_1+1)T],\hfill \end{array}\right.$$

(20)

if ${\delta }_1<{\delta }_2$.

Proof. 

For ${\delta }_1>{\delta }_2$, the dynamic of adjoint variable ${\tilde{\psi }}_2^{*}(t)$ decreases in the first subperiod and increases in the second subperiod. In addition, according to Theorem 1, ${\tilde{\psi }}_2^{*}(t)\in [min(-\frac{q}{p_1},-\frac{q}{p_2}),max(-\frac{q}{p_1},-\frac{q}{p_2})]$. Then, we have ${\tilde{\psi }}_2^{*}(t)<max(-\frac{q}{p_1},-\frac{q}{p_2})<0$.

Hence, to ensure that optimal control $v_i^{*}$ is environmentally sustainable, the condition ${min}_{t\ge 0}{\tilde{\psi }}_2^{*}(t)\ge -\frac{a_i{b}_i}{{\xi }_i}$ must be satisfied. As such, we have

$$\left\{\begin{array}{cc}\underset{t\ge 0}{min}{\tilde{\psi }}_2^{*}(t)={\tilde{\psi }}_2^{*}((k_1+\tau )T)\ge -\frac{a_i{b}_i}{{\xi }_i},\hfill & \theta \in [k_{1}T,(k_1+\tau )T],\hfill \\ \underset{t\ge 0}{min}{\tilde{\psi }}_2^{*}(t)={\tilde{\psi }}_2^{*}((k_1+\tau +1)T)\ge -\frac{a_i{b}_i}{{\xi }_i},\hfill & \theta \in [(k_1+\tau )T,(k_1+1)T].\hfill \end{array}\right.$$

(21)

Similarly, the condition for ${\delta }_1<{\delta }_2$ can be obtained. □

Figure 1 illustrates the dynamics of ${\tilde{\psi }}_2$ with different initial values in two situations: when distribution shift parameter $\theta$ is located before and after the switching time in one period. Without loss of generality, period T and switching ratio $\tau$ can be assigned randomly because these two parameters do not affect the overall result. Hence, we denote $T=1;\tau =0.5$. In addition, for parameter $\lambda$, which directly determines the expectation of the game-termination time, we denote $\lambda =0.5$. The other parameters are set as ${\delta }_1=0.9,{\delta }_2=0.45$, and $q=6.$

The blue line, the initial value ${\tilde{\psi }}_2(\theta )$ which is equal to the equilibrium value ${\psi }_{hlc}$, denotes an equilibrium solution of the adjoint variable, which varies periodically with equal amplitude within interval $[{\psi }_{22}^{*},{\psi }_{21}^{*}]$, where ${\psi }_{2i}^{*}=-\frac{q}{p_i},i=1,2$ are equilibrium positions for each mode, depending on the change of $\delta$. The equilibrium positions are represented by the sky-blue dash lines, and the red lines indicate nonequilibrium solutions; their initial values slightly deviate from the equilibrium value. Thus, the application of a small deviation to the initial value can cause the solution to diverge over time, either going into $+\infty$ or $-\infty$. The solution can then escape from two equilibrium points $[{\psi }_{22}^{*},{\psi }_{21}^{*}]$.

In this way, an equilibrium solution is uniquely determined, forming an HLC as time approaches an infinite horizon with infinite time-driven switches.

3.2. First Interval—$[0,\theta ]$

The payoff functional of the first interval, $[0,\theta ]$, is denoted as

$$J_{co1}(z_0,v_1,v_2,\dots ,v_n)=\sum _{i=1}^n{\int }_0^{\theta }[a_i{v}_i(b_i-v_i/2)-q_{i}z]dt.$$

(22)

The Hamiltonian is given by

$$H_{co1}(t)=\sum _{i=1}^n[a_i{v}_i(b_i-v_i/2)-q_{i}z]+{\psi }_1(\sum _{i=1}^n{\xi }_i{v}_i-\delta (t)z)$$

(23)

Then, we have the canonical system:

$$\begin{array}{cc}\hfill & \dot{z}=\sum _{i=1}^n{\xi }_i{v}_i-\delta (t)z,\hfill \\ \hfill & {\dot{\psi }}_1=\sum _{i=1}^n{q}_i+\delta (t){\psi }_1,{\psi }_1(\theta )={\tilde{\psi }}_2(\theta )={\psi }_{hlc}.\hfill \end{array}$$

(24)

Moreover, the continuity condition ${\psi }_1(\theta )={\tilde{\psi }}_2(\theta )={\psi }_{hlc}$ is based on the continuity of the optimal control, which is directly driven by the adjoint variable, based on the fact that the switching instants depend on time, i.e., are autonomous or time-driven [9,22].

According to the first order derivative of the Hamiltonian, the optimal control is obtained as follows:

$$v_i^{*}=\left\{\begin{array}{cc}{b}_i,\hfill & {\psi }_1>0,\hfill \\ b_i+\frac{{\xi }_i}{a_i}{\psi }_1,\hfill & {\psi }_1\in [-\frac{a_i{b}_i}{{\xi }_i},0],\hfill \\ 0,\hfill & {\psi }_1<-\frac{a_i{b}_i}{{\xi }_i}.\hfill \end{array}\right.$$

(25)

As the terminal value of ${\psi }_1(t)$ is determined, we consider the backward time dynamics of ${\psi }_1(t)$, which is described by the following system:

$${\dot{\psi }}_1(t)=\left\{\begin{array}{cc}-\sum _{i=1}^n{q}_1-\delta (t){\psi }_1,\hfill & t\in [hT,(h+\tau )T),\hfill \\ -\sum _{i=1}^n{q}_2-\delta (t){\psi }_1,\hfill & t\in [(h+\tau )T),(h+1)T)],\hfill \end{array}\right.$$

(26)

where $h\in {\mathbb{N}}_0$, with ${\psi }_1(\theta )={\psi }_{hlc}$. Each subsystem of (26) has a single stable equilibrium at ${\psi }_{1i}^{*}=-\frac{q_i}{{\delta }_i},i=1,2$.

Note the equilibrium points of adjoint variables in the first interval are less than those in the second interval; thus, the overall trend of adjoint variable ${\psi }_1(t)$ in the first interval, $[0,\theta ]$, increases in the backward time.

Accordingly, back to the forward time point of view, the optimal control in the first interval, $[0,\theta ]$, may first retain around the maximal admissible value and then decreases to the HLC.

3.3. Numerical Overall Adjoint Variable

The initial value of adjoint variable ${\psi }_{hlc}$ in the second interval is uniquely determined, and this is used as the terminal value in the first interval. Now, we can solve the cooperative adjoint variable, denoted by ${\psi }_{co}(t)$, in the whole time horizon $[0,+\infty ]$. Herein, the parameter settings were similar to those in Section 3.1.

Figure 2 shows the situation when distribution shift parameter $\theta$ is located before the switching time in the second period, $T\le \theta \le T+\tau$, where

$${\psi }_{co}(t)=\left\{\begin{array}{cc}{\psi }_1(t),\hfill & t\in [0,\theta ],\hfill \\ {\tilde{\psi }}_2(t),\hfill & t\in [\theta ,\infty ],\hfill \end{array}\right.{\psi }_1(\theta )={\tilde{\psi }}_2(\theta )={\psi }_{hlc}.$$

(27)

Figure 3 shows the situation when distribution shift parameter $\theta$ is located after the switching time in the second period, $T+\tau \le \theta \le 2T$.

The blue lines in the figure represent for the cooperative adjoint variable, ${\psi }_{co}(t)$; the sky-blue dash lines represent the equilibrium points of two intervals, and the green lines show the distribution shift parameter $\theta$. From Figure 2 and Figure 3, we can conclude that regardless of whether shift instant $\theta$ is located before or after the switching time in a period, the overall trend of ${\psi }_{co}(t)$ does not change.

Consequently, the overall optimal control of player i in the whole time duration $[0,\infty ]$ is derived as:

$$v_i^{*}=\left\{\begin{array}{cc}{b}_i,\hfill & {\psi }_{co}>0,\hfill \\ b_i+\frac{{\xi }_i}{a_i}{\psi }_{co},\hfill & {\psi }_{co}\in [-\frac{a_i{b}_i}{{\xi }_i},0],\hfill \\ 0,\hfill & {\psi }_{co}<-\frac{a_i{b}_i}{{\xi }_i}.\hfill \end{array}\right.$$

(28)

4. Cooperative Game with Different Shifted Exponential Distributions

In reliability engineering, the pollution-control equipment used by each player is different and with a different warranty period. The duration of the warranty period may vary depending on factors such as product type, brand, and contract terms, etc., and it is usually measured in months or years. Hence, the i-th player’s equipment fails abruptly at moment $T_{fi}$ as a random variable with a known probability distribution function $F_i(t),i=\overline{1,n}$, and the equipment may break down owing to the end of its lifetime or other natural disasters. The game lasts until one of the players’ equipments breaks down. Hence, this study also considered an n-player cooperative game of pollution control with different shifted exponential distributions. Furthermore, each player is assumed to possess specific equipment used in pollution control. Moreover, ${\left\{T_{fi}\right\}}_1^n$ are assumed to be independent random variables. Thus, the game duration is defined as $T_f=min\{T_{f1},\dots ,T_{fn}\}$.

In this case, players’ equipment is heterogeneous, and ${\left\{T_{fi}\right\}}_1^n$ adopts different shifted exponential distributions, as well as different distribution parameters, ${\left\{{\lambda }_i\right\}}_1^n$. Without loss of generality, we assumed that ${\theta }_1\le {\theta }_2\dots \le {\theta }_n$, where ${\theta }_n$ is the shifted parameter of player n, and it represents the largest shifted parameter among all players. Then, we have $F(t)=P\{T_f<t\}=1-{\prod }_{i=1}^n(1-F_i(t))$, see [15], where $F_i(t)$ is defined as

$$F_i(t)=\left\{\begin{array}{cc}1-e^{{\lambda }_i({\theta }_i-t)},\hfill & t>{\theta }_i,\hfill \\ 0,\hfill & t\in [0,{\theta }_i].\hfill \end{array}\right.$$

(29)

The duration of the game is a random variable with composite distribution function [23]. Thus, the cumulative distribution function, $F(t)$, with different shifted exponential distributions is denoted as

$$F(t)=\left\{\begin{array}{cc}0,\hfill & t\in [0,{\theta }_1],\hfill \\ 1-e^{{\lambda }_1({\theta }_1-t)},\hfill & t\in [{\theta }_1,{\theta }_2],\hfill \\ \dots \hfill \\ 1-e^{{\sum }_{i=1}^j\left[{\lambda }_i({\theta }_i-t)\right]},\hfill & t\in [{\theta }_j,{\theta }_{j+1}],\hfill \\ \dots \hfill \\ 1-e^{{\sum }_{i=1}^n\left[{\lambda }_i({\theta }_i-t)\right]},\hfill & t>{\theta }_n.\hfill \end{array}\right.$$

(30)

The cooperative payoff functional, (4), in this case, can be rewritten as the following sum:

$$\begin{array}{cc}\hfill J_{co}(z_0,v_1,v_2,\dots ,& v_n)=J_1(z_0,v_1,v_2,\dots ,v_n)+J_2(z_0,v_1,v_2,\dots ,v_n)+\dots +J_n(z_0,v_1,v_2,\dots ,v_n)\hfill \\ & =\sum _{i=1}^n[{\int }_0^{{\theta }_1}[a_i{v}_i(b_i-v_i/2)-q_{i}z]dt+{\int }_{{\theta }_1}^{{\theta }_2}{e}^{{\lambda }_1({\theta }_1-t)}[a_i{v}_i(b_i-v_i/2)-q_{i}z]dt\hfill \\ & +\dots +{\int }_{{\theta }_n}^{\infty }{e}^{{\sum }_{i=1}^n\left[{\lambda }_i({\theta }_i-t)\right]}[a_i{v}_i(b_i-v_i/2)-q_{i}z]dt].\hfill \end{array}$$

(31)

4.1. The Last Interval—$({\theta }_n,\infty ]$

The payoff functional in the last interval, $({\theta }_n,\infty ]$, is

$$J_{c{o}_n}(z_0,v_1,v_2,\dots ,v_n)=\sum _{i=1}^n{\int }_{{\theta }_n}^{\infty }{e}^{{\sum }_{i=1}^n\left[{\lambda }_i({\theta }_i-t)\right]}[a_i{v}_i(b_i-v_i/2)-q_{i}z]dt.$$

(32)

The Hamiltonian is given by

$$H_{c{o}_n}(t)=e^{{\sum }_{i=1}^n\left[{\lambda }_i({\theta }_i-t)\right]}\sum _{i=1}^n[a_i{v}_i(b_i-v_i/2)-q_{i}z]+{\psi }_n(\sum _{i=1}^n{\xi }_i{v}_i-\delta (t)z).$$

(33)

Then, we have the following canonical system:

$$\begin{array}{cc}\hfill & \dot{z}=\sum _{i=1}^n{\xi }_i{v}_i-\delta (t)z,\hfill \\ \hfill & {\dot{\psi }}_n=e^{{\sum }_{i=1}^n\left[{\lambda }_i({\theta }_i-t)\right]}\sum _{i=1}^n{q}_i+\delta (t){\psi }_n.\hfill \end{array}$$

(34)

Let ${\tilde{\psi }}_n=e^{-{\sum }_{i=1}^n\left[{\lambda }_i({\theta }_i-t)\right]}{\psi }_n$; then, we have ${\dot{\tilde{\psi }}}_n={\sum }_{i=1}^n{q}_i+(\delta (t)+{\sum }_{i=1}^n{\lambda }_i){\tilde{\psi }}_n$.

As the differential equation of ${\tilde{\psi }}_n(t)$ is similar to the identical shift case, ${\tilde{\psi }}_2(t)$, in the Section 2, we can still uniquely determine the equilibrium initial value, ${\tilde{\psi }}_n({\theta }_n)={\psi }_{hl{c}_N}$, that forms a unique HLC.

$${\psi }_{hl{c}_N}=\left\{\begin{array}{cc}\overline{L}(1-e^{p_{2N}T(\tau -1)})e^{p_{1N}{m}_n}-\frac{q}{p_{1N}},\hfill & m_n\in [0,\tau T],\hfill \\ \overline{L}(1-e^{p_{1N}\tau T})e^{p_{2N}(m_n-T)}-\frac{q}{p_{2N}},\hfill & m_n\in [\tau T,T],\hfill \end{array}\right.$$

(35)

where $q={\sum }_{i=1}^n{q}_i, {\lambda }_N={\sum }_{i=1}^n{\lambda }_i, p_{1N}={\delta }_1+{\lambda }_N, p_{2N}={\delta }_2+{\lambda }_N, m_n={\theta }_n-\lfloor \frac{{\theta }_n}{T}\rfloor T,$ $\overline{L}=\frac{q(p_{1N}-p_{2N})}{p_{1N}{p}_{2N}(e^{p_{2N}T(\tau -1)}-e^{p_{1N}\tau T})}.$

4.2. Former Intervals—$[0,{\theta }_1]\cup ({\theta }_1,{\theta }_2]\cup \dots \cup ({\theta }_{n-1},{\theta }_n]$

Next, the equilibrium value of the adjoint variable at moment ${\theta }_n$ is uniquely determined. Thus, the dynamics of ${\psi }_i(t),i=\overline{1,n-1}$ are also uniquely determined in backward time.

The payoff functional of player j, $j\in [1,n-1],j\in {\mathbb{N}}_0$, is given by

$$J_{c{o}_j}(z_0,v_1,v_2,\dots ,v_n)=\sum _{i=1}^n{\int }_{{\theta }_j}^{{\theta }_{j+1}}{e}^{{\sum }_{i=1}^j\left[{\lambda }_i({\theta }_i-t)\right]}[a_i{v}_i(b_i-v_i/2)-q_{i}z]dt.$$

(36)

Furthermore, the Hamiltonian is denoted as

$$H_{c{o}_j}(t)=e^{{\sum }_{i=1}^j\left[{\lambda }_i({\theta }_i-t)\right]}\sum _{i=1}^n[a_i{v}_i(b_i-v_i/2)-q_{i}z]+{\psi }_j(\sum _{i=1}^n{\xi }_i{v}_i-\delta (t)z).$$

(37)

The differential equation of ${\psi }_j(t)$ is given as

$${\dot{\psi }}_j=e^{{\sum }_{i=1}^j\left[{\lambda }_i({\theta }_i-t)\right]}\sum _{i=1}^n{q}_i+\delta (t){\psi }_n.$$

(38)

Let ${\tilde{\psi }}_j=e^{-{\sum }_{i=1}^j\left[{\lambda }_i({\theta }_i-t)\right]}{\psi }_j$; then, we have

$${\dot{\tilde{\psi }}}_j=\sum _{i=1}^n{q}_i+(\delta (t)+\sum _{i=1}^j{\lambda }_i){\tilde{\psi }}_j,{\tilde{\psi }}_j({\theta }_{j+1})={\tilde{\psi }}_{j+1}({\theta }_{j+1}).$$

In addition, for the first interval $[0,{\theta }_1]$, we have ${\dot{\psi }}_1(t)={\sum }_{i=1}^n{q}_i+\delta (t){\psi }_1(t),$ ${\psi }_1({\theta }_1)=\widetilde{{\psi }_2}({\theta }_1).$

Consequently, the overall cooperative adjoint variable with different shifted distributions is obtained as follows:

$${\psi }_{co}(t)=\left\{\begin{array}{cc}{\psi }_1(t),\hfill & t\in [0,{\theta }_1],\hfill \\ {\tilde{\psi }}_2(t),\hfill & t\in [{\theta }_1,{\theta }_2],\hfill \\ \dots \hfill \\ {\tilde{\psi }}_j(t),\hfill & t\in [{\theta }_j,{\theta }_{j+1}],\hfill \\ \dots \hfill \\ {\tilde{\psi }}_n(t),\hfill & t>{\theta }_n.\hfill \end{array}\right.$$

(39)

5. Numerical Optimal Solution

For simplicity, we considered an example of a two-player cooperative game-theoretic model of pollution control. As the overall cooperative adjoint variable is uniquely obtained in each case, optimal solutions are shown below.

To meet the condition of environmentally sustainable control, the parameters settled were set as follows: ${\delta }_1=0.9;{\delta }_2=0.45;T=1; \tau =0.5;\lambda =0.5;q=6; {\xi }_1=0.8;$ ${\xi }_2=0.7;a_1=1; a_2=1.2;b_1=b_2=10; z_0=0; \theta =1.2;{\theta }_1=1; {\theta }_2=2.2; {\lambda }_1=0.3;$ and ${\lambda }_2=0.5$.

5.1. Solution of Identical Shifted Exponential Distribution

The optimal control, state trajectory, and corresponding cooperative payoff are demonstrated in Figure 4.

The solution shows that the optimal emissions of each player change periodically after the distribution shift instant $\theta =1.2$. Further, the stock of pollution converged to a unique HLC and stabilized, and the cooperative payoff increased and then converged.

In some cases, before time instant $\theta$, there may exist a period of radical emissions. This could be interpreted as a more intense use of equipment by players during the warranty period.

5.2. Solution of Different Shifted Exponential Distributions

The optimal control, state trajectory, and corresponding cooperative payoff in this case are shown in Figure 5.

Figure 5 shows that after the expiration of each piece of equipment’s warranty period (${\theta }_i$), the control strategy of player i tends to be increasingly conservative. After the maximal warranty period ${\theta }_2$, the control strategy of each player transforms to a periodic solution in the form of an HLC. The stock of pollution also converges to a unique HLC and stabilizes, while the cooperative payoff increases and then converges. This result indicates that the proposed control strategy can still maintain profitability when the equipment of each player is nonhomogeneous with seasonal fluctuation ($\delta (t)$).

For the dynamic switched system of pollution levels of a lake considered in this paper, based on the obtained results and coupled with the random duration of the process, we proposed that, within the framework of a cooperative game, players may adopt a production strategy transitioning from aggressive to conservative (gradually decreasing output) before the maximum warranty period ${\theta }_2$. This strategy reaches its equilibrium value at ${\theta }_2$. Moreover, after ${\theta }_2$, players adopt a production strategy following an HLC pattern. This involves the gradual increase and progressive decrease in production during periods of relatively high and low lake self-cleaning rates, respectively. Consequently, this approach ensures the attainment of an optimally controlled outcome from the context of sustainable development.

Therefore, based on the considerations of real-world issues, the obtained results exhibit uniqueness, theoretical applicability, and practical relevance.

6. Conclusions

In this study, we analyzed the cooperative differential game for a typical hybrid optimal pollution-control problem with two types of time-driven switches: the seasonal fluctuation(self-cleaning rate of the lake) and the shifted parameter of exponential distributions due to the random game duration. A random terminal duration problem was transformed into a combination of an infinite horizon and a finite horizon(s) optimal control problem.

Further, we first considered a scenario with identical game duration and then examined another scenario with joint probability distribution of game duration resulting from the heterogeneity of players’ equipment. This paper discussed these two scenarios in detail and presented the results of each scenario analytically and numerically. Furthermore, an environmentally sustainable solution in the form of an HLC was uniquely determined for each scenario, ensuring both sustainable production revenue and environmental protection.

In a subsequent study, we will delve into the optimal control problem of each player in a noncooperative game featuring infinite time-driven switches and random game duration. In addition, we will provide comparisons of the results obtained from the noncooperative game with those obtained from the cooperative game to establish a justifiable allocation rule.