Quarantine and Vaccination in Hierarchical Epidemic Model

Abstract

The analysis of global epidemics, such as SARS, MERS, and COVID-19, suggests a hierarchical structure of the epidemic process. The pandemic wave starts locally and accelerates through human-to-human interactions, eventually spreading globally after achieving an efficient and sustained transmission. In this paper, we propose a hierarchical model for the virus spread that divides the spreading process into three levels: a city, a region, and a country. We define the virus spread at each level using a modified susceptible–exposed–infected–recovery–dead (SEIRD) model, which assumes migration between levels. Our proposed controlled hierarchical epidemic model incorporates quarantine and vaccination as complementary optimal control strategies. We analyze the balance between the cost of the active virus spread and the implementation of appropriate quarantine measures. Furthermore, we differentiate the levels of the hierarchy by their contribution to the cost of controlling the epidemic. Finally, we present a series of numerical experiments to support the theoretical results obtained.

1. Introduction

Global epidemics provide a serious medical challenge worldwide. Millions of mortality cases each year are estimated to be directly related to infectious diseases [1]. Attempts to suppress the spread of infections are usually divided into pharmaceutical and non-pharmaceutical interventions: for example, several different vaccines against the novel coronavirus were developed quickly and were available within months of the onset of the epidemic; on the other hand, measures were taken to reduce the interregional and urban mobility of citizens, and the so-called lockdowns [2]. The relationship between these measures and the trade-off in choosing the appropriate response to an epidemic is the subject of much research on epidemic prevention [3,4,5].

The main tool for studying the effectiveness of various intervention policies is the mathematical modelling of the spreading processes [6,7,8,9]. Both types of interventions are reflected in the work of researchers—both pharmaceutical [10,11] and non-pharmaceutical [3,12,13,14] interventions. In our model, we combine these types of interventions, introducing the treatment as a controlled parameter and vaccination as an exogenous one, while allowing the population to enter a state of self-isolation or quarantine. Importantly, we divide the population into several related levels, the size and configuration of the links between which allow us to interpret them as hierarchical levels.

The idea of differentiating populations in epidemic models according to different characteristics is not novel. In the COVID-19 period, many predictive models were based on the classical SEIR, but were a tuple of such models, each relating to a different age group (see, for example [15,16,17,18,19]. On the other hand, so-called spatial models of the epidemic spread, which also distinguish populations on a territorial basis, became widespread: this gave rise first to the metapopulation or multi-group/multi-cluster epidemic models [20,21,22], and later to network models (an overview of such models is provided, for example, in [23]).

In the current research, we were motivated by the unprecedented outbreak of COVID-19 that began in one location in China and spread throughout the world [24]. We assumed the epidemic process could be divided into several hierarchical levels or clusters: a city, a region, and a country. Depending on the size of the population in each level/cluster, the pathogen’s spread rate will differ. These assumptions lead to a hierarchical model with three levels and the possibility of migration between levels. Epidemics are assumed to originate from the first cluster, which is the smallest, and spread to the second and third clusters based on infection and migration rates. In this paper, we formulate the hierarchical epidemic model with controls, considering the quarantine and vaccination as countermeasures against the spread of the virus. The model assumes that quarantine measures can only be applied at the first level of the system, while vaccination and treatment can be applied at any level. We focus on the first cluster in the proposed model, as it has the potential to halt the spread of infection at the local level and prevent further transmission. Our aim is to investigate the effectiveness of a combination of different control strategies and compare their effectiveness.

In our previous work [25], we investigated the unidirectional spread of infection: from the initial level to the subsequent ones. Such dynamics were interpreted as the migration of individuals through land/air transport from the local level to the next. However, there is also an inverse relationship: even under the threat of infection, individuals migrate between levels and, among other things, can return to the source of infection due to compelling individual reasons: be it economic reasons, such as a place of study or work, or personal reasons, for example, the permanent residence of relatives and friends. Therefore, in this model, we modified the mechanism for the transition of individuals between levels. At the same time, the infection continues to initialize on the first layer and spreads in the direct path—from the first level, through the second and to the third, and in the opposite direction—from the third through the second back to the first. A difference from the previous work is the addition of a dead subgroup and analysis of the basic reproductive number $R0$ for our model. We also added the vaccination process, which is one of the effective methods of preventing the spread of the epidemic.

We present the optimal control problem for the hierarchical epidemic model with a intra-level migration and analyze the structure of the optimal protective measures. The analysis of the model includes the construction of the basic reproduction number $R_0$ to estimate the impact of the different parameters on the propagation of the pathogen in the different clusters. A series of numerical experiments are proposed to support the received theoretical results.

This paper is organized as follows: Section 2 introduces the initial model, Section 3 analyzes the system and the basic reproduction number, Section 4 formulates the optimal control problem and presents the structure of the optimal control strategies, Section 5 provides numerical simulations to validate the theoretical results, and Section 6 summarizes the main findings and concludes the paper.

2. Deterministic Epidemic Model

In contrast to classical SIRS models, which divide populations into three groups, this section presents a modified SQEIRD model (susceptible–quarantined–exposed–infected–recovered–dead) with three levels. The model considers one virus circulating in a population of size N and enables us to capture epidemic processes occurring on different levels. These levels can be defined as individual regions, villages, cities within a country, or different countries.

We assume that if an epidemic is initiated in a small village (the first cluster), it can extend to a city, region, or country (the second and third clusters). We formalize this epidemic situation in a mathematical model. Virus propagation starts at the first level of the hierarchy and continues on the i-th levels, $i=2,3$, based on migration rates. The spreading process is modeled using a susceptible–quarantined–exposed–infected–recovered–dead (SQEIRD) model at the first level hierarchy. This model divides the population into six subgroups: susceptible (S), quarantined (Q), exposed (E), infected (I), recovered (R), and dead (D). The susceptible agents are at risk of contracting the virus, and the recovered agents are immune. The quarantined agents are isolated and cannot come into contact with potentially infected agents, making them immune to infection. In this model, vaccination is used in the susceptible groups as a protection measure. WHO reports have demonstrated that vaccination is the most effective way to protect the population from infectious diseases. There is a risk of breaking quarantine rules and coming into contact with an exposed or infected agent, which increases the probability of infection.

The pathogen comes from infected and exposed subgroups into contact with susceptible and quarantine individuals that they have never before encountered. Members of the recovered subgroup receive the immunity as a result of a disease or vaccination. We assume that a subset of the exposed group in the population can transmit the virus without developing symptoms. Unlike the first cluster, the second and third clusters consist of the following subgroups: susceptible $\left(S\right)$, exposed $\left(E\right)$, infected $\left(I\right)$, recovered $\left(R\right)$, and dead $\left(D\right)$. The susceptible group in all clusters can transition to the exposed group at a rate of ${\beta }_i$, $i=\overline{1,3}$. Exposed agents develop symptoms and become infected at the rate of $k_i$, $i=\overline{1,3}$. The self-recovery rates are the same in all clusters, and are defined as ${\sigma }_i$, $i=\overline{1,3}$. The parameter $\delta$ indicates the rate at which susceptible agents choose isolation and become quarantined. With probability p, residents from the Quarantine group can contact an agent from the Exposed group and become exposed with a rate of $p{\beta }_1$. Vaccination is represented by a parameter ${\nu }_i$, which allows susceptible agents to become immune to the virus and belong to the recovered subgroup. The death rate at level i is denoted by ${\mu }_i$. Additionally, we assume that migration occurs between clusters, and that migration rates are $m_x^{ij}$, $x=S,E,R$, and $i,j=1,2,3$. The transition scheme is illustrated in Figure 1.

We describe the epidemic process as a set of nonlinear differential equations, where $N_i\left(t\right)$ denotes the population size at level i at time t. Specifically, $n_{S_i}\left(t\right)$ represents the number of susceptible individuals, $n_{Q_1}\left(t\right)$ represents the number of quarantined individuals, $n_{E_i}\left(t\right)$ represents the number of exposed individuals, $n_{I_i}\left(t\right)$ represents the number of infected individuals, $n_{R_i}\left(t\right)$ represents the number of recovered individuals, and $n_{D_i}\left(t\right)$ represents the number of dead individuals. The following conditions must be satisfied: $n_{S_1}\left(t\right)+n_{Q_1}\left(t\right)+n_{E_1}\left(t\right)+n_{I_1}\left(t\right)+n_{R_1}\left(t\right)+n_{D_1}\left(t\right)=N_1\left(t\right)$, $n_{S_j}\left(t\right)+n_{E_j}\left(t\right)+n_{I_j}\left(t\right)+n_{R_j}\left(t\right)+n_{D_j}\left(t\right)=N_j\left(t\right)$ for $j=\overline{2,3}$, and $N_1\left(t\right)+N_2\left(t\right)+N_3\left(t\right)=N$.

Based on the above definitions, the variables $S_i\left(t\right)$, $Q_1\left(t\right)$, $E_i\left(t\right)$, $I_i\left(t\right)$, $R_i\left(t\right)$, and $D_i\left(t\right)$ indicate the proportions of susceptible, quarantined, exposed, infected, recovered, and dead individuals at time $t\in [0,T]$, where

$$\begin{array}{c}{S}_i\left(t\right)=\frac{n_{S_i}\left(t\right)}{N},Q_1\left(t\right)=\frac{n_{Q_1}\left(t\right)}{N},E_i\left(t\right)=\frac{n_{E_i}\left(t\right)}{N},\hfill \\ I_i\left(t\right)=\frac{n_{I_i}\left(t\right)}{N},R_i\left(t\right)=\frac{n_{R_i}\left(t\right)}{N},D_i\left(t\right)=\frac{n_{D_i}\left(t\right)}{N}.\hfill \end{array}$$

At the start of the epidemic ($t=0$), the majority of individuals are in the susceptible state, and only a small fraction are infected. The initial states for all levels are as follows:

• $S_1\left(0\right)=S_1^0\in (0,1)$, $E_1\left(0\right)=E_1^0\in (0,1)$, $I_1\left(0\right)=I_1^0\in (0,1)$, $Q_1\left(0\right)=R_1\left(0\right)=D_1\left(0\right)=0$;
• $S_2\left(0\right)=S_2^0\in (0,1)$, $E_2\left(0\right)=I_2\left(0\right)=R_2\left(0\right)=D_2\left(0\right)=0$;
• $S_3\left(0\right)=S_3^0\in (0,1),E_3\left(0\right)=I_3\left(0\right)=R_3\left(0\right)=D_3\left(0\right)=0$.

The set of nonlinear differential equations describes the spread of the virus in each cluster/level of the population. On the first level:

$$\begin{array}{c}{\displaystyle \dot{S_1}=-{\beta }_1(E_1+I_1)S_1-{\nu }_1{S}_1-\delta S_1-m_S^{12}{S}_1+m_S^{21}{S}_2;}\hfill \\ {\displaystyle \dot{Q_1}=\delta S_1-p{\beta }_1{Q}_1(E_1+I_1);}\hfill \\ {\displaystyle \dot{E_1}={\beta }_1(E_1+I_1)S_1+p{\beta }_1{Q}_1(E_1+I_1)-k_1{E}_1-m_E^{12}{E}_1+m_E^{21}{E}_2;}\hfill \\ {\displaystyle \dot{I_1}=k_1{E}_1-({\sigma }_1+u_1)I_1-{\mu }_1{I}_1;}\hfill \\ {\displaystyle \dot{R_1}={\nu }_1{S}_1+({\sigma }_1+u_1)I_1-m_R^{12}{R}_1+m_R^{21}{R}_2;}\hfill \\ {\displaystyle \dot{D_1}={\mu }_1{I}_1.}\hfill \end{array}$$

(1)

On the second level:

$$\begin{array}{c}{\displaystyle \dot{S_2}=-{\beta }_2(E_2+I_2)S_2-{\nu }_2{S}_2+m_S^{12}{S}_1-m_S^{21}{S}_2+m_S^{32}{S}_3-m_S^{23}{S}_2;}\hfill \\ {\displaystyle \dot{E_2}={\beta }_2(E_2+I_2)S_2-k_2{E}_2+m_E^{12}{E}_1-m_E^{21}{E}_2+m_E^{32}{E}_3-m_E^{23}{E}_2;}\hfill \\ {\displaystyle \dot{I_2}=k_2{E}_2-({\sigma }_2+u_2)I_2-{\mu }_2{I}_2;}\hfill \\ {\displaystyle \dot{R_2}={\nu }_2{S}_2+({\sigma }_2+u_2)I_2+m_R^{12}{R}_1-m_R^{21}{R}_2+m_R^{32}{R}_3-m_R^{23}{R}_2;}\hfill \\ {\displaystyle \dot{D_2}={\mu }_2{I}_2.}\hfill \end{array}$$

(2)

On the third level:

$${\displaystyle \begin{array}{c}{\displaystyle \dot{S_3}=-{\beta }_3(E_3+I_3)S_3-{\nu }_3{S}_3-m_S^{32}{S}_3+m_S^{23}{S}_2;}\hfill \\ {\displaystyle \dot{E_3}={\beta }_3(E_3+I_3)S_3-k_3{E}_3-m_E^{32}{E}_3+m_E^{23}{E}_2;}\hfill \\ {\displaystyle \dot{I_3}=k_3{E}_3-({\sigma }_3+u_3)I_3-{\mu }_3{I}_3;}\hfill \\ {\displaystyle \dot{R_3}={\nu }_3{S}_3+({\sigma }_3+u_3)I_3-m_R^{32}{R}_3+m_R^{23}{R}_2;}\hfill \\ {\displaystyle \dot{D_3}={\mu }_3{I}_3.}\hfill \end{array}}$$

(3)

3. Basic Reproduction Number ${\mathbf{R}}_{\mathbf{0}}$

This section employs the next generation method (NGM) [26,27] to estimate the basic reproduction number $R_0$ in the model (1)–(3), which represents the average number of cases of an infectious disease that arise via transmission from a single infected individual and approximates the asymptotic behavior of an epidemic process. The value of $R_0$ is crucial in evaluating the infection propagation in the entire population, as the number of infected individuals will asymptotically decrease if $R_0<1$, and increase otherwise.

The NGM method involves transforming the original system into $\frac{d{x}_i}{dt}=F_i-V_i$, where the matrices F and V are given by $F=\frac{d{F}_i}{d{x}_i}$ and $V=\frac{d{V}_i}{d{x}_i}$, respectively. Here, i refers to the indices of all infected state variables in the initial system of differential equations.

In the model (1)–(3), all states $S_i,Q_1{E}_i,I_i,R_i,D_i,i=\overline{1,3}$ can be divided into two groups: infected and non-infected states. Specifically, the infected states are $E_i,I_i,i=\overline{1,3}$, and therefore, the matrices F and V have a size of $6\times 6$, according to NGM. These matrices are presented below.

$$F=\left[\begin{array}{cccccc}{\beta }_1{S}_1+p{\beta }_1{Q}_1& {\beta }_1{S}_1+p{\beta }_1{Q}_1& 0& 0& 0& 0\\ 0& 0& 0& 0& 0& 0\\ 0& 0& {\beta }_2{S}_2& {\beta }_2{S}_2& 0& 0\\ 0& 0& 0& 0& 0& 0\\ 0& 0& 0& 0& {\beta }_3{S}_3& {\beta }_3{S}_3\\ 0& 0& 0& 0& 0& 0\end{array}\right];$$

$$V=\left[\begin{array}{cccccc}{k}_1+m_E^{12}& 0& -m_E^{21}& 0& 0& 0\\ -k_1& {\sigma }_1+u_1+{\mu }_1& 0& 0& 0& 0\\ -m_E^{12}& 0& k_2+m_E^{21}+m_E^{23}& 0& -m_E^{32}& 0\\ 0& 0& -k_2& {\sigma }_2+u_2+{\mu }_2& 0& 0\\ 0& 0& -m_E^{23}& 0& k_3+m_E^{32}& 0\\ 0& 0& 0& 0& -k_3& {\sigma }_3+u_3+{\mu }_3\end{array}\right];$$

Disease-free equilibrium:

$${\displaystyle \begin{array}{c}{\displaystyle S_1^{*}=\frac{m_S^{21}{m}_S^{32}}{m_S^{12}{m}_S^{32}+m_S^{21}{m}_S^{32}+m_S^{23}{m}_S^{12}},Q_1^{*}=E_1^{*}=I_1^{*}=R_1^{*}=D_1^{*}=0,}\\ S_2^{*}=\frac{m_S^{12}{m}_S^{32}}{m_S^{12}{m}_S^{32}+m_S^{21}{m}_S^{32}+m_S^{23}{m}_S^{12}},E_2^{*}=I_2^{*}=R_2^{*}=D_2^{*}=0,\\ S_3^{*}=\frac{m_S^{23}{m}_S^{12}}{m_S^{12}{m}_S^{32}+m_S^{21}{m}_S^{32}+m_S^{23}{m}_S^{12}},E_3^{*}=I_3^{*}=R_3^{*}=D_3^{*}=0,\end{array}}$$

(4)

According to the NGM, the maximum value of the absolute eigenvalues of $F{V}^{-1}$ is equal to the reproduction number $R_0$. Numerically estimating various parameter combinations, we found that migration only has a significant impact when ${\sigma }_i$ or ${\beta }_i$ differs across levels $i=1,2,3$. The diagrams below depict how $R_0$ varies with different parameters. In the following numerical experiments, we used the following parameter values, unless otherwise specified: ${\beta }_i=0.25$, $k_i=0.142$, $u_i=0$, ${\sigma }_i=0.047$, $m_E^{12}=m_E^{23}=m_S^{12}=m_S^{23}=0.01,m_E^{21}=m_E^{32}=m_S^{21}=m_S^{32}=0.005$.

As shown in the left diagram of Figure 2, the reproduction number, $R_0$, varies linearly with the infection rate, $\beta$, and decreases as the control increases. On the other hand, the right diagram of Figure 2 depicts $R_0$ as a function of the migration rate, $\beta$, and control, which takes the shape of an almost constant hyperbola when the migration rate exceeds 0.5. This diagram also demonstrates that the reproduction number decreases as the control increases, and can fall below 1 even without the control if the migration rate exceeds 0.5.

4. Optimal Control Problem

The focus of the present study is to formulate an optimal control problem aimed at minimizing the damage caused by virus outbreaks and enhancing the protection of the population. The model employed in this study assumes that quarantine and treatment measures can shield the population from viral transmission. At each level of the hierarchical structure, treatment is applied as a control strategy $u_1\left(t\right)$, $u_2\left(t\right)$, $u_3\left(t\right)$ for infected nodes ($I\to R$), if the epidemic extends across all levels due to migration rates. To estimate the infection costs, the method proposed in [10,11,12] is adopted, which includes direct losses suffered by an infected person, such as treatment expenses, loss of productivity, and incapacity. The treatment costs, which indicate the value spent by the government to facilitate the health care system’s ability to treat infected individuals and enhance the probability of their recovery, are considered as external costs.

Cost functions: The costs associated with infections and treatments are represented by functions $f_i\left(I_i\left(t\right)\right)$ and $h_i\left(u_i\left(t\right)\right)$, respectively, at any given time t. These functions must satisfy certain conditions. The functions $f_i\left(I_i\right)$ are non-decreasing, twice-differentiable, and convex, with $f_i\left(0\right)=0$ and $f_i\left(I_i\right)>0$ for $I_i>0$, where $i=\overline{1,3}$. Similarly, the functions $h_i\left(u_i\left(t\right)\right)={\overline{h}}_i{u}_i^2\left(t\right)$ are twice-differentiable and increasing in $u_i\left(t\right)$, with $h_i\left(0\right)=0$ and $h_i\left(u_i\left(t\right)\right)>0$ for $i=\overline{1,3}$ when $u_i\left(t\right)>0$.

The functional for the aggregated system costs on the time interval $[0,T]$ is defined as follows:

$$J=J_1+J_2+J_3,$$

where

$$\begin{array}{c}{\displaystyle J_1={\int }_0^T\left(f_1\left(I_1\left(t\right)\right)+h_1\left(u_1\left(t\right)\right)\right)dt,}\\ \\ {\displaystyle J_2={\int }_0^T\left(f_2\left(I_2\left(t\right)\right)+h_2\left(u_2\left(t\right)\right)\right)dt,}\\ \\ {\displaystyle J_3={\int }_0^T\left(f_3\left(I_3\left(t\right)\right)+h_3\left(u_3\left(t\right)\right)\right)dt.}\end{array}$$

(5)

The goal of the optimal control problem is to minimize the costs given by the functional J:

$${\displaystyle \underset{u_1,u_2,u_3}{min}J.}$$

By applying Pontryagin’s maximum principle [28,29], we can construct the Hamiltonian and adjoint functions. The generalized Hamiltonian of the system is given as follows: $H=H_1+H_2+H_3$. Hamiltonian $H_1$ of the first level:

$$\begin{array}{c}{H}_1=-f_1\left(I_1\right)-h_1\left(u_1\right)+({\lambda }_{E_1}-{\lambda }_{S_1}){\beta }_1(E_1+I_1)S_1+({\lambda }_{R_1}-{\lambda }_{S_1}){\nu }_1{S}_1+\hfill \\ ({\lambda }_{Q_1}-{\lambda }_{S_1})\delta S_1+({\lambda }_{E_1}-{\lambda }_{Q_1})p{\beta }_1{Q}_1(E_1+I_1)+({\lambda }_{I_1}-{\lambda }_{E_1})k_1{E}_1+\hfill \\ ({\lambda }_{R_1}-{\lambda }_{I_1})({\sigma }_1+u_1)I_1+({\lambda }_{S_2}-{\lambda }_{S_1})m_S^{12}{S}_1+({\lambda }_{E_2}-{\lambda }_{E_1})m_E^{12}{E}_1+\hfill \\ ({\lambda }_{R_2}-{\lambda }_{R_1})m_R^{12}{R}_1+({\lambda }_{D_1}-{\lambda }_{I_1}){\mu }_1{I}_1.\hfill \end{array}$$

(6)

Hamiltonian $H_2$ of the second level:

$$\begin{array}{c}{H}_2=-f_2\left(I_2\right)-h_2\left(u_2\right)+({\lambda }_{E_2}-{\lambda }_{S_2}){\beta }_2(E_2+I_2)S_2+({\lambda }_{R_2}-{\lambda }_{S_2}){\nu }_2{S}_2+\hfill \\ ({\lambda }_{I_2}-{\lambda }_{E_2})k_2{E}_2+({\lambda }_{R_2}-{\lambda }_{I_2})({\sigma }_2+u_2)I_2+({\lambda }_{S_3}-{\lambda }_{S_2})m_S^{23}{S}_2+\hfill \\ ({\lambda }_{E_3}-{\lambda }_{E_2})m_E^{23}{E}_2+({\lambda }_{R_3}-{\lambda }_{R_2})m_R^{23}{R}_2+({\lambda }_{D_2}-{\lambda }_{I_2}){\mu }_2{I}_2+\hfill \\ ({\lambda }_{S_1}-{\lambda }_{S_2})m_S^{21}{S}_2+({\lambda }_{E_1}-{\lambda }_{E_2})m_E^{21}{E}_2+({\lambda }_{R_1}-{\lambda }_{R_2})m_R^{21}{R}_2.\hfill \end{array}$$

(7)

Hamiltonian $H_3$ of the third level:

$$\begin{array}{c}{H}_3=-f_3\left(I_3\right)-h_3\left(u_3\right)+({\lambda }_{E_3}-{\lambda }_{S_3}){\beta }_3(E_3+I_3)S_3+({\lambda }_{R_3}-{\lambda }_{S_3}){\nu }_3{S}_3+\hfill \\ ({\lambda }_{I_3}-{\lambda }_{E_3})k_3{E}_3+({\lambda }_{R_3}-{\lambda }_{I_3})({\sigma }_3+u_3)I_3+({\lambda }_{D_3}-{\lambda }_{I_3}){\mu }_3{I}_3+\hfill \\ ({\lambda }_{S_2}-{\lambda }_{S_3})m_S^{32}{S}_3+({\lambda }_{E_2}-{\lambda }_{E_3})m_E^{32}{E}_3+({\lambda }_{R_2}-{\lambda }_{R_3})m_R^{32}{R}_3.\hfill \end{array}$$

(8)

The adjoint functions of the first level are defined as follows: ${\lambda }_{S_1}\left(t\right)$, ${\lambda }_{Q_1}\left(t\right)$, ${\lambda }_{E_1}\left(t\right)$, ${\lambda }_{I_1}\left(t\right)$, and ${\lambda }_{R_1}\left(t\right)$.

$$\begin{array}{c}{\dot{\lambda }}_{S_1}\left(t\right)=({\lambda }_{S_1}-{\lambda }_{E_1}){\beta }_1(E_1+I_1)+({\lambda }_{S_1}-{\lambda }_{R_1}){\nu }_1+({\lambda }_{S_1}-{\lambda }_{Q_1})\delta +\hfill \\ ({\lambda }_{S_1}-{\lambda }_{S_2})m_S^{12};\hfill \\ {\dot{\lambda }}_{Q_1}\left(t\right)=({\lambda }_{Q_1}-{\lambda }_{E_1})p{\beta }_1(E_1+I_1);\hfill \\ {\dot{\lambda }}_{E_1}\left(t\right)=({\lambda }_{S_1}-{\lambda }_{E_1}){\beta }_1{S}_1+({\lambda }_{Q_1}-{\lambda }_{E_1})p{\beta }_1{Q}_1+({\lambda }_{E_1}-{\lambda }_{I_1})k_1+({\lambda }_{E_1}-{\lambda }_{E_2})m_E^{12};\hfill \\ {\dot{\lambda }}_{I_1}\left(t\right)=f_1^{\prime }\left(I_1\right)+({\lambda }_{S_1}-{\lambda }_{E_1}){\beta }_1{S}_1+({\lambda }_{Q_1}-{\lambda }_{E_1})p{\beta }_1{Q}_1+({\lambda }_{I_1}-{\lambda }_{R_1})({\sigma }_1+u_1)+\hfill \\ ({\lambda }_{I_1}-{\lambda }_{D_1}){\mu }_1;\hfill \\ {\dot{\lambda }}_{R_1}\left(t\right)=({\lambda }_{R_1}-{\lambda }_{R_2})m_E^{12};\hfill \\ {\dot{\lambda }}_{D_1}\left(t\right)=0,\hfill \end{array}$$

(9)

The transversality conditions are given by:

$${\lambda }_{S_1}\left(T\right)={\lambda }_{Q_1}\left(T\right)={\lambda }_{E_1}\left(T\right)={\lambda }_{I_1}\left(T\right)={\lambda }_{R_1}\left(T\right)={\lambda }_{D_1}\left(T\right)=0.$$

(10)

Analogously, adjoint functions ${\lambda }_{S_2}\left(t\right)$, ${\lambda }_{E_2}\left(t\right)$, ${\lambda }_{I_2}\left(t\right)$, and ${\lambda }_{R_2}\left(t\right)$ of the second level are:

$$\begin{array}{c}{\dot{\lambda }}_{S_2}\left(t\right)=({\lambda }_{S_2}-{\lambda }_{E_2}){\beta }_2(E_2+I_2)+({\lambda }_{S_2}-{\lambda }_{R_2}){\nu }_2+({\lambda }_{S_2}-{\lambda }_{S_1})m_S^{21}+\hfill \\ ({\lambda }_{S_2}-{\lambda }_{S_3})m_S^{23};\hfill \\ {\dot{\lambda }}_{E_2}\left(t\right)=({\lambda }_{S_2}-{\lambda }_{E_2}){\beta }_2{S}_2+({\lambda }_{E_2}-{\lambda }_{I_2})k_2+({\lambda }_{E_2}-{\lambda }_{E_1})m_E^{21}+({\lambda }_{E_2}-{\lambda }_{E_3})m_E^{23};\hfill \\ {\dot{\lambda }}_{I_2}\left(t\right)=f_2^{\prime }\left(I_2\right)+({\lambda }_{S_2}-{\lambda }_{E_2}){\beta }_2{S}_2+({\lambda }_{I_2}-{\lambda }_{R_2})({\sigma }_2+u_2)+({\lambda }_{I_2}-{\lambda }_{D_2}){\mu }_2\hfill \\ {\dot{\lambda }}_{R_2}\left(t\right)=({\lambda }_{R_2}-{\lambda }_{R_1})m_E^{12}+({\lambda }_{R_2}-{\lambda }_{R_3})m_E^{23};\hfill \\ {\dot{\lambda }}_{D_2}\left(t\right)=0,\hfill \end{array}$$

(11)

with the transversality conditions given by

$${\lambda }_{S_2}\left(T\right)={\lambda }_{E_2}\left(T\right)={\lambda }_{I_2}\left(T\right)={\lambda }_{R_2}\left(T\right)={\lambda }_{D_2}\left(T\right)=0.$$

(12)

Adjoint functions ${\lambda }_{S_3}\left(t\right)$, ${\lambda }_{E_3}\left(t\right)$, ${\lambda }_{I_3}\left(t\right)$, and ${\lambda }_{R_3}\left(t\right)$ of the third level:

$$\begin{array}{c}{\dot{\lambda }}_{S_3}\left(t\right)=({\lambda }_{S_3}-{\lambda }_{E_3}){\beta }_3(E_3+I_3)+({\lambda }_{S_3}-{\lambda }_{R_3}){\nu }_3+({\lambda }_{S_3}-{\lambda }_{S_3})m_S^{32};\hfill \\ {\dot{\lambda }}_{E_3}\left(t\right)=({\lambda }_{S_3}-{\lambda }_{E_3}){\beta }_3{S}_3+({\lambda }_{E_3}-{\lambda }_{I_3})k_3+({\lambda }_{E_3}-{\lambda }_{E_3})m_E^{32};\hfill \\ {\dot{\lambda }}_{I_3}\left(t\right)=f_3^{\prime }\left(I_3\right)+({\lambda }_{S_3}-{\lambda }_{E_3}){\beta }_3{S}_3+({\lambda }_{I_3}-{\lambda }_{R_3})({\sigma }_3+u_3)+({\lambda }_{I_3}-{\lambda }_{D_3}){\mu }_3;\hfill \\ {\dot{\lambda }}_{R_3}\left(t\right)=({\lambda }_{R_3}-{\lambda }_{R_2})m_E^{32};\hfill \\ {\dot{\lambda }}_{D_3}\left(t\right)=0,\hfill \end{array}$$

(13)

with the transversality conditions given by

$${\lambda }_{S_3}\left(T\right)={\lambda }_{E_3}\left(T\right)={\lambda }_{I_3}\left(T\right)={\lambda }_{R_3}\left(T\right)={\lambda }_{D_3}\left(T\right)=0.$$

(14)

The existence of continuous and piece-wise continuously differentiable co-state functions

$${\lambda }_r\left(t\right),r\in S_1,Q_1,E_1,\dots ,R_3,D_3,$$

which satisfy (9)–(14) at every time $t\in [0,T]$, together with continuous functions $u_1^{*}\left(t\right)$, $u_2^{*}\left(t\right)$, and $u_3^{*}\left(t\right)$, is guaranteed by Pontryagin’s maximum principle:

$$\begin{array}{c}(u_1^{*},u_2^{*},u_3^{*})\in \arg \underset{u_1,u_2,u_3\in [0,u_{max}]}{max}H(\lambda ,S_1,Q_1,\dots ,R_3,D_3,u_1,u_2,u_3).\hfill \end{array}$$

(15)

Following the Pontryagin’s maximum principle [11,28,29,30,31], we should analyze the behaviour of the derivative $d{H}_i/d{u}_i$. We obtain that

$$\begin{array}{c}{\displaystyle \frac{\partial H_i}{\partial u_i}=-{\dot{h}}_i\left(u_i\right)+({\lambda }_{R_i}-{\lambda }_{I_i})I_i,i=\overline{1,3}.}\hfill \end{array}$$

(16)

We denote the last terms in (16) as switching functions ${\phi }_i\left(t\right)$, $i=\overline{1,3}$:

$$\begin{array}{c}{\phi }_1\left(t\right)=({\lambda }_{R_1}\left(t\right)-{\lambda }_{I_1}\left(t\right))I_1\left(t\right),\hfill \\ {\phi }_2\left(t\right)=({\lambda }_{R_2}\left(t\right)-{\lambda }_{I_2}\left(t\right))I_2\left(t\right),\hfill \\ {\phi }_3\left(t\right)=({\lambda }_{R_3}\left(t\right)-{\lambda }_{I_3}\left(t\right))I_3\left(t\right).\hfill \end{array}$$

(17)

Proposition 1.

Assuming that $h_i(·)$ functions are concave, the optimal control structure is expressed as follows for $i=1,2,3$:

$$u_i^{*}\left(t\right)=\left\{\begin{array}{c}0,for{\phi }_i\left(t\right)<h_i\left(u_{max}\right);\hfill \\ u_{max},for{\phi }_i\left(t\right)\ge h_i\left(u_{max}\right).\hfill \end{array}\right.$$

(18)

Proposition 2.

If $h_i(·)$ are strictly convex functions, the optimal control structure is as follows for any $i=\overline{1,3}$

$$u_i^{*}\left(t\right)=\left\{\begin{array}{c}0,for{\phi }_i\left(t\right)\le h_i^{\prime }\left(0\right);\hfill \\ h^{\prime -1}\left({\phi }_i\right),for{h}_i^{\prime }\left(0\right)<{\phi }_i\left(t\right)\le h_i^{\prime }\left(u_{max}\right);\hfill \\ u_{max},for{h}_i^{\prime }\left(u_{max}\right)<{\phi }_i\left(t\right).\hfill \end{array}\right.$$

(19)

where $u_{max}\in [0,1]$.

To prove these propositions, we can rewrite the Hamiltonian in terms of the function ${\phi }_i\left(t\right)$. After that, we obtain:

$$\begin{array}{cc}H=\hfill & ({\phi }_1{u}_1-h_1\left(u_1\right))+({\phi }_2{u}_2-h_2\left(u_2\right))+({\phi }_3{u}_3-h_3\left(u_3\right))-f_1\left(I_1\right)-f_2\left(I_2\right)-\hfill \\ & f_3\left(I_3\right)+({\lambda }_{E_1}-{\lambda }_{S_1}){\beta }_1(E_1+I_1)S_1+({\lambda }_{Q_1}-{\lambda }_{S_1})\delta S_1+({\lambda }_{R_1}-{\lambda }_{S_1}){\nu }_1{S}_1+\hfill \\ & ({\lambda }_{E_1}-{\lambda }_{Q_1})p{\beta }_1{Q}_1(E_1+{\lambda }_{I_1})+({\lambda }_{I_1}-{\lambda }_{E_1})k_1{E}_1+({\lambda }_{R_1}-{\lambda }_{I_1}){\sigma }_1{I}_1+\hfill \\ & ({\lambda }_{S_2}-{\lambda }_{S_1})m_S^{12}{S}_1+({\lambda }_{E_2}-{\lambda }_{E_1})m_E^{12}{E}_1+({\lambda }_{R_2}-{\lambda }_{R_1})m_R^{12}{R}_1+\hfill \\ & ({\lambda }_{D_1}-{\lambda }_{I_1}){\mu }_1{I}_1+({\lambda }_{E_2}-{\lambda }_{S_2}){\beta }_2(E_2+I_2)S_2+({\lambda }_{R_2}-{\lambda }_{S_2})v_2{S}_2+\hfill \\ & ({\lambda }_{I_2}-{\lambda }_{E_2})k_2{E}_2+({\lambda }_{R_2}-{\lambda }_{I_2}){\sigma }_2{I}_2+({\lambda }_{S_3}-{\lambda }_{S_2})m_S^{23}{S}_2+\hfill \\ & ({\lambda }_{S_1}-{\lambda }_{S_2})m_S^{21}{S}_2+({\lambda }_{E_1}-{\lambda }_{E_2})m_E^{21}{E}_2+({\lambda }_{R_1}-{\lambda }_{R_2})m_R^{21}{R}_2+\hfill \\ & ({\lambda }_{E_3}-{\lambda }_{E_2})m_E^{23}{E}_2+({\lambda }_{R_3}-{\lambda }_{R_2})m_R^{23}{R}_2+({\lambda }_{D_2}-{\lambda }_{I_2}){\mu }_2{I}_2+\hfill \\ & ({\lambda }_{R_3}-{\lambda }_{S_3})v_3{S}_3+({\lambda }_{E_3}-{\lambda }_{S_3}){\beta }_3(E_3+I_3)S_3+({\lambda }_{I_3}-{\lambda }_{E_3})K_3{E}_3+\hfill \\ & ({\lambda }_{D_3}-{\lambda }_{I_3}){\mu }_3{I}_3+({\lambda }_{S_2}-{\lambda }_{S_3})m_S^{32}{S}_3+({\lambda }_{E_2}-{\lambda }_{E_3})m_E^{32}{E}_3+\hfill \\ & ({\lambda }_{R_2}-{\lambda }_{R_3})m_R^{32}{R}_3+({\lambda }_{R_3}-{\lambda }_{I_3}){\sigma }_3{I}_3\hfill \end{array}$$

(20)

The optimization problem can be decomposed into three subproblems, where we consider the optimal controls $u_1^{*}\left(t\right),u_2^{*}\left(t\right)$, and $u_3^{*}\left(t\right)$ separately. This can be achieved by solving:

$$\begin{array}{c}max{u}_1[-h_1\left(u_1\right)+{\phi }_1{u}_1]+max{u}_2[-h_2\left(u_2\right)+{\phi }_2{u}_2]+\underset{u_3}{max}[-h_3\left(u_3\right)+{\phi }_3{u}_3].\hfill \end{array}$$

(21)

For any admissible control $\overline{u_i}$, and for all $t\in [0,T]$ according to (20), we have:

$$\begin{array}{c}-h_i\left(u_i\right)+u_i{\phi }_i\ge -h_i\left(\overline{u_i}\right)+\overline{u_i}{\phi }_i.\hfill \end{array}$$

(22)

As $\overline{u_i}=0$ is an admissible control, we obtain:

$$\begin{array}{c}-h_i\left(u_i\right)+u_i{\phi }_i\ge -h_i\left(0\right)+0·{\phi }_i=0.\hfill \end{array}$$

(23)

To determine the optimal control structure using Pontryagin’s maximum principle, we consider the following derivatives:

$$\begin{array}{c}{\displaystyle \frac{\partial H_i}{\partial u_i}=-{\dot{h}}_i\left(u_i\right)+{\phi }_i=0,i=\overline{1,3}.}\hfill \end{array}$$

(24)

Given that $h_i(·)$ are increasing functions and $I_i(·)\ge 0$, we can maximize the Hamiltonian by ensuring that ${\phi }_i\left(t\right)={\dot{h}}_i\left(u_i\left(t\right)\right)\ge 0$ for $i=1,2,3$, as stated in the proposition. Using Equation (17) and the fact that $I_i\left(t\right)\ge 0$ for all $t\in [0,T]$, we can rewrite this condition as ${\lambda }_{R_i}\left(t\right)-{\lambda }_{I_i}\left(t\right)\ge 0$ for $i=\overline{1,3}$.

To complete the proof of the proposition, we need to establish Lemma 1:

Lemma 1.

For all $t\in [0,T]$, we have ${\lambda }_{R_i}\left(t\right)-{\lambda }_{I_i}\left(t\right)\ge 0$, $i=\overline{1,3}$.

The proof of Lemma 1 consists of two parts. First, we consider the case when $t=T$ and show that the derivatives of the functions ${\lambda }_{R_i}\left(t\right)-{\lambda }_{I_i}\left(t\right)$, $i=\overline{1,3}$, are non-positive. Second, we prove by contradiction the negativity of these functions on the whole interval $[0,T]$. The complete proofs of Lemma 1 follow the same technique as that in [11,30].

Note that since $h_i(·)$ are increasing functions, the Hamiltonian is convex in $u_i(·)$, if $h_i(·)$ are also concave, i.e., $h^{\prime \prime }i(·)\le 0$, according to Equations (6)–(8). As a result, there are two different options for $u_i\in [0,u_{max}]$ that maximize the Hamiltonian, where $u_{max}\in [0,1]$.

If $-h_i\left(0\right)+{\phi }_i\left(t\right)·0>-h_i\left(u_{max}\right)+{\phi }_i\left(t\right)·u_{max}$ or $h_i\left(u_{max}\right)>{\phi }_i\left(t\right)u_{max}$, then the optimal control is $u_i=0$ (see Figure 3(left)). Otherwise, the optimal control is $u_i=u_{max}$ (see Figure 3(right)).

4.1. Functions $h_i(·)$ Are Concave

Assuming $h_i(·)$ is a concave function with $h^{\prime \prime }i(·)\le 0$, it follows from (6)–(8) that the Hamiltonian is a convex function of $u_i(·)$ for $i=\overline{1,3}$. There exist two possible values of $u_i\in [0,u_{max}]$ that maximize the Hamiltonian, where $u_{max}\in [0,1]$.

If $-h_i\left(0\right)+{\phi }_i\left(t\right)·0>-h_i\left(u_{max}\right)+{\phi }_i\left(t\right)·u_{max}$ or $h_i\left(u_{max}\right)>{\phi }_i\left(t\right)u_{max}$, then the optimal control is $u_i=0$ (see Figure 3(left)); otherwise, $u_i=u_{max}$ (see Figure 3(right)).

For $i=\overline{1,3}$ the optimal control parameters $u_i\left(t\right)$ are defined as follows:

$$u_i^{*}\left(t\right)=\left\{\begin{array}{c}0,for{\phi }_i\left(t\right)u_{max}<h_i\left(u_{max}\right),\hfill \\ u_{max},for{\phi }_i\left(t\right)u_{max}\ge h_i\left(u_{max}\right).\hfill \end{array}\right.$$

(25)

4.2. Functions $h_i(·)$ Are Strictly Convex

Assuming $h_i(·)$ is a strictly convex function ($h_i^{\prime \prime }(·)>0$), the Hamiltonian is a concave function. We consider the derivative:

$$\begin{array}{c}{\displaystyle \frac{\partial }{\partial x}(-h_i\left(x\right)+{\phi }_i\left(t\right)x){\mid }_{x=x_i}=0,}\hfill \end{array}$$

(26)

where $x\in [0,u_{max}]$, $u^i\left(t\right)=x_i$, and $u_{max}\in [0,1]$. There exist three different types of points at which the Hamiltonian reaches its maximum (Figure 4). To find them, we need to consider the derivatives of the Hamiltonian at $u_i=0$ and $u_i=u_{max}$. If the derivatives (26) at $u_i=0$ are non-increasing ($-h_i^{\prime }\left(0\right)+{\phi }_i\left(t\right)\le 0$), then the value of the control that maximizes the Hamiltonian is less than 0, and according to our restrictions ($u_i\in [0,u_{max}]$), the optimal control will be equal to 0 (Figure 4 (left)). If the derivatives at $u_i=u_{max}$ are increasing ($-h_i^{\prime }\left(u_{max}\right)+{\phi }_i\left(t\right)>0$), this implies that the value of the control that maximizes the Hamiltonian is greater than $u_{max}$. Hence, the optimal control will be 1 (Figure 4 (right)). Otherwise, we can find the value of $u^i\in (0,u_{max})$ (see Figure 4 (middle)):

$$u_i^{*}\left(t\right)=\left\{\begin{array}{c}0,for{\phi }_i\left(t\right)\le h_i^{\prime }\left(0\right),\hfill \\ h^{\prime -1}\left({\phi }_i\left(t\right)\right),for{h}_i^{\prime }\left(0\right)<{\phi }_i\left(t\right)\le h_i^{\prime }\left(u_{max}\right),\hfill \\ u_{max},for{h}_i^{\prime }\left(u_{max}\right)<{\phi }_i\left(t\right),i=1,2,3.\hfill \end{array}\right.$$

(27)

5. Numerical Simulation

In this section, we conduct a series of numerical experiments to validate our findings. We consider a country with a population of 10,000,000, where the first level has 1,500,000 residents, the second level has 3,000,000 residents, and the third level has 5,500,000 residents. The time interval is one year ($T=365$ days).

We start with the following initial distribution among susceptible, exposed, and infected groups: $S_1^0=0.1489$, $E_1\left(0\right)=0.001$, and $I_1^0=0.0001$, which correspond to 10,000 exposed and 1000 infected residents at $t_0=0$. The transmission rate is ${\beta }_i=0.25$, and the self-recovery rate is ${\sigma }_i=0.047$, which takes approximately 21 days for a resident to recover from the virus without any treatment [32]. The incubation period is $k_i=0.142$ for $i=\overline{1,3}$. We assume that symptoms appear after about 6 days, and exposed residents become infected. The cost functions for the infected group are $f_i\left(I_i\left(t\right)\right)={\overline{f}}_i{I}_i\left(t\right)$, and the treatment cost functions are defined as $h_i\left(u_i\left(t\right)\right)={\overline{h}}_i{u}_i^2\left(t\right)$ for $i=\overline{1,3}$. In this model, we assume that the maximum value of the control is $u_{max}=0.02$. More details about the parameters and initial data used in the experiments can be found in

Table 1. Parameters used for simulations in experiments.

Parameter Name	Exp. 1	Exp. 2	Exp. 3a	Exp. 3b
Fraction of	0.1489	0.1489	0.1489	0.1489
susceptible agents	0.3	0.3	0.3	0.3
at time $t=0$ ($S_i^0$)	0.55	0.55	0.55	0.55
Fraction of	0.001	0.001	0.001	0.001
exposed agents	0	0	0	0
at time $t=0$ ($E_i^0$)	0	0	0	0
Fraction of	0.0001	0.0001	0.0001	0.0001
infected agents	0	0	0	0
at time $t=0$ ($I_i^0$)	0	0	0	0
Infection rate	0.25	0.25	0.25	0.25
from $S_i$ to $E_i$ (${\beta }_i$)
Recovery rate (${\sigma }_i$)	0.047	0.047	0.047	0.047
Asymptomatic to	0.142	0.142	0.142	0.142
Infected ($k_i$)
Voluntary self-isolation ($\delta$)	0	0.001	0.1	0.00001
Return from self-isolation (p)	0	0.1	0	0.3
Vaccination rate (${\nu }_i$)	0	0.005	0.005	0.005
Death rate (${\mu }_i$)	0.005	0.005	0.005	0.005
Migration rates	0.01	0.01	0.01	0.01
($m_x^{ii+1},x=\{S,E,R\}$)
Migration rates ($m_x^{21}$)	0.001	0.001	0.001	0.001
Migration rates ($m_x^{32}$)	0.005	0.005	0.005	0.005
Infection costs ${\overline{f}}_i$	100	100	100	100
Treatment costs ${\overline{h}}_i$	20	20	20	20
Maximum values	0.02	0.02	0.02	0.02
of control ($u_{max}$)
Aggregated costs J	1.6866 $\times {10}^{10}$	$5.1959\times {10}^9$	$3.8543\times {10}^9$	$5.2672\times {10}^9$
Uncontrolled case
Aggregated costs J	$1.1305\times {10}^{10}$	$2.5804\times {10}^9$	$1.7372\times {10}^9$	$2.6239\times {10}^9$
Controlled case

.

Experiment 1. In the current experiment, we present the SQEIRD model under the assumption that the quarantine and vaccination are not possible for susceptible individuals ($\delta =0$, $p=0$, and $\nu =0$). Using the specially developed procedure and taking into consideration the cluster structure of the population, we obtain the result that the epidemic starts and reaches its maximum faster on the first level than on levels 2 and 3 in the uncontrolled case. At the second and the third clusters, the epidemic starts with delays. Figure 5 and Figure 6 demonstrate this fact.

The behaviour of the system in the controlled case is shown in Figure 7. We can notice a significant decrease in the number of the infected at all levels, which leads to a reduction in costs (see Figure 8c).

Figure 9 shows the change in the fractions of the exposed, infected, and dead population at all three levels. Since we do not apply any quarantine measures, the fraction $Q_1$ is equal to zero on all considered intervals $[0,T]$.

Figure 8 summarize the changes in all fractions of the population (Figure 8a), the optimal treatment policies structure $u_i\left(t\right)$ (Figure 8b), and the aggregate system costs. In the uncontrolled case, the costs are equal to $1.6866\times {10}^{10}$ monetary units (m.u.), while in the controlled case, they are $1.1305\times {10}^{10}$ m.u. (Figure 8c).

Experiment 2. In the current experiment, we consider the SQEIRD model with quarantine and vaccination. We assume that the government has an access to a vaccine that forms an acquired immunity to the virus and also has imposed quarantine measures.

This experiment shows the effects of vaccination and quarantine measures on the development of an epidemic situation. The behavior of the system in the controlled case is shown in Figure 10.

Figure 11 shows the change in the fractions of the quarantined, exposed, infected, and dead population at all three levels. According to the quarantine measures, the citizens should stay at home, but there is a small fraction of residents that do not comply with these measures, which leads to their infection.

Figure 12a shows the aggregate changes in all fractions of the population, while Figure 12b represents the structure of the optimal treatment policies $u_i\left(t\right)$, $i=1,2,3$. The controlled case results in a significantly lower aggregated system cost of $2.5804\times {10}^9$ m.u. compared to the uncontrolled case’s cost of $5.1959\times {10}^9$ m.u. (Figure 12c).

The implementation of quarantine measures and vaccination in the optimal control strategy leads to a reduction in the overall epidemic spread, with a maximum number of infected individuals of $2.24\times {10}^5$ on the third level, which is much lower than the $1.57\times {10}^6$ peak observed in Experiment 1. Additionally, the peak of infected residents on the third level occurs later, on the 168th day, as opposed to Experiment 1, where it occurred on the 126th day. These results suggest that optimal controllers can effectively conserve the health system’s resources, including medical staff and equipment, and prevent its collapse.

Experiment 3. In this experiment, we examine the impact of agent responsibility on the system’s behavior.

First, we consider a scenario in which all individuals are responsible for their actions (Experiment 3a). It is assumed that taking responsibility for their actions has a positive effect on the overall epidemic spread and minimizes the overall costs. Residents voluntarily remain in quarantine throughout the time interval T = [0;365]. The transition from the susceptible subgroup to quarantine (voluntary self-isolation) occurs at a rate of $\delta =0.1$, which is higher than in Experiment 2 ($\delta =0.001$). However, in contrast to the previous case, the transition rate from quarantine to the exposed subgroup is equal to $p=0$, which is lower than in the previous experiment. All other system parameters remain the same. In this section of the experiment, we investigate the impact of agent responsibility on the behavior of the system. In Experiment 3a, we assume that all individuals are responsible for their actions, and residents voluntarily remain in quarantine until the end of the interval T = 365. The transition rate from the susceptible subgroup to the quarantine subgroup (voluntary self-isolation) is $\delta =0.1$, which is higher than in Experiment 2 ($\delta =0.001$). However, unlike the previous case, the transition rate from the quarantine to exposed is $p=0$, which is lower than in Experiment 2. The other parameters of the system remain the same.

Our results demonstrate that the peak number of infected agents is lower in Experiment 3a ($1.5\times {10}^5$) than in the case where residents leave quarantine independently without the government’s consent ($2.4\times {10}^5$). The total system costs in the uncontrolled case are $3.8543\times {10}^9$ m.u., while in the controlled case, they are $1.7372\times {10}^9$ m.u. Comparing these costs with those of the previous experiment confirms the hypothesis that the general responsibility of residents in the population leads to lower overall epidemic costs.

In Experiment 3b, we assume that all individuals in the population are irresponsible and prefer to violate the quarantine isolation rules. Under this assumption, the transition rate from susceptible to quarantined subgroups is $\delta =0.00001$, which is lower than in Experiments 2 and 3a. At the same time, quarantined individuals become exposed at a higher rate of $p=0.3$ compared to the case of responsible residents. All other parameters remain the same. The peak number of infected agents is equal to $2.7\times {10}^5$. It can be noted that the irresponsibility of residents leads to the rapid spread of the virus and a larger number of infected individuals even in the controlled case. Aggregate system costs in the uncontrolled case are equal to $5.2672\times {10}^9$ m.u., compared to $2.6239\times {10}^9$ m.u. in the controlled case. Comparing the costs with the previous experiment, the hypothesis of the general responsibility of residents in the population is also confirmed. The irresponsible residents increase the aggregated cost of the epidemic.

Experiment 4. In this experiment, we conduct a series of experiments, in each of which, we switch the control on and off at different levels. After that, we compare the resulting costs at each individual level. All parameters are taken the same as in Experiment 2. In this case, we run a series of experiments, in which we turn the control on and off at different levels. We then compare the resulting costs at each level. Costs at level i are $J_i$, $i=\overline{1,3}$. We decided to look at five different cases:

• Switching on optimal control at all three levels;
• Switching on optimal control at levels 1 and 2;
• Switching on optimal control at level 1 only;
• Switching on optimal control at levels 2 and 3;
• Without any control (uncontrolled case).

We decided to investigate the effect of control at different levels on the total cost of the epidemic.

Table 2. Costs on each level relative to the costs in uncontrolled case.

	Control on All Three Levels	Control on Levels 1 and 2	Control Only on Level 1	Control on Levels 2 and 3	Control Only on Level 3	Without Any Control
Costs J1	0.72	0.74	0.76	0.97	0.99	1.00
Costs J2	0.45	0.56	0.98	0.45	0.84	1.00
Costs J3	0.5	1.00	1.00	0.5	0.51	1.00
Total costs J	0.5	0.95	0.99	0.5	0.55	1.00

shows the relative costs at each level $J_i$ and the total costs J. We took the uncontrolled case as the baseline and compared all the other costs to it, by dividing our costs by the costs in the uncontrolled case.

As a result, two theses can be identified. The first thesis is the more obvious one. Switching on the optimal control at a level results in a reduction in cost at that level compared to the cost in the case without the control. The second thesis is a little less obvious. If we take into account the total costs in each of the experiments, the following trend can be observed. Switching on the optimal control on the third level leads to a reduction in overall costs. This effect is due to the fact that the migration rate is greater from level 1 to level 2 and from level 2 to level 3 than in the opposite direction. Many agents migrate to level 3 to avoid the epidemic, but at the same time, asymptomatic agents in the exposed subgroup migrate and bring the epidemic to this level.

6. Conclusions

In this study, a multilevel modification of the susceptible–quarantine–exposed–infected–recovered–dead (SQEIRD) model has been proposed to describe the spread of a virus across three population clusters with quarantine measures and vaccination. The optimal control structure and feasible controls have been determined for a specific class of cost functions. The numerical simulations have demonstrated that compliance with quarantine requirements at the first level and vaccination at all levels can reduce the overall epidemic spread and system costs. The impact of different parameters on the basic reproduction number R0 in the modified system has been analyzed. In the future, specific cases of the epidemic spread in different countries using appropriate statistical datasets will be investigated. Additionally, the hierarchical SQEIRD model will be explored in complex networks with different topologies and using vaccination as a control strategy. Overall, this study provides insights for designing effective control strategies to mitigate the impact of epidemics.