Evolutionary-Game-Theory-Based Epidemiological Model for Prediction of Infections with Application to Demand Forecasting in Pharmaceutical Inventory Management Problems

Abstract

Pharmaceuticals play a critical role in the eradication of infectious diseases. Effective pharmaceutical inventory management is important for controlling epidemics since medical resources such as pharmaceuticals, medical staff, and hospitals are limited. In this study, a novel epidemiological model is proposed to evaluate the resource requirements for pharmaceuticals and is applied to analyze different pharmaceutical inventory management strategies. We formulate the relationship between the number of infected individuals and the risk of infection to account for virus mutation. Evolutionary game theory is integrated into an epidemiological model to represent human behavioral choices. The proposed model can be developed to forecast the demand for pharmaceuticals and analyze how human behavior affects the demand of pharmaceuticals. This study found that making people aware of the risk of disease has a positive impact on both reducing the number of infections and managing the pharmaceutical inventory. The main contribution of this study is to enhance areas of research in pharmaceutical inventory management. This study revealed that the correct recognition of the risk of disease leads to appropriate pharmaceutical management. There are a few studies on the application of infectious disease models to inventory control problems. This study provides clues toward proper pharmaceutical management.

1. Introduction

In December 2019, a novel coronavirus (COVID-19) broke out and caused a worldwide epidemic of infections. As of now, the cumulative number of deaths worldwide has exceeded 6.8 million [1]. The spread of the virus has caused challenges in many countries and regions, including the threatened collapse of healthcare systems and economic stagnation due to requests for voluntary furloughs and closings of offices. When an infectious disease outbreak occurs, pharmaceuticals play an extremely important role in bringing the epidemic to an end. Vaccination provides long-term herd immunity [2]. Since pharmaceutical resources are limited, they must be properly managed in order to effectively contain an epidemic [3]. The efficient management of pharmaceutical supply chains is essential to combat drug-treatable disease epidemics [4]. Shortages of medicines can have a major impact on epidemic control [4]. Therefore, an integrated model combining infectious diseases and the corresponding pharmaceutical supply chain is needed to understand the spread of diseases and their impact on the supply chain [5]. Therefore, in this study, we first propose a mathematical model of infectious diseases that takes into account the characteristics of infectious diseases. The model is then extended to an inventory management model to forecast the demand for pharmaceuticals.

Mathematical models of infectious diseases can describe and analyze infection dynamics. They also play a less obvious but important role in solving specific coordination problems and social dilemmas that arise during a pandemic [6]. The SIR model is the most basic model of infectious diseases. This model considers three states: susceptible, infected, and recovered. The SIR model describes changes in these states over time. The SEIR model is extended to include the incubation period state. Other models that take into account the effects of antibodies from vaccination, severity of symptoms, births, deaths, and age have also been studied [7,8,9,10]. Thus, by extending the model to take disease characteristics into account, it is possible to create a model that more closely resembles real-world situations.

Research has also been conducted to incorporate not only the characteristics of the disease but also the behavior and intentions of people into the model. Some studies have used game theory to take into account people’s fertility rates, to express whether they are vaccinated or not, or to consider two strategies: vaccination and social distance [11,12]. Studies considering the effect of lockdown policies on the number of infected people [13], the effect of wearing masks [14], and the effect of vaccination on infection rates [15,16] have been conducted using evolutionary game theory. Amaral et al. [17] used evolutionary game theory to express the relationship between human behavior and the prevalence of infectious diseases using the SIR model. Evolutionary game theory is one of the areas of game theory that studies organisms and human behavior based on the idea of biological evolution. The motivation is that, by using game theory, it is possible to express the probability that people change their strategy. This allows us to represent human behavior in a model of infectious diseases.

One of the characteristics of COVID-19 is a propensity for recurring cycles of population-wide infections. As shown in Figure 1, the number of infected people has exhibited undulating patterns of growth and decline. Although it is difficult to capture this pattern of infection using the SIR model or SEIR model, the application of evolutionary game theory offers a viable approach to depicting the undulating rise and fall in the number of infected persons [18]. In their study, agents choose whether to voluntarily refrain from going out to reduce the risk of infection and keep themselves quarantined at home or to behave as they would under normal circumstances. The agent decides which strategy to adopt, taking into account the risk of disease transmission and the various costs of self-isolation. This allows us to show the impact of self-restraint on the number of people infected during a COVID-19 epidemic. However, previous studies have used the SIR model, which does not take into account the incubation period of infectious disease. In addition, factors such as the effects of viral mutations and pharmaceuticals are not taken into account.

The incubation period is one of the most important factors in analyzing COVID-19 [19]. The mutability of the virus also has a significant impact on the prevalence of COVID-19. In this study, we extend the model of references [17] to the SEIR model to take into account the incubation period of the infection, and an evolutionary game model is integrated to express human behavioral choices. Then, we formulate the relationship between the number of infected persons and the risk of infection to account for virus mutation. Furthermore, the proposed infectious disease model is used to forecast the demand for pharmaceuticals and analyze how people’s behavioral choices affect pharmaceutical inventory management. Rather than a mandatory quarantine policy such as a lockdown, this study deals with a policy that encourages people to refrain voluntarily. We investgate the impact of this policy on the number of infected people and the pharmaceutical inventory. Numerical experiments confirm that, by increasing the infection rate as the infection spreads, the maximum number of infected persons in each infection peak continues to increase until the maximum number of simultaneous infections is reached. This reveals that the higher infection rate caused by the mutation of the virus affects the number of infected persons. The impact of human behavioral choices on the inventory of pharmaceuticals is then clarified. The purpose of this study is to analyze how people’s decisions affect the spread of infectious diseases and the management of pharmaceutical inventories.

This study makes several contributions to the current literature. First, we clarified the effects of human behavior and viral mutations on infectious epidemics. In particular, there have been few studies of infectious disease models that take viral mutations into account. This is useful in considering countermeasures to reduce epidemics. Second, the study revealed that correct recognition of the risk of disease leads to appropriate pharmaceutical management. Few studies have been conducted on the application of infectious disease models to inventory control problems. This is a clue to proper pharmaceutical management.

The paper is organized as follows: Section 1 describes the background and objectives of the study, and Section 2 reviews related works. Section 3 explains the mathematical model of infectious diseases and evolutionary game theory. Section 4 describes the model proposed in this study. In Section 5, numerical experiments are conducted using the model presented in Section 4, and the results are presented. Section 6 presents a discussion of the experimental results, and the findings are presented. Section 7 presents conclusions and future work.

2. Literature Review

In this section, we review the literature on infectious disease models and pharmaceutical inventory management. After the expansion of COVID-19, research on mathematical models of infectious diseases capable of describing and analyzing infection dynamics has obtained considerable attention. In this study, the SEIR model is used to describe the infection epidemic; the SEIR model is extended to incorporate some real-world features. Since many such studies have already been conducted, it is important to review the area of infectious disease modeling in order to conduct this study. Furthermore, the infectious disease model is extended by integrating an inventory management problem in this study. The review of the area of pharmaceutical inventory management is important for the formulation of the inventory management model in this study. Therefore, we conduct a literature review of these two areas.

We first review the literature on infectious disease models. Lu et al. [20] proposed a new two-stage epidemic model with a dynamic control strategy to describe the spread of COVID-19. They analyzed appropriate control strategies to minimize control costs and the normal operation of society is ensured. This study did not consider virus mutations, nor the impact of pharmaceuticals. In addition, their study has not dealt with people’s behavior. Kabir et al. [13] proposed a model that combines the compartmental epidemiology model with the concept of behavioral dynamics in evolutionary game theory. Numerical analysis shows that emergency aid funds disbursed by the government are effective in reducing the duration and overall economic costs of a pandemic. This paper did not consider the impact of viral mutations or pharmaceuticals. Our study focuses more on the impact of pharmaceuticals.

Annas et al. [9] involved stability analysis and numerical simulation of the SEIR model for the spread of COVID-19. The SEIR model with vaccination and sequestration factors as model parameters was used for model construction, and the generation matrix method was used for model analysis to obtain the basic reproduction number and the global stability of the COVID-19 distribution model. The simulation results show that the vaccine promotes the cure of COVID-19 and that maximum quarantine can delay the spread of COVID-19. This study did not address viral variation and focused solely on a single infection cycle.

Kabir et al. [14] proposed a new intervention game model that combines a mathematical model of epidemiology with evolutionary game theory. The model mainly analyzes the effect of mask-wearing or not. Numerical results reveal that behind the mask-wearing dilemma lies a diverse and rich social dilemma structure. Their study failed to take into account viral mutations. It also considered only a single infection cycle. While their study focused on the human behavior of wearing masks, our study also focuses on the effects of pharmaceuticals. Gosak et al. [21] used game theory to formulate the interaction of spontaneous social distancing in a partially infected population. Their study also did not consider pharmaceuticals. However, it provides a meaningful method for describing the dynamics of infection epidemics.

Amaral et al. [17] used evolutionary game theory to propose an epidemiological SIR model that integrates social strategies, individual risk perception, and virus spread into a single process. While the final scale of infection is determined primarily by the infection rate, risk perception is shown to be fundamental in controlling the magnitude of the infection peak. However, this study did not take into account factors such as the incubation period of the infection or viral mutations, nor did it take into account pharmaceuticals such as vaccines.

Agusto et al. [22] built a mathematical model of COVID-19 infection with isolation and hospitalization combined with a dynamic game model of human adaptive behavior. Susceptible and infected individuals adopt different behavioral strategies based on their perceptions of disease prevalence and burden, and their susceptibility to isolation measures, and evolve their strategies using social learning algorithms. The study concludes that incentives for altruistic behaviors, such as voluntary self-isolation by infected individuals, are necessary to reduce the social burden of pandemics. This study also did not take into account the impact of pharmaceuticals. Our study also focuses on people’s behavior.

Then, we review the literature that combines infectious disease and inventory management models. Liu et al. [23] proposed a unique time-varying forecasting model for the dynamic demand for medical resources based on the SEIR model. The entire medical resource allocation process is structured as a multi-stage integer programming problem. At each stage, a cost minimization subproblem is solved subject to the time-varying demand. The corresponding optimal allocation results are then used as inputs to a process that controls the spread of influenza and determines the demand for the next stage. This study demonstrates that the subproblem may help decision-makers prepare for a pandemic, including how to dynamically allocate limited resources. Since this study is about influenza, we propose a model that takes into account the characteristics of COVID-19. We also focus on the influence of human behavior.

Liu et al. [24] proposed a dynamic logistics model for healthcare resource allocation that can be used to control epidemic spread. The model combines a predictive mechanism constructed for the demand for medicines during the course of epidemic spread with a logistics planning system to meet the predicted demand and minimize the total cost. The study did not consider characteristics of COVID-19, such as viral mutations, or human behavior. Shamsi Gamchi et al. [25] addressed a novel dual-purpose vehicle routing problem for distributing vaccines among different regions to control the spread of infectious diseases. The model developed aims to minimize the social costs incurred by considering different priority groups under the SIR epidemic model and the cost of vehicles used simultaneously. Viral mutations were not taken into account in this study, nor did it take into account the effects of actions taken by individuals.

Paul et al. [26] proposed a generic framework for pharmaceutical inventory management, consisting of hospital, forecasting, and inventory management modules. The disease spread forecasting model is shown to be superior to naïve policies in terms of mitigating infections and saving inventory. The authors then analyzed the quantification of the impact of important factors such as the selection of appropriate ordering policies, safety stockpiles, and epidemic declaration thresholds on epidemic dynamics. The study did not consider virus mutations or human behavior. Rehman et al. [27] examined various inventory policies such as reorder points, recurring orders, and just-in-time to minimize inventory management costs for medical supplies. They used an infectious disease model to determine the dynamics of COVID-19 and the percentage of infected patients admitted to the hospital. Based on this information, various situations are created that take into account lockdown, social awareness, etc., and appropriate inventory policies are proposed to reduce inventory management costs. Viral mutations were not considered in this study. Our study also uses evolutionary games to represent human behavior.

A large number of literature surveys were conducted, indicating that studies have been conducted from a variety of perspectives. Among the reviewed studies, the study on infectious disease models was very helpful in developing a method to represent human behavior in the models. The study on pharmaceutical inventory management was very helpful in developing the infectious disease model into an inventory management method. The relevant literature reviewed is summarized in

Table 1. Review of relevant literature.

Authors	I	V	M	P	E
Lu, X. et al. [20]	✓		✓
Kabir, K. M. A. et al. [14]	✓			✓
Amaral, M. A. et al. [17]			✓	✓
Liu, M. et al. [24]	✓				✓
Shamsi, G. N. et al. [25]	✓				✓
Paul, S. et al. [26]	✓		✓		✓
Agusto et al. [22]	✓		✓	✓
Our Study	✓	✓	✓	✓	✓

I: incubation period, V: viral mutation, M: multiple-infection cycle, P: people’s behavior, E: effects of pharmaceuticals.

. As shown in this table, only a limited number of studies on infectious disease models consider the impact of pharmaceuticals. Also, few studies on inventory control models consider human behavior and analyze its impact. In addition, there are few studies that integrate infectious disease models and inventory control models, and we believe that more research in this area would be useful.

3. Epidemiological Model and Evolutionary Game Theory

In Section 3.1 and Section 3.2, we explain the basic concepts of the epidemiological model and evolutionary game theory, which are necessary to conduct this research. In Section 3.3, the work of Amaral et al. [9] that used evolutionary games to express and analyze the relationship between human behavior and infectious disease epidemics in a mathematical model is also explained. The reason for dealing with these models is that the infectious disease model can describe temporal changes such as the number of infected persons, and evolutionary game theory can represent human behavior.

3.1. Epidemiological Model

The epidemiological model was formulated by Kelmack and McKendrick in the UK in 1927 as a model used to describe a single infectious disease epidemic in a population with differential equations [28]. Subsequently, a mathematical review was conducted mainly by European researchers at the end of the 1970s, and the threshold theorem was established.

The first use of epidemiological models in Japan was for epidemic forecasting and prevention policy during the AIDS epidemic in the late 1980s. Since many mathematical scientists entered the field at that time, research on epidemiological models has made great progress. Even after the AIDS pandemic, epidemics of infectious diseases such as BSE, SARS, and swine influenza occurred in the 2000s, but the policy implementation of using an epidemiological model has not progressed. The reason for this is that research on epidemiological models in mathematical biology has mainly focused on qualitative mathematical analysis, with little practical research being conducted.

With the COVID-19 epidemic, data analysis and forecasting using epidemiological models were conducted in real-time. The results had a significant impact on government policy and were a milestone in the fight against infectious diseases in Japan. This led to the rapid spread of the basic concepts of the susceptible, infectious, recovered (SIR) model and the basic reproduction number.

3.1.1. SIR Model

The SIR model is the most basic epidemiological model. The model describes changes in three quantities over time, denoted by $S$, $I$, and $R$ respectively, where $S$ represents the number of susceptible individuals, $I$ represents the number of infected individuals, and $R$ represents the number of immune individuals. The SIR model is given by the ordinary differential equations in system (1). Figure 2 shows a model diagram that describes the relationship between each state.

$$\left\{\begin{array}{c}\frac{dS}{dt}=-\beta SI\\ \frac{dI}{dt}=\beta SI-\gamma I\\ \frac{dR}{dt}=\gamma I\end{array}\right.$$

(1)

Each parameter is explained as follows: $\beta$ is the effective contact rate, also called the infection rate, $\gamma$ is the recovery rate, and $1/\gamma$ is the average duration of infection. The SIR model assumes $S^{\prime }+I^{\prime }+R^{\prime }=0$, which means that the number of individuals does not change due to births or deaths. The model does not take into account the possibility that a once-infected person may become infected again.

3.1.2. SEIR Model

The susceptible, exposed, infectious, recovered (SEIR) model is an extension of the SIR model to account for the incubation period of infectious diseases. It is expressed by adding $E$, which represents the number of persons during the incubation period. The flow of the disease is as follows: susceptibility, exposure, infectiousness, and recovery. The SEIR model is given by the ordinary differential equations in system (2). Figure 3 shows a model diagram that describes the relationship between each state.

$$\left\{\begin{array}{c}\begin{array}{c}\frac{dS}{dt}=-\beta SI\\ \frac{dE}{dt}=\beta SI-\epsilon E\\ \frac{dI}{dt}=\epsilon E-\gamma I\end{array}\\ \frac{dR}{dt}=\gamma I\end{array}\right.$$

(2)

As in the SIR model, $\beta$ and $\gamma$ denote the infection and recovery rates. In addition, $\epsilon$ denotes the incidence rate and $1/\epsilon$ denotes the number of days of the incubation period. The SEIR model does not take into account the possibility of the re-infection of once-infected persons.

The infectious disease model can describe temporal changes in the number of infections. These concepts of infectious disease models are the basis of the models in this study, and by developing these basic models, the characteristics of various infectious diseases can be taken into account. Most of the studies of infectious disease models discussed in the literature review are developments of this most basic model.

3.2. Evolutionary Game Theory

Evolutionary game theory is an area of game theory that studies the behavior of organisms and humans based on the idea of biological evolution. The evolutionary games are the dynamics of organisms and human behavior by focusing on the Nash equilibrium point as a group equilibrium. The concepts of evolutionary game theory are essential in this study to describe the human behavior of voluntary segregation.

3.2.1. Evolutionarily Stable Strategy

In this study, evolutionary game theory is used to model the human behavior in infectious disease epidemics. Evolutionarily stable strategies, a very important concept in evolutionary game theory, are discussed in this section.

In the basic idea of biological evolution, the number of individuals with high adaptability proliferates and the number of individuals with low adaptability decreases [29]. The distribution of individuals with various characteristics within a population change over time. In the study of populations, it is important to investigate the distribution of individuals that can realize a steady state in the long term. The steady state of a population is a state in which the distribution of individuals does not change. Natural selection is a concept in biological evolution. This is the idea that the population of highly adapted individuals increases, and the population of less adapted individuals decreases. An important question in biological evolution is how the distribution of a population changes when different types of individuals invade the population. A new individual invading a population is called a mutation. A steady state is said to be evolutionarily stable when the state is maintained without proliferation of mutations even when a small number of mutations invade the population.

Then, we discuss evolutionarily stable strategies. Consider a steady state where all the individuals in a population chooses strategy $s$. Suppose that a mutation that selects a different strategy $t$ with a small ratio $\epsilon >0$ invades the population. Such a population can be denoted as $\left(1-\epsilon \right)s+\epsilon t$. Figure 3 shows the invasion of strategy $t$ into the population of strategy $s$. The mutation that selects a different strategy $t$ with a ratio $\epsilon >0$ is called a mutation of strategy $s$.

The conditions for strategy $s$ to be evolutionarily stable can be expressed in the following equation. Let $u(s,\left(1-\epsilon \right)s+\epsilon t)$ be the payoff of adaptation of strategy $s$ in the population $\left(1-\epsilon \right)s+\epsilon t$, and $u(t,\left(1-\epsilon \right)s+\epsilon t)$ be the payoff of adaptation of strategy t. The condition under which strategy $s$ is more adaptive than mutation $t$ is expressed in Equation (3). Figure 4 shows the invasion of mutations into the population.

$$u\left(s,\left(1-\epsilon \right)s+\epsilon t\right)>u\left(t,\left(1-\epsilon \right)s+\epsilon t\right)$$

(3)

When this condition is satisfied, strategy $s$ is an evolutionarily stable strategy.

3.2.2. Fermi Rule

Following the usual evolutionary game dynamics, whether agent $i$ changes to agent $j$’s strategy depends on the payoffs of both [17]. To express the probability that an agent switches its strategy, we employ the Fermi rule in this study. The Fermi rule expresses the probability $\Theta \left({\pi }_i,{\pi }_j\right)$ of agent $i$ changing to agent $j$’s strategy as in Equation (4), where ${\pi }_i$ and ${\pi }_j$ are the payoffs of agents $i$ and $j$, and k can measure the degree of noise.

$$\Theta \left({\pi }_i,{\pi }_j\right)=\frac{1}{1+e^{\raisebox{1ex}{$-\left({\pi }_j-{\pi }_i\right)$}\!\left/ \!\raisebox{-1ex}{$k$}\right.}}$$

(4)

3.3. Mathematical Model Expressing the Relationship between Human Behavior and Infectious Disease Outbreaks

Amaral et al. [17] used evolutionary game theory to represent human behavior in infectious disease models. In their study, agents choose whether to voluntarily isolate themselves to reduce the risk of infection or to behave as in a normal situation. These actions are denoted as strategy $Q$ (quarantine) and strategy $N$ (normal), respectively. In addition, it is assumed that the risk of infection is lower for strategy $Q$ than for strategy $N$. In their extended SIR model, susceptible individuals and infected indivuduals can choose one of the strategy. The model diagram is shown in Figure 5. The model can be expressed by equations in system (5).

$$\left\{\begin{array}{c}\begin{array}{c}{S}_N^{\prime }=-S_N\left({\beta }_N{I}_N+{\beta }_{\alpha }{I}_Q\right)+\tau {\phi }_S\\ S_Q^{\prime }=-S_Q\left({\beta }_{\alpha }{I}_N+{\beta }_Q{I}_Q\right)-\tau {\phi }_S\\ I_N^{\prime }=S_N\left({\beta }_N{I}_N+{\beta }_{\alpha }{I}_Q\right)-\gamma I_n+\tau {\phi }_I\end{array}\\ I_Q^{\prime }=S_Q\left({\beta }_{\alpha }{I}_N+{\beta }_Q{I}_Q\right)-\gamma I_Q-\tau {\phi }_I\\ R^{\prime }=\gamma \left(I_N+I_Q\right)\end{array}\right.$$

(5)

In system (5), we denote the infection rate of $S_Q$ as ${\beta }_Q$ and that of $S_N$ as ${\beta }_N$. In this model, interaction between the two strategies may occur, such as when $S_Q$ and $I_N$ are infected by contact. The infection rate in such a case is denoted by ${\beta }_{\alpha }$. The magnitude of each infection rate is ${\beta }_Q<{\beta }_{\alpha }<{\beta }_N$.

${\phi }_S$, ${\phi }_I$ denote the strategy transition rate in each state as expressed in Equations (6) and (7). The parameter $\tau$ represents how quickly the new strategy is adopted with respect to the timescale of the infection epidemic.

$${\phi }_S=S_Q\left(S_N+I_N\right)\Theta \left({\pi }_Q,{\pi }_N\right)-S_N\left(S_Q+I_Q\right)\Theta \left({\pi }_N,{\pi }_Q\right)$$

(6)

$${\phi }_I=I_Q\left(S_N+I_N\right)\Theta \left({\pi }_Q,{\pi }_N\right)-I_N\left(S_Q+I_Q\right)\Theta \left({\pi }_N,{\pi }_Q\right)$$

(7)

where ${\pi }_Q, {\pi }_N$ denote the payoffs of each strategy. $\Theta \left({\pi }_Q,{\pi }_N\right)$ represents the probability of changing from strategy $Q$ to strategy $N$, and $\Theta \left({\pi }_N,{\pi }_Q\right)$ represents the probability of changing from strategy $N$ to strategy $Q$.

4. Problem Description and Model Formulation

This section describes the proposed model. Section 4.1 provides the problem description. In Section 4.2.1, we extend the model of Amaral et al. [17] presented in Section 3.3 and propose a model that takes into account the characteristics of COVID-19 infection, which have not been considered in previous studies. Section 4.2.2 proposes a model that considers the severity of symptoms and adds a hospital component in addition to the incubation period and viral mutation. Section 4.2.3 proposes an inventory management model that forecasts demand using the proposed infectious disease model.

4.1. Problem Description

During the COVID-19 epidemic, many people mitigate the risk of infection by refraining from going out and minimizing contact with others. However, there are also people who do not refrain from going out and behaving as usual. The probability of contracting COVID-19 infection differs greatly between those who refrain from going out and stay at home and those who behave as usual [17]. This study integrates the individuals’ behaviors into the epidemic model.

Also, we take into account that new mutant strains with high infection rates are emerging. The number of infected persons in Japan is higher in the second and subsequent waves than in the first wave. This is due to the fact that the infection rate is increasing as the virus mutates. The infection rate is assumed to increase as the epidemic progresses [30].

In the proposed model, factors such as population increase or decrease due to births and deaths, inflows of people from overseas, and outflows of people overseas are not considered. The model also does not take into account the changes in infection rates due to vaccination status.

4.2. Model Formulation

Section 4.2.1 presents the SEIR model with human behavior. This model is also used to forecast the demand for pharmaceuticals. In Section 4.2.2, we present a model that takes into account hospital care, and in Section 4.2.3, a pharmaceutical inventory management model is developed.

4.2.1. SEIR Model with Incubation Period and Viral Mutation

There is an incubation period before symptoms develop after infection with COVID-19. In addition, the number of cases in Japan shows that the scale of the infectious epidemic is gradually increasing. This is mainly due to the effect of virus mutation [30]. In this section, we propose the model to examine the effect of the incubation period of COVID-19 infection and the increase in the infection rate due to virus mutation on the number of infected persons. The model proposed in this section consists of four states: susceptible $\left(S\right)$, infected $\left(I\right)$, incubation period $\left(E\right)$, and recovered $\left(R\right)$. In states $S$, $E$, and $I$, agents have two strategies: (1) voluntarily refraining from going out and staying at home (strategy $Q$), and (2) acting as they would under normal circumstances (strategy $N$). The former strategy have lower probability of infection than the latter. Let $S_Q$ denote the agent that takes strategy $Q$ in state $S$ and $S_N$ denote the agent that takes strategy $N$. Similarly, in states $E$ and $I$, we denote by $E_Q$, $E_N$, $I_Q$, and $I_N$, respectively. A diagram of the model is shown in Figure 6.

Here, the infection rate of $S_Q$ is represented by ${\beta }_Q$ and the infection rate of $S_N$ is denoted by ${\beta }_N$. In this model, it can be assumed that ${\beta }_Q<{\beta }_N$ because the risk of infection is smaller when voluntary isolation is used [17]. In rare cases, the interaction between the two strategies may occur, such as when $S_Q$ and $I_N$ come into contact with each other. The infection rate in such a case is denoted by ${\beta }_a$, where ${\beta }_Q<{\beta }_a<{\beta }_N$. Since ${\beta }_Q$ and ${\beta }_N$ are considered to increase with the prevalence of infection, they are expressed as in Equations (8) and (9), respectively. ${\beta }_{Q0}$ and ${\beta }_{N0}$ represent the initial values of ${\beta }_Q$ and ${\beta }_N$.

$${\beta }_Q={\beta }_{Q0}+a\left(I_Q+I_N+R\right)$$

(8)

$${\beta }_N={\beta }_{N0}+b\left(I_Q+I_N+R\right)$$

(9)

For simplicity, ${\beta }_a$ is expressed as a weighted average of ${\beta }_Q$ and ${\beta }_N$ using the parameter $c$ with $0<c<1$ as in Equation (10).

$${\beta }_a=\frac{c\left({\beta }_Q+{\beta }_N\right)}{2}$$

(10)

Since the incubation period of the disease is the same regardless of the strategy, the incidence rate for both $E_Q$ and $E_N$ is $\epsilon$. The recovery rate is also $\gamma$ for both $I_Q$ and $I_N$. ${\phi }_S$, ${\phi }_E$, and ${\phi }_I$ denote the rate of change in strategy in each state.

The payoffs for each strategy are shown in Equations (11) and (12).

$${\pi }_Q=-\Omega$$

(11)

$${\pi }_N=-\delta {\beta }_{N}I$$

(12)

where $\Omega$ represents the cost over the isolation period and $\delta$ is the risk of the disease as perceived by the individual.

The probability of whether agent $i$ adopts agent $j’$s strategy using the Fermi rule in Equation (4) [31]. Using Equation (4), the strategy transition rates for each of the states $S$,$E$, and $I$ are expressed in Equations (13) to (15) as follows:

$${\phi }_S=S_Q\left(S_N+E_N+I_N\right)\Theta \left({\pi }_Q,{\pi }_N\right)-S_N\left(S_Q+E_Q+I_Q\right)\Theta \left({\pi }_N,{\pi }_Q\right)$$

(13)

$${\phi }_E=E_Q\left(S_N+E_N+I_N\right)\Theta \left({\pi }_Q,{\pi }_N\right)-E_N\left(S_Q+E_Q+I_Q\right)\Theta \left({\pi }_N,{\pi }_Q\right)$$

(14)

$${\phi }_I=I_Q\left(S_N+E_N+I_N\right)\Theta \left({\pi }_Q,{\pi }_N\right)-I_N\left(S_Q+E_Q+I_Q\right)\Theta \left({\pi }_N,{\pi }_Q\right)$$

(15)

The SEIR model with incubation period and viral mutation is shown in system (16) as follows:

$$\left\{\begin{array}{c}{S}_N^{\prime }=-S_N\left({\beta }_N{I}_N+{\beta }_{\alpha }{I}_Q\right)+\tau {\phi }_S\\ S_Q^{\prime }=-S_Q\left({\beta }_{\alpha }{I}_N+{\beta }_Q{I}_Q\right)-\tau {\phi }_S\\ E_N^{\prime }=S_N\left({\beta }_N{I}_N+{\beta }_{\alpha }{I}_Q\right)-\epsilon E_n+\tau {\phi }_E\\ E_Q^{\prime }=S_Q\left({\beta }_{\alpha }{I}_N+{\beta }_Q{I}_Q\right)-\epsilon E_Q-\tau {\phi }_E\\ I_N^{\prime }=\epsilon E_n-\gamma I_n+\tau {\phi }_I\\ I_Q^{\prime }=\epsilon E_Q-\gamma I_Q-\tau {\phi }_I\\ R^{\prime }=\gamma \left(I_n+I_Q\right)\end{array}\right.$$

(16)

where the parameter $\tau$ represents how quickly the new strategy is adopted in relation to the timescale of the infection epidemic.

The model (16) represents the relationship between susceptible agents, incubation period agents, infected agents, and recovered agents. In addition, ${\phi }_S$, ${\phi }_E$, and ${\phi }_I$ represent the change in the agent’s strategy due to the evolutionary game. This means that the agent’s behavior of self-isolation can be incorporated into the SEIR model, which represents an epidemic.

4.2.2. SEIR Model for Consideration of Hospital Care

When infected with COVID-19, patients with severe symptoms should be treated in a hospital. In Japan, the basic policy is that only those with severe symptoms should receive treatment at a hospital, while those with mild symptoms or asymptomatic patients should be cared for at home. However, there are not enough hospital beds in an epidemic of COVID-19 infection, and there are cases in which patients with severe illness cannot be treated in a hospital. We propose a model in this section to examine the level of symptoms at which infected patients should be admitted to hospitals so that those with severe illnesses can be treated in hospitals on a priority basis. The same parameters and conditions as in the model presented in Section 4.2.1 are omitted in this section. Agents choose their own strategies only for the set of states $S$ and $E$. We assume that there are two types of infected patients: those who are cared for in the hospital and those who are cared for at home. The person who recovers at home is denoted as state $I_m$. The person who recovers in the hospital is denoted by state $H$. The proposed model is shown in Figure 7.

The infection rates ${\beta }_Q$ and ${\beta }_N$ are expressed by Equations (8) and (9) in Section 4.2.1. Parameter $\alpha$ is a parameter that expresses how many people in the incubation period will be treated in the hospital when they develop the disease and become infected. The recovery rate for state $H$ is denoted by ${\gamma }_A$, and that for state $I_m$ by ${\gamma }_B$. The payoffs for each strategy are expressed in Equations (11) and (12).

Using Equation (4) in Section 3.2.1, the strategy transition rates for each state of $S$ and $E$ are expressed in Equations (17) and (18) as

$${\phi }_S=S_Q\left(S_N+E_N\right)\Theta \left({\pi }_Q,{\pi }_N\right)-S_N\left(S_Q+E_Q\right)\Theta \left({\pi }_N,{\pi }_Q\right)$$

(17)

$${\phi }_E=E_Q\left(S_N+E_N\right)\Theta \left({\pi }_Q,{\pi }_N\right)-E_N\left(S_Q+E_Q\right)\Theta \left({\pi }_N,{\pi }_Q\right)$$

(18)

The equations of the model are shown in system (19).

$$\left\{\begin{array}{c}{S}_N^{\prime }=-{{\beta }_{N}S}_N(H+I_m)+\tau {\phi }_S\\ S_Q^{\prime }=-{{\beta }_{Q}S}_Q(H+I_m)-\tau {\phi }_S\\ E_N^{\prime }={{\beta }_{N}S}_N(H+I_m)-\epsilon E_N+\tau {\phi }_E\\ E_Q^{\prime }={{\beta }_{Q}S}_Q(H+I_m)-\epsilon E_Q-\tau {\phi }_E\\ H^{\prime }=\alpha \epsilon {(E}_N+E_Q)-\gamma H\\ I_m^{\prime }=\epsilon {(1-\alpha )(E}_N+E_Q)-\gamma I_m\\ R^{\prime }={\gamma }_{A}H+{\gamma }_B{I}_m\end{array}\right.$$

(19)

The model (19) represents the relationship between susceptible, incubators, infected, and recovering agents. In addition, ${\phi }_S$ and ${\phi }_E$ represent the change in the agent’s strategy due to the evolutionary game. It can be said that the agent’s behavior of self-isolation is incorporated into the SEIR model, which represents the infection epidemic. In addition, by adopting state $H$ and parameter $\alpha$, the number of patients in the hospital, which cannot be represented by the ordinary SEIR model, can be expressed.

4.2.3. Pharmaceutical Inventory Management

This section describes a pharmaceutical inventory management approach that uses an infectious disease model to forecast demand. The proposed method uses a demand forecasting model and an inventory control model. The demand forecasting model uses the infectious disease model described in Section 4.2.2. A study by Paul et al. [26] on the inventory management of pharmaceuticals using an infectious disease model is presented. We develop the inventory management method of based on the study by Paul et al. [26]. In this study, an inventory management model that forecasts demand using the infectious disease model is been proposed.

The parameters and equations in Section 4.2.2 are used to forecast the demand for pharmaceuticals. The predicted demand for pharmaceuticals ($D_d$) in period $d$ can be expressed in Equation (20) using the number of people in the incubation period who develop symptoms and become infected ($\epsilon E_N+{\epsilon E}_Q$) and the unit of the number of pharmaceuticals required per patient ($n$).

$$D_d=n\epsilon {(E}_N+E_Q)$$

(20)

The order quantity ($OR$) is determined based on the forecast demand ($D_d$) as in Equation (21). The adjustment parameters $\alpha$ and $\beta$ are used to adjust for drug order discrepancies and inventory discrepancies.

$$OR=\max \left\{0,D_d+\alpha \left({DO}_d-{MO}_d\right)+\beta \left({DI}_d-{MI}_d\right)\right\}$$

(21)

The desired order quantity (${DO}_d$) is calculated as the product of the production lead time ($L$) and the expected demand ($D_d$). The desired inventory (${DI}_d$) is calculated based on the desired inventory rate ($DC$), the forecast demand ($D_d$), and the safety stock ($SS$). In this study, the safety stock is assumed to be a given value in order to simplify the inventory control method.

$${DO}_d=L\times D_d$$

(22)

$${DI}_d=DC\times D_d+SS$$

(23)

The quantity of medicines on order (${MO}_d$) and the quantity in stock (${MI}_d$) are expressed in Equations (24) and (25).

$${MO}_{d+1}={MO}_d+{OR}_d-\frac{{MO}_d}{L}$$

(24)

$${MI}_{d+1}={MI}_d+\frac{{MO}_d}{L}-C_d-P_d$$

(25)

where $C_d$ represents the number of pharmaceuticals used, expressed as in Equation (26) using the number of people who recovered in the SEIR model.

$$C_d=n\gamma (I_N+I_Q)$$

(26)

Since medicines have an expiration date, $p_d$ represents the quantity of medicines to be disposed of due to expiration. The shelf life of a drug is $T_s$, which is expressed by Equation (27). The proposed model is shown in Figure 8.

$$p_d=\frac{{MI}_d}{T_s}$$

(27)

5. Numerical Experiments

We solve the simultaneous differential equations of the model using the fourth-order Runge-Kutta method. We then show how the model behaves by examining the effects of varying the values of the parameter ${\beta }_{N0}$, which represents the infection rate of $S_N$ and the individual’s perceived risk of the disease, $\delta$, in the model presented in Section 4.2.1.

5.1. Experimental Results Using an Infectious Disease Model

We set the initial values of each parameter and each state of the SEIR model that takes into account the incubation period and viral mutation as shown in Section 4.2.1, as well as the initial values of each state as shown in

Table 2. Parameter values for the model in Section 4.2.1.

Parameter	Values
${\beta }_{Q0}$	1.0
a	1.0
b	5.0
c	0.1
$\epsilon$	1/7
$\gamma$	1.0
Ω	1.0
$\tau$	1.0
k	0.1

and

Table 3. Initial values for each state of the model in Section 4.2.1.

State	Initial Values
$S_0$	$0.99$
$E_0$	0
$I_0$	0.01
$R_0$	0
$Q_0$	0.5
$N_0$	0.5

. Refer to the references [8,9,17] for the setting of each parameter value.

We set the initial values of each parameter value and each state of the SEIR model considering hospital care shown in Section 4.2.2 as shown in

Table 4. Parameter values for the model in Section 4.2.2.

Parameter	Values
${\beta }_{Q0}$	1.0
${\beta }_{N0}$	15
a	1.0
b	5.0
$\epsilon$	1/7
${\gamma }_A$	1.0
${\gamma }_B$	1.0
Ω	1.0
$\tau$	1.0
k	0.1

and

Table 5. Initial values for each state of the model in Section 4.2.2.

State	Initial Values
$S_0$	$0.99$
$E_0$	0
$I_0$	0.01
$H_0$	0
$R_0$	0
$Q_0$	0.5
$N_0$	0.5

.

5.1.1. Results of SEIR Model with Incubation Period and Virus Mutation

First, the model is examined by setting ${\beta }_{N0}=15$ and $\delta =15$. Figure 9 shows the population rate over time. Figure 10 shows the population for two strategies, Figure 11 shows the number of people infected with each strategy, Figure 12 shows the total number of infected people, and Figure 13 shows the number of adopters for each strategy. As shown in Figure 11 and Figure 12, the number of infected people keep increasing and decreasing. The payoff of $Q$, the self-isolation strategy, becomes larger when the percentage of infected agents becomes large at the beginning. The first peak of $S_Q$ occurs when most agents begin to select self-isolate strategy. When most agents self-isolate, the infection rate remains low, and the total fraction of infected agents begins to decrease. However, as the number of infected agents I approaches zero, the payoff of strategy $N$, which acts as usual, begins to grow. Eventually, the payoff of strategy $Q$ becomes larger than the payoff of strategy $N$, so the number of agents that act normally without quarantine increases. This corresponds to a rapid increase in $S_N$ near the beginning of the second wave of infection. In addition, as the number of people who act freely without quarantine increases, a second infection peak will inevitably occur. As the number of infected people increases, more people change their strategies and self-isolate, and $S_N$ begins to decrease. This process is repeated many times.

5.1.2. Impact of Individual Perceived Disease Risk $\delta$ on the Model

We discuss the effect of $\delta$, a parameter that represents an individual’s perceived risk of disease, on the behavior of the model presented in Section 4.2.1. In Figure 14, Figure 15, Figure 16 and Figure 17, we show the number of people in each state when $\delta$ = 0, 10, 15, and 20, respectively. Looking at Figure 14, there is only one infection cycle; in Figure 15, two infection cycles occur, each with a smaller peak; in Figure 16 and Figure 17, more infection cycles occur, and the peak of each infection cycle is smaller. In summary, when the risk is small, agents are infected early and the maximum number of simultaneous infections is large. When the risk is high, agents self-isolate for relatively long periods of time, and multiple infection cycles are distributed with smaller infection peaks. This is a naturally occurring behavior when evolutionary game dynamics are taken into account. Figure 18 shows the behavior of the number of infected persons $I$ when varying $\delta$.

One of the key features of this model is the occurrence of infection waves during the epidemic period, when there is no possibility of re-infection. The higher the value of $\delta$, the more often these waves of infection occur, and the smaller the peak of each wave. The higher the risk perception of the disease, the more likely it is to disperse infections over a long period of time.

5.1.3. Results of the SEIR Model Considering Hospital Care

In this section, we set the probability of severe COVID-19 infection to 1.6%. Figure 19 shows the results when $\alpha =0.016$, and Figure 20 shows the number of infected patients. Figure 21 shows the number of patients in the hospital by setting $\alpha =0.016$, $\alpha =0.030$, $\alpha =0.050$, and $\alpha =0.10$, respectively. This model was created to verify how many beds are available by comparing the number of patients in hospitals with the number of beds available for COVID-19 infections in actual hospitals. However, at this stage, the actual number of infected patients in Japan and the number of infected patients shown in the model are so different, and therefore cannot be compared. Therefore, it is necessary to improve each parameter to represent the actual number of infected patients in the future.

5.2. Experiments with Inventory Control Models

This section presents experimental results using the inventory control model presented in Section 4.2.3. Each parameter value is set as shown in

Table 6. Parameter values for the model in Section 4.2.3.

Parameter	Values
$n$	1
L	7
DC	1
SS	0.1
$\epsilon$	1/7
$\gamma$	1.0
$T_s$	7

. Refer to reference [26] for the setting of each parameter value.

We set $\left(\alpha , \beta \right)=(1, 1)$ and $\delta =15$ as a baseline model. The results are shown in Figure 22.

5.2.1. Impact of Individual Perceived Disease Risk $\delta$ on the Model

In this study, we consider how people’s decisions affect the quantity of orders and inventory. Figure 23 and Figure 24 show the increase and decrease in order quantity and inventory quantity for $\delta =$ 4, 8, 12, 16, and $20$, respectively. Figure 24 shows that the inventory sometimes decreases significantly when $\delta$ is small compared to when $\delta$ is large. Since it is ideal to have a certain level of inventory, the larger $\delta$ is the appropriate inventory level. Therefore, it can be said that making people aware of the risk of disease is also effective in the inventory control of pharmaceuticals.

5.2.2. Effect of Ordering Policy $(\alpha ,\beta )$ on Inventory Quantity

Ordering policies are very important in inventory management. In this section, we discuss the effect of the parameters $(\alpha , \beta )$ representing the ordering policy on inventory holdings. Figure 25 shows the inventory quantity calculated for five combinations of the adjustment parameters $(\alpha , \beta )$ of the ordering policy. When $(\alpha , \beta )$ = $(0.5, 0.5)$, the inventory quantity sometimes shows negative values. It indicates that the inventory is out of stock. When $(\alpha , \beta )$ = $(1, 0.5)$, the inventory quantity is relatively low, indicating a possibility of running out of stock. When $(\alpha , \beta )$ = $(1, 1$), $(\alpha , \beta )$ = $(0.5, 1)$, and $(\alpha , \beta )$ = $(2, 2)$, there is some room in the inventory. This result shows that when $\alpha$ is small, a large quantity of final inventory is left over; a small value of $\beta$ will result in less excess inventory, but may cause stockout.

6. Discussion and Findings

Discussion and findings on the experimental results are presented in this section.

6.1. Discussion of Experiments with Infectious Disease Models

The model described in the first experiment, which extends previous studies and takes into account the incubation period of the infectious disease and viral mutations, shows in Figure 18 the response of individuals to the infectious level of the disease. As a result, secondary infections occur or infectious diseases start to re-emerge after the majority of the population begins to act freely without quarantine. The main parameter examined in this study is $\delta$, which expresses the extent to which the population is aware of the individual costs of infection. This parameter has little effect on the final number of infections but has the greatest effect on the number of infections at the peak of the epidemic. For higher values of the parameter $\delta$, the number of people infected at the peak of infection is lower, but multiple peaks of infection occur. The occurrence of multiple peaks of infection overlaps with the actual increase and decrease in the number of infected people in Japan. In the current crisis of infectious disease epidemics, it is extremely important to reduce the number of people infected during the peak period in order to prevent the collapse of the healthcare system. Therefore, it is effective to spread awareness of the risk of disease, which can spread the increase in the number of infected patients over a long period of time and reduce the number of infected patients at the peak of infection.

The second experiment was conducted to verify the number of hospital beds available by comparing the number of hospital beds for COVID-19 infection with the number of patients in the actual hospitals. However, the number of infected patients shown by the model differs greatly from the actual number of infected patients in Japan. Therefore, it is necessary to modify the parameters of the model to make it comparable with the actual data.

6.2. Discussion of Experiments with Inventory Control Models

The experiment in Section 5.2.1 shows the effect of people’s decisions on the inventory of pharmaceuticals. Regardless of people’s perception of the risk of disease, the quantity of inventory increases or decreases as the number of infected people increases or decreases, because more orders are placed when the number of infected people increases or decreases. Excess inventory is risky because medicines have expiration dates. Therefore, it is effective to spread awareness of the risk of the disease over a long period of time to reduce the number of infected people at the time of peak infection, and, at the same time, to manage the inventory of pharmaceuticals appropriately.

The experiment in Section 5.2.2 shows the effect of the ordering policy on the inventory of pharmaceuticals. It shows how the parameters $(\alpha , \beta )$, which represent the ordering policy, affect the inventory. Future work is to optimize $(\alpha , \beta )$ and to study the optimal ordering policy.

6.3. Findings

The findings from this study show that making people aware of the risk of disease has a positive impact on both reducing the number of people infected and managing the inventory of pharmaceuticals. Specifically, the number of people infected can be dispersed and the number of people infected at peak times can be controlled. As a result, a rapid increase in the demand for pharmaceuticals does not occur, and the risk of holding excess inventory is reduced. The risk of holding excess inventory is particularly high because pharmaceuticals have expiration dates. Therefore, it is very important to make people correctly aware of the risk of disease and the cost of their actions in the inventory management of pharmaceuticals.

7. Conclusions

Two processes as complex as social behavior and disease epidemics are usually analyzed separately for simplicity. In order to explain two complex processes and to reveal the essential elements of their dynamics and interactions, it is important to create a model that is simple and captures the observed phenomenon as much as possible. In this study, two infection models are proposed. First, we proposed the SEIR model, which extends previous studies and takes into account the incubation period of infectious diseases and viral mutations. Second, we proposed a new SEIR model that takes into account hospital care. The proposed model is used to forecast demand and is adapted to an inventory management problem. The results of the operation of these models are presented and the effects of changing the parameters are examined. The results show that people’s perception of the risk of disease has a significant impact on the number of infections and the inventory of medicines. We also found that an increase in the infection rate due to viral mutations has an impact on the spread of the epidemic.

We found that the effectiveness of the policy of encouraging voluntary isolation considered in this study depends on people’s perceived risk of disease. Therefore, the usefulness of this policy is enhanced when the risk of disease is perceived as accurately as possible by the population.

Several future directions need to be noted regarding the present study. In this study, human behavior, hospitals, and other factors were incorporated into the SEIR model. In the future, parameter estimation based on real data will be important for actual forecasting. Therefore, future work is to improve the model and perform parameter estimation using real data. The other future works are to compare the model’s results with the actual number of infected people and the number of people receiving medical care, and to study the degree to which people with symptoms should be treated in a hospital. In addition, a more detailed analysis of the impact of ordering policy is needed in its adaptation to the inventory control problem.