Cost Evaluation for Capacity Planning Based on Patients’ Pathways via Semi-Markov Reward Modelling

Abstract

In the present paper, we develop a non-homogeneous semi-Markov reward model, deriving expressions for a healthcare system’s expected structure along with the expected costs generated by medical services and patients’ holding times in the states. We provide a novel definition and investigation for states’ availability, which is critical for capacity planning based on service demand in an environment of limited resources. The study is based on patients’ mobility through hospital care, where each patient spends an amount of time in every state of the hospital (emergency room, short-term acute care, hospitalization, surgery room, and intensive care unit). Multiple outcomes, such as discharge or death, can also be taken into account. We envisage a situation where any discharges are immediately replaced by a number of new admissions that carry on the pathways of the patients who exit. By assuming an expanding system, the new idea of states’ inflows is considered due to new patients who create pathways through hospital care, along with internal entrances. The theoretical results are illustrated numerically with simulated hospital data informed by aggregated public data of the Greek public health sector. The framework can be used for both strategic planning and cost evaluation purposes for hospital resources.

1. Introduction

Systems appear in every discipline and deterministic or stochastic mathematical modelling is applied for their study. For example, different population systems can be divided into a set of states, taking into account some of their basic characteristics, such as their socio-economic status, status or rank in a hierarchy system. Members of the system usually transition through these states in a probabilistic manner based on historical data of the visited states, as well as the duration of stay within each state. According to those parameters, a semi-Markov chain can be developed for the study of those systems [1]. One example of a mathematical model is the non-homogeneous semi-Markov model, that has been applied to numerous domains, such as manpower systems [2,3]. A broad and accessible overview of non-homogeneous Markov chains and systems can be found in [4]. Moreover, the addition of a reward process allows us to study the operational characteristics of a wide variety of other systems as well. The attachment of a reward structure in a semi-Markov model obviously increases the complexity of analysis but also provides thoroughness in modelling because in real-life, every action generates a reward, either positive or negative [5].

By using a Markov chain to study systems, we assume that the probability of transition from one category to another does not depend on the length of stay. Furthermore, the Markovian models assume that the holding times follow exponential or geometric distributions, in the continuous and discrete case, respectively. Thus, if the data do not fit well to the aforementioned distributions, the estimations of the sojourn times will turn out to be unreliable. Hence, semi-Markov models generally provide better goodness-of-fit, as they could incorporate arbitrary distributions for the holding times, allowing flexibility. This holds true especially in cases where the holding times actually depend on the next transition. On the other hand, semi-Markov chains require higher computational workload for calculating the recursive and analytic relationships for the underlying process due to the larger number of estimated parameters. On the contrary, time dependence is a desirable characteristic to be included in the process since it provides additional useful information. In this case, the transitions of such a system are not merely described by a typical Markov chain procedure, therefore semi-Markov models are considered as more rigorous stochastic tools that provide a framework accommodating a variety of applied probability models [6,7,8]. Semi-Markov processes have found applications in different domains, including manpower planning, financial credit risk, word sequencing and DNA analysis [9,10,11,12,13,14,15,16].

Other stochastic models have found application in the finance and healthcare sectors. Economic assessment of medical care often compares healthcare initiatives using cost models and offers solutions that result in both cost-effective methods for the healthcare provider and advantages for the patient. Since governments all over the world are constantly faced with rising medical costs and are consequently unable to meet the demands for greater resources, such approaches are especially important when it comes to providing information that facilitates the efficient and fair allocation of limited resources [17]. In these kinds of situations, a variety of modeling approaches have been used, mostly individual-based micro-economic models like decision trees or Markov models, or population- or cohort-based macro-economic models like regression [18,19,20].

Previous studies developed models of patient duration of stay incorporating the use of Bayesian belief networks with Coxian phase-type distributions for modelling the length of stay of a group of elderly patients in hospital [21]. Other results and applications of the phase type distributions include modelling the cost of treating stroke patients within a healthcare facility using a mixture of Coxian phase-type models with multiple absorbing states [18,21]. Furthermore, the moments of total costs have also been obtained for an individual assuming Poisson arrivals [22]. A previous study provided important information to health service managers and policy makers to help them identify sequential patterns which require attention for efficiently managing healthcare resources and developing effective healthcare management policies via non-homogeneous Markov models [23].

Also, theoretical results for the moments and distribution of a semi-Markov cost model with discounting have been provided in analytic form for an open healthcare system [24], a Markov reward model for a healthcare system with a constant size, and in addition, with fixed growth which declines to zero as time tends to infinity [25]. The same researchers also presented a Markov reward model for a healthcare system with Poisson admissions where expressions for the distribution, the mean and variances of costs are derived [26]. A Markov model was used to describe the movements of geriatric patients within a hospital system, where the spend-down cost of running down services is estimated given that there are no more admissions and different costs assigned to states [27]. Finally, a census approach to model bed occupancy for geriatric patients by the implementation of a stochastic compartmental Markov model has been developed [28].

It has also been suggested that using Markov models for economic evaluation in healthcare sectors is an intuitive approach to handle outcomes and costs concurrently. Markov models can be easily modified in a variety of ways to expand beyond their limits, despite being criticized for their narrow assumptions. For instance, within an analytical framework like a semi-Markov model, the fundamental presumptions of the Markov model, such as the Markov property, can be modified. This method is very adaptable and can accurately capture the complexity and variety that are frequently present in diverse healthcare systems. The expected population structure of the healthcare system and the evaluation of costs generated by the hospital services of every kind (treatments, surgeries, medical care, etc.) are parameters of great importance, and hospital managers would benefit immensely if they had advanced knowledge of patients’ duration of stay and the corresponding derived costs [19,21,22,23,24,25,26,27,28,29,30,31].

In the present study, a non-homogeneous semi-Markov reward model is considered where rewards are random variables associated with state occupancies and transitions. The novel part of the current paper is derived from the inclusion of states’ inflows and availability, which is critical for capacity planning based on services demand in an environment of fixed resources. In Section 2, the theory of the model is provided and results related to the population’s structure and states’ inflows are given and expressions related to state’s current availability follow. Also, expressions for the expected costs generated by the system and corresponding to patients’ paths are developed. Finally, in Section 3, most of the theoretical results are illustrated numerically with simulated hospital data informed from aggregated public data of the Greek public health sector. Finally, conclusions and suggestions for further research are provided in Section 4.

2. Methods

2.1. Population Structure

Let us consider a population which is stratified into a set of states $S=\left\{\mathrm{1,2},3,\dots ,k\right\}$ according to various characteristics. The states are assumed to be exclusive and exhaustive, so that each member of the system may be in one and only one state at any given time. The population structure of the system at time $t$ is described by a vector $\mathit{N}\left(t\right)=\left[N_1\left(t\right),\dots ,N_k\left(t\right)\right]$, where $N_i\left(t\right)$ is the expected number of members of the system in state $i$ at time $t$. Let $T\left(t\right)$ be the expected number of members of the system at time $t$.

In the present paper, time is considered to be a discrete parameter and we assume that the individual transitions between the states occur according to a non-homogeneous semi-Markov chain (embedded non-homogeneous semi-Markov chain). The embedded non-homogeneous semi-Markov chain in the system defines the stochastic process which describes the movements of every patient through the healthcare system. Thus, the patients’ pathways, which are made up from transitions and successive holding times between the states, are governed by the sequence of the transition probability matrices of the Markov chain ${\left\{\mathit{P}\left(t\right)\right\}}_{t=0}^{\infty }$ and the sequence of the holding time mass function matrices ${\left\{\mathit{H}\left(m\right)\right\}}_{m=0}^{\infty }$.

Moreover, we assume that the system is open and the population is expanding, i.e., $\Delta T\left(t\right)=T\left(t\right)-T\left(t-1\right)>0$, so at every time unit three kinds of movements can occur: internal transitions, new entrances and exits. In this respect, let us denote by ${\left\{\mathit{F}\left(t\right)\right\}}_{t=0}^{\mathrm{\infty }}$, the sequence of substochastic matrices where $\mathit{F}\left(t\right)=\{{f_{ij}\left(t\right)\}}_{i,j\in \mathit{S}}$, where $f_{ij}\left(t\right)$ define the transition probabilities between the states which are controlled by the entrance time $t$ to state $i$. Let also ${\mathit{p}}_{k+1}\left(t\right)$ be the row vector of wastage, whose $i$th element is the probability of leaving the system from $i$, given that the entrance in state $i$ occurred at time $t$ and ${\mathit{p}}_0\left(t\right)$ be the column vector of replacements, whose $j$-th element is the probability of entering the system in state $j$ as a replacement of a member who entered his last state at time $t$. We consider that every member in the system holds a specific position (i.e., post in a company, bed in a hospital) called “membership” [32] and initially there are $T\left(0\right)$ memberships in the system. Thus, a member entering the system creates a particular membership which moves within the states with the other members. When a member leaves, the membership is taken by a new recruit and moves within the system with the replacement, and so on. Denote by ${\left\{\mathit{P}\left(t\right)\right\}}_{t=0}^{\mathrm{\infty }}$, the sequence of stochastic matrices where $\mathit{P}\left(t\right)=\{{p_{ij}\left(t\right)\}}_{ij\in \mathit{S}}$, and $p_{ij}\left(t\right)=f_{ij}\left(t\right)+p_{i,k+1}\left(t\right)p_{oj}\left(t\right)$ define the transition probabilities for the memberships which equivalently define transitions between the states either with actual transitions of the members or by replacement of the leaving members by new recruits. The sequence of stochastic matrices $\mathit{Q}\left(n,t\right)={\left\{q_{ij}\left(n,t\right)\right\}}_{i,j\in S},$ which define the interval transition probabilities for the memberships of the embedded semi-Markov chain to the system, is described by the following recursive Equation [2]:

$$\mathit{Q}\left(n,s\right)={}^{>}\mathit{W}\left(n,s\right)+{\sum }_{m=1}^n\mathit{C}\left(s,m\right)\mathit{Q}\left(n-m,s+m\right)$$

(1)

where $\mathit{C}\left(s,m\right)$ = $\mathit{P}\left(s\right)⌑\mathit{H}\left(m\right)$ is the Hadamard product of the matrices $\mathit{P}\left(s\right)$ and $\mathit{H}\left(m\right)$, ${\left\{\mathit{H}\left(m\right)\right\}}_{m=0}^{\infty }$ is the holding time mass function matrix of the embedded semi-Markov chain, ^>$\mathit{W}\left(n,s\right)$ is a diagonal matrix with elements the survival functions of the holding times in the states. It can be easily proved that $\mathit{C}\left(s,m\right)$ the so-called Core matrix of the process, is the most important parameter of the imbedded semi-Markov chain since it combines the two stochastic processes imbedded in the system, the Markov process and the one of the holding times. The analytic solution of the recursive Equation (1) is proven in [2] as follows

$$\begin{gathered} \mathit{Q}\left(n,s\right)={}^{>}\mathit{W}\left(n,s\right)+\mathit{C}\left(s, n\right)+{\sum }_{j=2}^n\left[\mathit{C}\left(s,j-1\right)+{\sum }_{k=1}^{j-2}{\mathit{S}}_j\left(k,s,m_k\right)\right][{}^{>}\mathit{W}\left(n-j+1,s+j-1\right)+ \\ \mathit{C}\left(s+j-1, n-j+1\right)] \end{gathered}$$

(2)

where

$${\mathit{S}}_j\left(k,s,m_k\right)={\sum }_{m_k=2}^{j-k}{\sum }_{m_{k-1}=1+m_k}^{j-k+1}\dots {\sum }_{m_1=1+m_2}^{j-1}{\prod }_{r=-1}^{k-1}\mathit{C}\left(s+m_{k-r}-1,m_{k-r-1}-m_{k-r}\right)$$

for every $j\ge k+2$ while for every $j<k+2$, ${\mathit{S}}_j\left(k,s,m_k\right)=0$. Furthermore, the system’s expected population structure can be determined by the following equation just by the basic parameters of the system as follows [2]:

$$\mathit{N}\left(t\right)=\mathit{N}\left(0\right)\mathit{Q}\left(t,0\right)+{\sum }_{m=1}^t\Delta T\left(m\right){\mathit{r}}_0\left(m\right)\mathit{Q}\left(t-m,m\right),$$

(3)

where ${\mathit{r}}_0\left(m\right)$ is the recruitment vector of the system. Relation (3) completely determines the expected population structure of the system. The analytic solution for Equations (1) and (3), in relation to the basic parameters of the system, is provided in [2].

2.2. States’ Inflows

In some models (e.g., healthcare modelling, manpower planning) the number of members that enter a specific state either via recruitment without replacement or internal transitions at any given time is crucial information for decision analysis and capacity planning based on service demand in an environment of limited hospital resources. The aforementioned expected number of members can be described by a vector $\mathit{M}\left(t\right)=\left[M_1\left(t\right),\dots ,M_k\left(t\right)\right]$, where $M_i\left(t\right)$ is the expected number of the new recruits to state $i$ at time $t$ creating additional memberships to the state plus the memberships which enter state $i$ via internal transitions. Using probabilistic arguments, the above-defined expectations can be recursively defined by the following:

$$\mathit{M}\left(n\right)= \mathit{N}\left(0\right)\mathit{E}\left(n,0\right)+ {\sum }_{m=1}^n\Delta T\left(m\right){\mathit{r}}_0\left(m\right)\mathit{E}\left(n-m,m\right),$$

(4)

where $\mathit{M}\left(n\right)=[{{\rm M}}_1(n),\dots ,{{\rm M}}_k(n)]$ and $\mathit{E}\left(n,s\right)$ is the matrix where its elements equal to the entrance probabilities for the memberships for the interval $[s,s+n)$ [33].

2.3. States’ Current Availability

The difference $N_i\left(t\right)-M_i\left(t\right)$ is the mean number of memberships which entered state $i$ during the time interval $\left[0,t\right)$ and remain at that state at least until time $t$. Therefore, the above difference provides useful information related to the current availability of the state. Moreover, if we assume that $C_i$ defines state’s $i$ capacity then the difference $C_i{-[N}_i\left(t\right)-M_i\left(t\right)$] indicates the current availability of the state. Hence, the vector $\mathit{C}-[\mathit{N}\left(t\right)-\mathit{M}\left(t\right)$], where $\mathit{C}=\left[C_1,\dots ,C_k\right]$, provides estimations which are critical for capacity planning based on service demand in an environment of scarce resources and investigating optimal solutions.

2.4. Attachment of Costs

Define as $y_{ij}\left(t\right)$ the cost that a membership generates at time $t$ after entering state $i$ during the interval $\left[t,t+1\right)$ when its successor state is $j$ and $b_{ij}\left(m\right)$ as the cost generated by the membership’s transition from state $i$ to $j$, after holding time $m$ time units in state $i$. Also, define as $\mathit{V}\left(s,n,\beta \right)={\left[v_1\left(s,n,\beta \right),\dots ,v_k\left(s,n,\beta \right)\right]}^T$, the vector whose i-th element, $v_i\left(s,n,\beta \right),$ is the expected present value of cost that is generated by a membership during the time interval $[s,s+n)$ under the condition that the membership entered state i at time s and the discount factor equals β, $0<\beta <1$. This coefficient is usually calculated by $\beta =\frac{1}{1-r }$, where $r$ is the interest rate, when referring to economic rewards. It is known from [4] that:

$$\mathit{V}\left(s,n,\beta \right)={}^{>}\mathit{Y}\left(s,n,\beta \right)+{}^{>}\mathit{R}\left(s,n,\beta \right)+{\sum }_{m=1}^n\mathit{P}\left(s\right)⌑\mathit{H}\left(m\right){\beta }^{m+s}\mathit{V}\left(s+m,n-m,\beta \right),$$

(5)

where

$$\begin{gathered} {}^{>}\mathit{Y}\left(s,n,\beta \right)={\left[{}^{>}y_1\left(s,n,\beta \right),\dots ,{}^{>}y_k\left(s,n,\beta \right)\right]}^T, \\ {}^{>}y_i\left(s,n,\beta \right)={\sum }_{x=1}^k{\sum }_{m=n+1}^{\infty }{p}_{ix}{\left(s\right)h}_{ix}\left(m\right)y_{ix}\left(s,n,\beta \right), \\ {y}_{ix}\left(s,n,\beta \right)={\sum }_{\mu =0}^{n-1}{\beta }^{\mu +s}{y}_{ix}\left(\mu +s\right), \\ \mathit{R}\left(s,n,\beta \right)={\left[r_1\left(s,n,\beta \right),\dots ,r_k\left(s,n,\beta \right)\right]}^T, \\ {r}_i\left(s,n,\beta \right)={\sum }_{x=1}^k{\sum }_{m=1}^n{p}_{ix}\left(s\right)h_{ix}\left(m\right)\left[y_{ix}\left(s,m,\beta \right)+{\beta }^{m+s}{b}_{ix}\left(m\right)\right]. \end{gathered}$$

Furthermore, the analytic solution of Equation (5) can be derived if we follow the corresponding steps of a similar proof in [4]. The result is given below

$$\begin{gathered} \mathit{V}\left(s,n,\beta \right)={}^{>}\mathit{Y}\left(s,n,\beta \right)+{}^{>}\mathit{R}\left(s,n,\beta \right)+{\sum }_{j=2}^n\left[\mathit{C}\left(s,j-1,\beta \right)+{\sum }_{k=1}^{j-2}{\mathit{S}}_j\left(k,s,m_k,\beta \right)\right] \\ [{}^{>}\mathit{Y}\left(s+j-1,n-j+1,\beta \right)+\mathit{R}\left(s+j-1, n-j+1,\beta \right)], \end{gathered}$$

(6)

where ${\mathbf{S}}_j\left(k,s,m_k\right)={\sum }_{m_k=2}^{j-k}{\sum }_{m_{k-1}=1+m_k}^{j-k+1}\dots {\sum }_{m_1=1+m_2}^{j-1}{\prod }_{r=-1}^{k-1}\mathit{C}\left(s+m_{k-r}-1,m_{k-r-1}-m_{k-r}\right)$ for every $j\ge k+2$ while for every $j<k+2$, ${\mathit{S}}_j\left(k,s,m_k\right)=0.$ Let us now define as$\mathit{T}\mathit{V}\left(s,n,\beta \right)={\left[{TV}_1\left(s,n,\beta \right),\dots ,{TV}_k\left(s,n,\beta \right)\right]}^T$, the vector whose i-th element, ${TV}_i\left(s,n,\beta \right),$ is the expected present value of cost that is generated by the memberships of the system during the time interval $[s,s+n)$ under the condition that the memberships entered the system in state $i$ at time s and the discount factor equals $\beta$ [4]. Then we have

$$\mathit{T}\mathit{V}\left(0,t,\beta \right)= {\left[\mathit{N}\left(0\right)\right]}^T⌑\mathit{V}\left(0,t,\beta \right)+{\sum }_{m=1}^t\Delta T\left(m\right){\left[{\mathit{r}}_0\left(m\right)\right]}^T⌑\mathit{V}\left(m,t-m,\beta \right),$$

(7)

where

$$\begin{gathered} \mathit{T}\mathit{V}\left(0,t,\beta \right)= {\left[T{V}_1\left(0,t,\beta \right), T{V}_2\left(0,t,\beta \right),\dots , T{V}_k\left(0,t,\beta \right)\right]}^T, \\ \mathit{V}\left(s,t,\beta \right)={\left[v_1\left(s,t,\beta \right) v_2\left(s,t,\beta \right)\dots v_k\left(s,t,\beta \right)\right]}^T, \\ \mathit{N}\left(0\right)=\left[N_1\left(0\right) N_2\left(0\right)\dots N_k\left(0\right)\right]\mathrm{and} \\ {\mathit{r}}_0\left(m\right) \left[r_{01}\left(m\right), r_{02}\left(m\right),\dots , r_{0k}\left(m\right)\right]. \end{gathered}$$

Last, the total cost generated in the system by the memberships’ pathways until time t equals to the sum of the elements of $\mathit{T}\mathit{V}\left(0,t,\beta \right).$

3. Illustration

For the illustration, the data regarding patients’ stay within each state, and the corresponding generated costs, were informed by public data of the Greek public health sector, considering a public hospital and a population of patients which are stratified into a set of hospitals’ states. We denote by S = {Emergency Room (ER), Short-Term Acute Care (STAC), Hospitalization (H), Surgery Room, (SR), Intensive Care Unit (ICU)} the set of states that are assumed to be exclusive and exhaustive, so that each patient of the hospital may be in one and only one state at any time given. The possible trajectories within the hospital are visualized in Figure 1. We assume that the internal transition probabilities are governed by the following substochastic matrix:

$$\mathit{F}=\begin{array}{c}ER\\ STAC\\ H\\ SR\\ ICU\end{array}\left[\begin{array}{ccccc}0.00& 0.00& 0.10& 0.07& 0.03\\ 0.00& 0.00& 0.15& 0.10& 0.05\\ 0.00& 0.00& 0.00& 0.25& 0.05\\ 0.00& 0.00& 0.70& 0.00& 0.25\\ 0.00& 0.00& 0.90& 0.05& 0.00\end{array}\right].$$

It is assumed that a patient enters the hospital either through ER with probability 0.75 or through STAC with probability 0.25, defining ${\mathit{p}}_0$ $=$ [$\begin{array}{ccc}0.75,& 0.25,& 0,\end{array} \begin{array}{cc}0,& 0\end{array}$]. The death of the patient, the transition of the patient in other hospital or rehabilitation home for various medical reasons and the discharge to normal residence could be considered as exits from the hospital, and so the loss vector is assumed to be ${\mathit{p}}_{k+1}=$[$\begin{array}{ccc}0.80,& 0.70,& 0.70,\end{array} \begin{array}{cc}0.05,& 0.05\end{array}$]. As a consequence, the matrix of the embedded Markov chain P is the following:

$$\mathit{P}=\begin{array}{c}ER\\ STAC\\ H\\ OR\\ ICU\end{array}\left[\begin{array}{ccccc}0.6000& 0.2000& 0.1000& 0.0700& 0.0300\\ 0.5250& 0.1750& 0.1500& 0.1000& 0.0500\\ 0.5250& 0.1750& 0.0000& 0.2500& 0.0500\\ 0.0375& 0.0125& 0.7000& 0.0000& 0.2500\\ 0.0375& 0.0125& 0.9000& 0.0500& 0.0000\end{array}\right].$$

One can notice that according to the matrix P, some patients’ pathways correspond to zero probabilities. We assume that the transition probability matrix $\mathit{P}$ is non-homogeneous, hence we add a Gaussian noise term $e_{ij}\left(t\right)~N\left(0, \sigma \left(t\right)≔\frac{1}{1000+t^2}\right)$, in each cell, in order to create a sequence of matrices $\mathit{P}\left(t\right)$, in order to impose small random deviations to the estimated transition probabilities, reflecting the irregularities that can be observed in detailed real data. The choice of a Gaussian distribution is twofold. First, the noise term has zero expectation, therefore the average point estimates of the transition probabilities are not influenced by noise. Second, the variance of the noise term converges to zero, when $t\to \infty$, hence the process evolves into a homogeneous semi-Markov chain after a long time period. The waiting time of the patients in the states ER, STAC and SR is one day. The stochastic matrix P is irreducible, therefore, the $\underset{t\to \mathrm{\infty }}{\mathrm{l}\mathrm{i}\mathrm{m}}{\mathit{P}}^t$ exists and is equal to the stable matrix $\mathit{P}$*, which is the asymptotic matrix of the embedded Markov chain:

$${\mathit{P}}^{*}=\begin{array}{c}ER\\ STAC\\ H\\ OR\\ ICU\end{array}\left[\begin{array}{ccccc}0.4835& 0.1611& 0.1958& 0.1018& 0.0578\\ 0.4835& 0.1611& 0.1958& 0.1018& 0.0578\\ 0.4835& 0.1611& 0.1958& 0.1018& 0.0578\\ 0.4835& 0.1611& 0.1958& 0.1018& 0.0578\\ 0.4835& 0.1611& 0.1958& 0.1018& 0.0578\end{array}\right].$$

Patients that exit the hospital are replaced by some of the incoming patients who occupy their corresponding memberships in the hospital. Let us also note that the rest of the incoming patients that enter the hospital through its expansion according to $\Delta {\rm T}\left(t\right)$, create new memberships either through ER or STAC with probabilities 0.75 and 0.25, respectively, i.e., ${\mathit{r}}_0=$ [$\begin{array}{ccc}0.75,& 0.25,& 0,\end{array} \begin{array}{cc}0,& 0\end{array}$]. In addition, based on the semi-Markov reward model, the expected healthcare cost that is generated by the patients is estimated up to a specific time point. This cost is assumed to be independent of time and includes the total daily healthcare cost that is generated by the patients, as well as the medical consumables that are necessary for their treatment.

In 2012, the Diagnosis-Related Groups (DRG) table was created, which presents the maximum budget that can be sponsored to a public hospital for all possible cases of diseases and treatments. For each disease, the DRG includes the daily healthcare cost, the cost of health examinations, the cost of use or maintenance of medical equipment, such as respirators, stretcher chairs and any other cost associated for the treatment of the patient. According to the ministerial decree, which is published in the Government Gazette (ΦΕΚ 946Β’/27 March 2012), for any disease, the average daily healthcare cost is EUR 140. The DRG does not include the cost of medical consumables. According to a recent balance sheet of large public hospital in Greece, EUR 2,275,023.83 are spent annually for medical consumables, and on average, we assume that the cost of the medical consumables that correspond to each patient is EUR 10.62. Thus, the cost that is earned from a patient occupying either STAC or H is equal to EUR 150.62. The DRG does not include the daily healthcare cost in the ICU, as well as the cost of surgeries. According to the Government Gazette (ΦΕΚ 4898Β’/1 November 2018), the daily healthcare cost in the ICU is EUR 800 for each one of the first three days and EUR 550 for each day from the fourth day of occupying and onwards. Moreover, according to the United Mobility System, the cost of a surgery is EUR 1215.61 on average.

In this implementation, this cost is independent of patients’ holding time in the states. Moreover, if the patient does not transition from the ICU to H, then the cost of transition to H as well as the cost of transition to SR is equal to the cost of use/maintenance of stretcher or chair which transfers the patient. The cost that the patient generates for making a transition from the ICU to H or SR include the cost of use/maintenance of a medical machine such as respirator that may be used due to the patient’s health problem. We assume that the cost of use/maintenance of these medical machines per patient is calculated at EUR 10 on average. If the patient moves to the hospital with a private means of transportation, i.e., an ambulance is not used for the transition of the patient to hospital and also the patient does not use a stretcher or chair for the transition to ER or to STAC, then the cost of transition to ER or to STAC is zero. More generally in the current illustration, we assume that cost is generated in a hospital for the transition of the patient to the ER or to the STAC. The specific cost includes the cost of use/maintenance of the stretcher or the chair and the cost of ambulance’s fuel, which according to Government Gazette is defined to be EUR 0.15/km and a patient traverses 50 km on average by ambulance. Thus, the cost of the transition of a patient to the hospital by ambulance is equal to EUR 7.5 on average and we assume that the cost of use/maintenance of the stretcher or the chair is EUR 5. As a result, the cost of the transition a patient to the ER or to the STAC is defined to be EUR 12.50. Summarizing the above information, we define the following matrix of patients’ memberships, as $=$ ${\left\{b_{ij}\right\}}_{i,j\in S}$:

$$\mathit{B}=\begin{array}{c}ER\\ STAC\\ H\\ SR\\ ICU\end{array}\left[\begin{array}{ccccc}12.50& 12.50& 5.00& 5.00& 5.00\\ 12.50& 12.50& 5.00& 5.00& 5.00\\ 12.50& 12.50& 0.00& 5.00& 5.00\\ 12.50& 12.50& 5.00& 0.00& 5.00\\ 12.50& 12.50& 15.00& 15.00& 0.00\end{array}\right].$$

In the following, we present the estimated healthcare system’s structure along with the estimated cost for three scenarios with different initial population structure, expansion rate, and holding time distributions. All the calculations have been made through R (version 4.3).

3.1. Scenario 1

In a simple scenario, we assume that the expansion of the system is equal to a constant, $\Delta {\rm T}\left(t\right)=20,$ the initial system population vector is ${\rm N}\left(0\right)=[231, 152, 106, 34, 77]$, and the holding time distribution of the states are described by geometric distributions, as follows:

$$\mathit{H}\left(1\right)=\begin{array}{c}ER\\ STAC\\ H\\ SR\\ ICU\end{array}\left[\begin{array}{ccccc}1.00& 1.00& 1.00& 1.00& 1.00\\ 1.00& 1.00& 1.00& 1.00& 1.00\\ \frac{1}{8}& \frac{1}{8}& 1.00& \frac{1}{6}& \frac{1}{10}\\ 1.00& 1.00& 1.00& 1.00& 1.00\\ \frac{1}{5}& \frac{1}{5}& \frac{1}{3}& \frac{1}{3}& 1.00\end{array}\right],$$

while for $m>1$, we have

$$\mathit{H}\left(m\right)=\begin{array}{c}ER\\ STAC\\ H\\ SR\\ ICU\end{array}\left[\begin{array}{ccccc}0.0000& 0.0000& 0.0000& 0.0000& 0.0000\\ 0.0000& 0.0000& 0.0000& 0.0000& 0.0000\\ \frac{1}{8}{\left(\frac{7}{8}\right)}^{m-1}& \frac{1}{8}{\left(\frac{7}{8}\right)}^{m-1}& 0.0000& \frac{1}{6}{\left(\frac{5}{6}\right)}^{m-1}& \frac{1}{10}{\left(\frac{9}{10}\right)}^{m-1}\\ 0.0000& 0.0000& 0.0000& 0.0000& 0.0000\\ \frac{1}{5}{\left(\frac{4}{5}\right)}^{m-1}& \frac{1}{5}{\left(\frac{4}{5}\right)}^{m-1}& \frac{1}{3}{\left(\frac{2}{3}\right)}^{m-1}& \frac{1}{3}{\left(\frac{2}{3}\right)}^{m-1}& 0.0000\end{array}\right].$$

The estimated system structure over time $N\left(t\right)$ is presented in Figure 2 for the first 100 days by visualizing each of the components $N_i\left(t\right)$ of the five-dimensional vector. The relationship between time (measured in days) and the state size approximates a linear form. By assuming a simpler method, a linear regression model can be applied, in order to predict the number of patients within each state for a given day. For instance, based on the estimated regression coefficients (

Table 1. Estimated linear regression coefficients between state size and time.

	Intercept	β (Slope)
Emergency	143.275	3.976
Short-term acute care	49.262	1.314
Hospitalization	312.6	12.4
Surgery room	27.6764	0.8401
Intensive Care Unit	47.142	1.465

), it is expected that when $t=10$, the estimated state size will be $62$ patients within the ICU.

Figure 3 illustrates the number of occupied beds in both hospitalization and the intensive care unit for a period of 100 days.

Patients’ memberships generate costs associated with their stay within each state, and we consider the following matrix $\mathit{Y}=$ ${\left\{y_{ij}\right\}}_{i,j\in S}$ that summarizes these costs as:

$$\mathit{Y}=\begin{array}{c}ER\\ STAC\\ H\\ SR\\ ICU\end{array}\left[\begin{array}{ccccc}10.62& 10.62& 10.62& 10.62& 10.62\\ 150.62& 150.62& 150.62& 150.62& 150.62\\ 150.62& 150.62& 0.00& 150.62& 150.62\\ 1215.61& 1215.61& 1215.61& 0.00& 1215.61\\ 800.00& 800.00& 800.00& 800.00& 0.00\end{array}\right].$$

During the initial time period, patients’ cost within the ICU state is higher than the other costs, but after a week and onwards, the expected healthcare cost of all patients is higher for the state ER (Figure 4). In addition, the cost that is associated with medical services in the surgery rooms is the lowest for the whole time period.

The estimation of the average cost generated by a patient’s treatment within a state in the hospital in the long-term is equal to EUR 218.58, and this could be calculated by the column matrix ${\mathit{Q}}^{\ast }{\mathit{W}}^{-1}\mathit{R}$ [4], where:

$${\mathit{Q}}^{\ast }=\begin{array}{c}ER\\ STAC\\ H\\ SR\\ ICU\end{array}\left[\begin{array}{ccccc}0.20040& 0.06681& 0.61637& 0.04220& 0.07421\\ 0.20040& 0.06681& 0.61637& 0.04220& 0.07421\\ 0.20040& 0.06681& 0.61637& 0.04220& 0.07421\\ 0.20040& 0.06681& 0.61637& 0.04220& 0.07421\\ 0.20040& 0.06681& 0.61637& 0.04220& 0.07421\end{array}\right],$$

$${\mathit{W}}^{-1}=\left[\begin{array}{ccccc}1.00& 0.00& 0.00& 0.00& 0.00\\ 0.00& 1.00& 0.00& 0.00& 0.00\\ 0.00& 0.00& 0.13& 0.00& 0.00\\ 0.00& 0.00& 0.00& 1.00& 0.00\\ 0.00& 0.00& 0.00& 0.00& 0.32\end{array}\right], \mathrm{and} \mathit{R}=\begin{array}{c}ER\\ STAC\\ H\\ SR\\ ICU\end{array}\left[\begin{array}{c}21\\ 160\\ 1151\\ 1220\\ 2505\end{array}\right].$$

3.2. Scenario 2

Expanding the previous scenario, it is assumed that the system’s expansion rate is larger for smaller values of $t$, and decreases later on. Hence, initially, there is an increased inflow in the hospital, however as $t$ increases in size, we expect the number of new patients in the hospital to converge to 1, with the following rate $\Delta {\rm T}\left(t\right)=⌊\frac{20}{t}+\frac{100}{t^2}⌋+1$, while the holding time distributions are the same with the first scenario. Starting with an empty healthcare system, with ${\rm N}\left(0\right)=[0, 0, 0, \mathrm{0,0}]$, Figure 5 presents the evolution of the expected structure of the system for a given period of 100 days. At 100 days, the population size reaches 327 patients, while the patients are distributed according to $N\left(100\right)=[67, 22, 200, 14, 24].$ Initially, it is observed that the populations within the emergency room and STAC unit increase, but reach a certain threshold where they drop and then remain at an approximately stable level. On the other hand, the number of patients that are hospitalized increase rapidly at the beginning, however, after two weeks the rate of increase is constant.

By assuming that the hospital’s resources are limited, we also consider a hospital that contains a maximum of 103 ICU beds, and the initial structure of the hospital is not empty, according to ${\rm N}\left(0\right)=[231, 152, 106, 34, 77]$. The initial occupancy of the ICU is 74.76% and the total number of patients within the hospital is ${\rm T}\left(0\right)=600$. The results of the system’s structure are illustrated in Figure 6, which presents that in the current hospital there is always ICU availability, and according to the initial population vector $N\left(0\right)$, the number of patients in the ER is higher than the number of the patients in any other state.

Assuming the same population structure and expansion rate, we estimated the number of beds which are occupied in H as well as the ICU (Figure 7). Firstly, we observe that the estimated number of beds which are occupied in H is higher than the corresponding number of beds in the ICU. Initially, 106 beds are occupied in H, but for the second day, this number is estimated to be 92, and this is the only daily decrease in the number of beds occupied in H, while from the third day and onwards, we estimate that the above number is either increasing or remaining constant. The maximum daily increase in occupancies in hospitalization beds is equal to 68 beds. Moreover, the estimated number of beds which are occupied in the ICU decreases for about 18 days and then increases linearly. In more detail, regarding the ICU, the maximum daily decrease is equal to 25 beds, since initially 77 beds are occupied, whereas for the second day, 52 beds are occupied.

According to Figure 8, the increase in the cost generated by patients’ pathways through time has an approximately linear form. Initially, the cost of the ICU section appears to be the highest across all states, however, later on, the cost associated with ER surpasses all the other states, followed by STAC, while patients entering the surgery room generate the lowest cost.

3.3. Scenario 3

For the last scenario, the initial structure vector is considered to be ${\rm N}\left(0\right)=[300, 152, 106, 34, 50]$, thus we assume a reduced number of patients within the ICU. Also, the holding time distributions were assumed to be different for the ICU state, e.g., the average holding time within the ICU before discharge is assumed to be 7 days, while the average holding time before a surgery operation or hospitalization is assumed to be 5 days. From the visualization, one can observe that the occupied ICU beds surpass the hospital’s capacity (N = 103) approximately after 5 days, then exceeds slightly after 16 days and then the number of patients within the ICU continues to increase (Figure 9).

Figure 10 illustrates the number of occupied beds in both hospitalization and the intensive care unit for a period of 100 days and the visualization of the associated costs for each state are presented in Figure 11.

It is observed that the healthcare cost that is generated initially while patients enter in any state does not exhibit strong daily changes. The expected long-term patient cost for this scenario is EUR 247.11. Therefore, a 2-day increase in the average patient’s stay within the ICU results in a 13% larger patient cost in the long-run, compared to the first scenario, which is given by ${\mathit{Q}}^{\ast }{\mathit{W}}^{-1}\mathit{R}$, where:

$${\mathit{Q}}^{\ast }=\begin{array}{c}ER\\ STAC\\ H\\ SR\\ ICU\end{array}\left[\begin{array}{ccccc}0.19126& 0.06376& 0.58823& 0.04028& 0.11647\\ 0.19126& 0.06376& 0.58823& 0.04028& 0.11647\\ 0.19126& 0.06376& 0.58823& 0.04028& 0.11647\\ 0.19126& 0.06376& 0.58823& 0.04028& 0.11647\\ 0.19126& 0.06376& 0.58823& 0.04028& 0.11647\end{array}\right],$$

and

$${\mathit{W}}^{-1}=\left[\begin{array}{ccccc}1.00& 0.00& 0.00& 0.00& 0.00\\ 0.00& 1.00& 0.00& 0.00& 0.00\\ 0.00& 0.00& 0.13& 0.00& 0.00\\ 0.00& 0.00& 0.00& 1.00& 0.00\\ 0.00& 0.00& 0.00& 0.00& 0.20\end{array}\right], \mathrm{and} \mathit{R}=\begin{array}{c}ER\\ STAC\\ H\\ SR\\ ICU\end{array}\left[\begin{array}{c}21\\ 160\\ 1151\\ 1220\\ 4109\end{array}\right]$$

4. Conclusions

The aim of the current study was to examine patients’ pathways in an open healthcare system via the lens of a non-homogeneous semi-Markov reward system with discounting. The proposed model incorporated two novel components, namely states’ inflows and availability. The inclusion of the inflows allows for the measurement of the expected number of new recruits of every state (i.e., new members in the system plus entrances through internal transitions), at every time point, providing an estimation of the patients’ inflows to the hospital’s states. This leads to the estimation of various attributes of the system, such as the capacity and the availability of the states. The model also incorporated information about costs associated with patients’ transitions or stay within the states.

The theoretical results were accompanied with different case illustrations of open healthcare systems with various characteristics (initial structure, holding times distributions, and expansion rates). To our knowledge, this is the first time that the aforementioned theoretical framework is accompanied with an application to a healthcare system based on aggregated data from the Greek public health sector. Three different scenarios were developed to simulate a real healthcare system including open systems with constant or non-linear decreasing expansion rates. Through the findings, one can examine and predict the upcoming structure of the system for each state separately, aiming to assist resource allocation in healthcare units.

Through the illustration, the availability of each state is calculated providing useful information for capacity planning in a healthcare environment with limited resources. More specifically, the current availability of wards in critical units within a hospital can be estimated, such as the intensive care unit or surgery rooms. For example, in cases where the modelling procedure suggests that the number of patients exceeds the hospitals’ capacity after a time period, this may serve as a warning that the current healthcare resources are not adequate to provide medical services for all patients. Last, expressions for the expected costs generated by the system and corresponding to patients’ paths are estimated as well as the average patient’s long-term cost that is generated.

Our application can guide healthcare policy makers to configure different strategies in order to optimize the resources of a healthcare system, and also provide estimations of future costs under various scenarios. The results encourage the use of this model in exploring cost sensitivity arising from different treatment strategies and investigating optimal solutions.