Sepsis Treatment Recommendation Using Sensitivity to Input Medicine Dosage in Deep Neural Networks

Abstract

Sepsis is a life-threatening condition that ranks among the foremost global causes of mortality. Its treatment is marked by significant expenses and the incorporation of diverse symptomatology. Consequently, an array of investigative efforts has been dedicated to sepsis, spanning the classification of its stages, early detection, prognosis prediction, and therapeutic recommendations. Notably, the complex and contentious nature of sepsis management underscores the necessity for precision in combination therapies. In this research endeavor, this study proposes an advanced methodology for sepsis treatment recommendations grounded in deep neural networks. The approach entails the construction of an ensemble deep learning model geared towards predicting the subsequent Sequential Organ Failure Assessment (SOFA) score. Employing this trained ensemble model, the study embarks on the task of optimizing sepsis treatment dosages. The empirical results conclusively demonstrate the superior performance of the proposed ensemble model relative to those of the conventional methods, signifying its capacity to offer treatment prescriptions akin to or surpassing those rendered by medical practitioners. The model consistently outperforms the alternative approaches in predicting the SOFA score and aligns the treatment recommendations with those of medical professionals, exhibiting a high degree of similarity. This innovative approach holds promise for advancing personalized medicine and improving patients’ outcomes in sepsis treatment.

1. Introduction

Sepsis represents a complex clinical syndrome arising from the activation of the body’s immune and coagulation systems in response to an infection. It is fundamentally distinguished by the development of toxemia and bacteremia, setting it apart from other infectious and inflammatory processes. Toxemia refers to the presence of bacterial toxins in the bloodstream, leading to a systemic inflammatory response. Bacteremia, on the other hand, signifies the presence of live bacteria in the blood, which can result from the infection’s dissemination. This combination of immune system activation, coagulation system involvement, toxemia, and bacteremia characterizes sepsis as a severe and life-threatening condition that requires immediate medical attention.

This puzzling medical condition is difficult to predict accurately and does not have a definitive treatment. Globally, sepsis stands as the foremost cause of hospital-related fatalities, contributing to approximately 11 million deaths annually [1]. In the United States, the economic burden of sepsis exceeds a staggering USD 24 billion annually [2].

In the pursuit of sepsis prevention and treatment, multifaceted research endeavors have been undertaken and categorized into various approaches. Notably, within the realm of bioinformatics, data-driven methodologies have been employed to proactively detect sepsis, stratify its severity, predict the prognostic outcomes, and recommend treatment strategies [3]. Early diagnosis, given its potential to mitigate morbidity, mortality, and healthcare costs, is of paramount significance within this domain.

Numerous research initiatives have proposed early diagnosis methods, with some employing Long Short-Term Memory (LSTM) techniques to facilitate the advanced detection of septic shock. These approaches involve the transfer of temporal information from previous data points to the current one, while concurrently extracting the current features [4]. Lauritsen et al. have created an early detection methodology characterized by the training of features derived from clinical temporal data. Their pioneering work combines Convolutional Neural Networks (CNN) and LSTM networks to achieve an enhanced diagnostic accuracy [5].

In clinical practice, the classification of sepsis holds paramount significance. The failure to promptly diagnose severe sepsis can lead to a steep escalation in the risk of mortality. Consequently, various research endeavors have been undertaken to address this challenge. Klouwenberg et al. have undertaken a quantitative assessment of the impact of minor alterations in the definitions and measurement criteria for Systemic Inflammatory Response Syndrome (SIRS) and organ failure in the context of sepsis, severe sepsis, and septic shock [6]. Furthermore, Parente, J. D. et al. have proposed a clinical decision-making methodology to facilitate the diagnosis of severe sepsis and septic shock [7].

Sepsis patients account for nearly a quarter of intensive care unit (ICU) admissions in Western countries, with a mortality rate of 12.8% for sepsis and a significantly higher one of 45.7% for septic shock. Hence, the accurate estimation of prognosis is of paramount importance for both medical practitioners and patients. Perng, J. W. et al. have applied predictive modeling to anticipate diverse outcomes in cases of a suspected infection [8]. Ribas, V. J. et al. have employed logistic regression on latent factors to forecast mortality in severe sepsis cases [9].

The advent of deep learning and the resurgence of reinforcement learning have spurred active research in the domain of treatment recommendation. Katharine E. Henry discusses how prior experiences with emergency room settings and systems have influenced healthcare providers’ interactions and alert confirmations. They also explain the ways to improve the adoption and impact of early warning systems [10]. Melissa Y Yan discusses the utilization of unstructured clinical text in machine learning (ML) to enhance sepsis outcomes [11]. Virtually all the approaches for recommending sepsis treatments leverage deep reinforcement learning methodologies. These approaches derive treatment policies through the utilization of Deep Q networks, optimizing the critical parameters. The pioneering approach in 2017 introduced the use of Double Q networks to learn the treatment policies [12]. Building upon this foundation, the subsequent research has explored continuous state-space model-based reinforcement learning (RL) to unearth high-quality treatment policies [13]. Additionally, Yu, C. et al. focuses on identifying the salient factors impacting mortality during sepsis treatment [14]. Huang, Y. et al. uses the Deep Deterministic Policy Gradient (DDPG) algorithm to learn the policies that recommend dosages of Vasopressors and Intravenous fluids for sepsis patients [15]. Despite these efforts, the management of septic patients remains exceptionally challenging due to the significant inter-individual variability in patients’ responses to medical interventions and the absence of universally accepted sepsis treatment protocols.

In addition to sepsis treatment, the majority of treatment recommendation methods have adopted reinforcement learning as a fundamental approach for providing treatment recommendations. However, it is important to acknowledge that while reinforcement learning offers evident advantages, it also exhibits certain limitations, particularly when applied to medical datasets.

One of the primary limitations arises from the necessity to compute rewards for actions, a process that requires the creation of a virtual environment and a virtual agent. This virtualization is infeasible in clinical settings, where real patients are situated in actual healthcare environments. Second, the restricted number of available actions within reinforcement learning makes it challenging to accommodate sophisticated treatment strategies. Furthermore, aligning sepsis patients’ treatment with their current clinical state poses difficulties, as reinforcement learning typically trains policies rather than prescribing specific treatment methods. Finally, the qualitative analysis of learned treatment policies does not furnish unequivocal performance metrics.

This paper explicitly concentrates on addressing these aforementioned challenges within the domain of sepsis treatment. The research objectives encompass two key aspects:

(1)

An algorithm for SOFA score estimation: We introduce an innovative algorithm for estimating the next Sequential Organ Failure Assessment (SOFA) score. This algorithm leverages a deep ensemble model to project a patient’s expected SOFA score progression in response to treatment interventions, addressing the variability in patients’ responses.

(2)

An algorithm for sepsis treatment recommendation: Our novel approach focuses on making modest adjustments to the dosages of sepsis treatments, circumventing the limitations associated with traditional reinforcement learning methods. This patient-specific and data-driven strategy enhances the quality of sepsis patients’ care.

2. Materials and Methods

This section delineates a novel sepsis treatment recommendation methodology. The process commences with the training of a predictive model for estimating the subsequent Sequential Organ Failure Assessment (SOFA) score. Subsequently, a set of actionable interventions is introduced into the prediction model. The selection of the optimal treatment strategy is determined through the application of a “winner-takes-all” approach, as illustrated in Figure 1.

To elaborate further, the predictive model for the next SOFA score is engineered to assimilate patient-specific data, clinical variables, and historical information. It leverages deep learning techniques to establish a predictive framework capable of projecting the patient’s expected SOFA score progression in response to different treatment interventions. The model’s capacity to generalize and adapt to varying patient profiles is a pivotal attribute for personalized treatment recommendations.

The “winner-takes-all” strategy involves the assessment of multiple treatment options, each associated with different expected outcomes based on the predictive model. The recommendation system selects the intervention that maximizes the anticipated reduction in the SOFA score, optimizing the patient’s response to the treatment. This integrated approach, coupling the SOFA score prediction and treatment recommendation, offers a tailored and data-driven strategy for optimizing sepsis patients’ care, potentially mitigating the severity of the condition and enhancing the overall clinical outcomes.

2.1. Dataset Description

To facilitate the implementation of this methodology, this study harnesses the “Medical Information Mart for Intensive Care version III” dataset (MIMIC-III) [16,17,18]. MIMIC-III encompasses a rich repository of comprehensive clinical and demographic information extracted from two prominent intensive care unit (ICU) databases within the United States, aggregating data from a substantial cohort of over 100,000 patients.

The dataset is composed of various tables, each serving a distinct purpose in capturing the multifaceted aspects of patient care and clinical processes. Some of the key tables used in this study include:

• * admissions: This table contains information related to patient admissions, including patient identifiers (subject_id), admission identifiers (hadm_id), admission time (admittime), discharge time (dischtime), death status (deathtime), admission type (admission_type), admission location (admission_location), discharge location (discharge_location), insurance type (insurance), religion, marital status (marital_status), ethnicity, and diagnosis. These details provide valuable context for each patient’s hospitalization.
• * callout: The callout table is instrumental in tracking patient movements within the hospital. It includes data, such as patient identifiers (subject_id), admission identifiers (hadm_id), intensive care unit stay identifiers (icustay_id), requested time for transfers (requested_datetime), expected time for transfers (expected_datetime), and completed transfer times (completed_datetime). This information ensures a comprehensive understanding of patient transitions between care units.
• * caregivers: This table contains information about the healthcare providers involved in patient care. It includes caregiver identifiers (cgid) and caregiver types (label), which can be essential in assessing the care team’s composition and roles.
• * chartevents: The chartevents table records a wide array of clinical parameters, including patient identifiers (subject_id), admission identifiers (hadm_id), intensive care unit stay identifiers (icustay_id), caregiver identifiers (cgid), measurement items (itemid), measured values (valuenum), units of measurement (valueuom), and measurement times (charttime). These details offer insights into the patients’ physiological data, treatment responses, and vital signs.
• * cptevents: This table contains the Current Procedural Terminology (CPT) codes for medical procedures. It includes patient identifiers (subject_id), admission identifiers (hadm_id), and CPT codes (costcenter). Understanding the procedures performed is crucial in evaluating patients’ care and interventions.
• * d_cpt: Diving deeper into the CPT codes, the d_cpt table provides descriptions for these codes, including categories (category) and section ranges (sectionrange). This information helps interpret the meaning and significance of CPT codes.

These are just a few examples of the tables within the extensive MIMIC-III dataset, each contributing unique insights into the clinical journey of patients. This rich collection of data allows the comprehensive analysis and modeling of patient conditions, treatments, and outcomes, making it an invaluable resource for the study at hand.

2.2. Feature Preprocessing

The dataset utilized in this study is particularly valuable for assessing the impact of Intravenous (IV) and Vasopressor treatments on patients. To facilitate a comprehensive and comparative evaluation of the outcomes resulting using the dosage-adjustment methodology proposed in this research and the treatment regimens recommended by medical professionals, a crucial preprocessing step was undertaken.

This preprocessing procedure involved augmenting each entry in the dataset with the subsequent Sequential Organ Failure Assessment (SOFA) score recorded at 4 h intervals following the administration of treatments. The SOFA score for the time point immediately following each recorded entry was introduced into the current data row. This crucial augmentation allowed a direct correlation between the treatment administered and the subsequent SOFA score, enabling the detailed analysis of the treatment’s impact on patients.

In the course of data preprocessing, a meticulous iterative process was implemented to insert the next SOFA score from the following time point into each corresponding row. However, it is important to note that in cases where obtaining the subsequent SOFA score was not feasible, such entries were judiciously eliminated to ensure data integrity and alignment. This process of data augmentation and cleansing is illustrated in Figure 2, underscoring the importance of aligning patient data with their corresponding SOFA scores for subsequent analysis and evaluation.

This data preprocessing phase is essential to ensure the dataset’s quality and relevance in investigating the effectiveness of the proposed treatment dosage adjustments and their impact on the patients’ outcomes.

2.3. Model Architecture

This study developed a regression model for the next Sequential Organ Failure Assessment (SOFA) score, a mortality prediction metric based on the severity of dysfunction in six organ systems. This score ranges from 0 (indicating normal organ function) to 4 (representing the most severe organ dysfunction). Since the SOFA score is based on ordinal scales associated with fatality rates, the task was framed as a regression problem, with the model aiming to predict the next SOFA score.

The network architecture employed a multi-layer perceptron (MLP) consisting of three to five layers, including input and output layers. A dropout layer with a 50% dropout rate between the hidden layers was introduced to prevent overfitting. The output layer used the softmax activation function. Each input “x” represented clinical data for each time step, and the network produced a single value “µ(x).” The hyperparameters were fine-tuned using the Adam optimization method on the training dataset.

To enhance model accuracy and effectively reduce variance error, an ensemble approach comprising five distinct MLP models was adopted. In this ensemble approach, the Bagging method was utilized. Bagging is an ensemble technique in which multiple subsets of the training data are created through bootstrap sampling. Each subset is used to train a separate base model independently. After training, their predictions are combined to make a final prediction. Given the regression nature of the approach, the final prediction was computed through the process of averaging the individual predictions, a prevalent technique in regression problems. This method entails aggregating the predictions from the individual base models, and the final output is determined by their averaged values. The formula for Bagging is expressed as follows:

$$Ensemble Prediction=\frac{1}{n}\mathrm{\ast }\mathsf{\Sigma }[H_i(\mathrm{x})]$$

(1)

where ‘n’ denotes the count of base models used in the ensemble. ‘H_i(x)’ represents the prediction of each specific base model (‘i’). Each model predicted the next SOFA score based on predicted probabilities for regression. These models were assigned varying weight volumes, and their predictions were combined into a single predictor. The ensemble method was leveraged to predict the next SOFA score for each time step, as illustrated in Figure 3.

2.4. Treatment Recommendation

The crux of the treatment recommendation system lies in the ability of the next Sequential Organ Failure Assessment (SOFA) prediction network to accurately forecast the subsequent SOFA score for a given treatment regimen, encompassing all the patients. Consequently, the authors have chosen the proposed ensemble model as the foundation for identifying the optimal treatment strategies.

To systematically explore the optimal treatment dosages for sepsis, the authors undertook the task of generating the next SOFA score for each time step by incrementally adjusting the dosage levels. The Intravenous (IV) and Vasopressor dosages were discretized into 100 categories each, resulting in a total of 10,000 potential candidates at each time step. In other words, a total of 10,000 different combinations of Intravenous (IV) and Vasopressor dosages were used as inputs for the trained ensemble model, resulting in the acquisition of 10,000 respective next SOFA scores for each combination. The goal was to select a treatment approach that results in the lowest possible next SOFA score among the 10,000 different values obtained for next SOFA.

As illustrated in Figure 4, an example of the dosage adjustments is depicted. The proposed treatment recommendation mechanism selects from the pool of 10,000 predicted candidates derived from the next SOFA deep learning model. This process is integral to optimizing the sepsis treatment for individual patients, while leveraging the predictive capabilities of the ensemble model to achieve more favorable clinical outcomes.

2.5. Evaluation

In the evaluation of the next Sequential Organ Failure Assessment (SOFA) prediction, three distinct metrics were employed to facilitate a comprehensive comparison with both the proposed method and other existing approaches. These metrics encompass the Root-Mean-Squared Error (RMSE), Concordance Correlation Coefficient (CCC), and the R-square score. The RMSE, a widely accepted measure for quantifying predictive model accuracy, is formally defined as follows:

$$RMSE=\sqrt{{\sum }_{i=1}^n\frac{{(\widehat{y_i}-y_i)}^2}{n}}$$

(2)

Here, $\widehat{y_i}$ represents predicted value, $y_i$ signifies ground truth, and n denotes the number of observations.

The Concordance Correlation Coefficient (CCC) assesses the agreement between the predicted values and ground truth data, which is defined as:

$${\rho }_C=\frac{2\rho {\sigma }_{\widehat{y}}{\sigma }_y}{{\rho }_{\widehat{y}}^2+{\rho }_y^2+{({\mu }_{\widehat{y}}-{\mu }_y)}^2}$$

(3)

where Near ± 1 is perfect concordance, and 0 is no correlation. The R square (Coefficient of determination) score is a regression score function. The best possible score is 1.0, and it can be negative. R square is defined as

$$R^2=1-\frac{{\sum }_i{(y_i-{\widehat{y}}_i)}^2}{{\sum }_i{(y_i-\mu )}^2}$$

(4)

To evaluate the treatment recommendation system, a comparison is made between the treatment prescriptions generated using the method and those prescribed by medical practitioners, thereby assessing the alignment between the recommendations and the doctor’s decisions.

2.6. Experimental Setting

To assess the model’s performance, k-fold cross-validation with k = 5 was conducted. In each fold, the data were randomly divided into a training set (80%) and a testing set (20%). The hyperparameters were judiciously chosen based on the dataset’s feature characteristics [19].

Each network shares analogous architectural designs and exhibits a comparable number of parameters. In all the datasets, a 5-layer neural network was employed. The training process was carried out utilizing the Adam optimization method, with a learning rate set at 10⁻⁵. Xavier initialization was uniformly applied across all the layers, while a dropout probability of 0.5 was selectively implemented solely within the third layer to mitigate overfitting concerns.

3. Results

To conduct a comprehensive qualitative assessment, the has authors have incorporated several machine learning methodologies alongside the proposed approach. Furthermore, visual evaluation of the results was conducted through the generation of graphical representations.

To ensure a rigorous and well-rounded evaluation across multiple dimensions, three distinct evaluation metrics were selected.

By examining Figure 5, it becomes evident that the model excels in achieving a close alignment between the red trend line and the blue ground truth line, effectively capturing the underlying data distribution. This result is indicative of the model’s robust performance relative to those of the other models in the comparison.

In the evaluation based on the histogram, the study achieved favorable results (Figure 6), highlighted by a concentrated distribution of discrepancies that indicates the model’s commendable accuracy in approximating the ground truth data.

4. Discussion

This study presents a sepsis treatment recommendation system integrated with a next SOFA score prediction model. The paramount aspect of our research is the meticulous pursuit of the most precise next SOFA score prediction model, recognizing its pivotal role in the overall performance of the treatment system. The ensuing table provides a comprehensive overview of the RMSE results, shedding light on the predictive accuracy of the model (

Table 1. Comparison of RMSE scores for different methods.

Methods	RMSE
LogisticRegression	2.572
LinearRegression	2.079
DecisionTreeRegressor	2.751
MLP	2.0004
RandomForestRegressor	1.988
Soft_voting (logistic + decisiontree + LF)	2.102
AdaBoost (5 LinearRegressions)	2.182
GradientBoostingRegressor	2.74
AdaBoost (3 MLPs)	2.050
Proposed	1.946

).

These results show the predictive accuracy of each method in estimating the SOFA score, with the proposed approach demonstrating a superior performance. The evaluation employed standard assessment metrics, including the Concordance Correlation Coefficient (CCC) and R-square (

Table 2. Comparison of CCC scores for different methods.

Methods	CCC
LogisticRegression	0.512
LinearRegression	0.606
DecisionTreeRegressor	0.491
MLP	0.653
RandomForestRegressor	0.651
Soft_voting (logistic+decisiontree+LF)	0.621
AdaBoost (5 LinearRegressions)	0.550
GradientBoostingRegressor	0.179
AdaBoost (3 MLPs)	0.638
Proposed	0.668

and

Table 3. Comparison of R square scores for different methods.

Methods	R Square Score
LogisticRegression	0.074
LinearRegression	0.437
DecisionTreeRegressor	−0.028
MLP	0.454
RandomForestRegressor	0.480
Soft_voting (logistic+decisiontree+LF)	0.427
AdaBoost (5 LinearRegressions)	0.417
GradientBoostingRegressor	0.162
AdaBoost (3 MLPs)	0.440
Proposed	0.491

). The table clearly illustrates that the proposed method consistently outperforms the conventional approaches, yielding notably more accurate results.

In the comparative analysis with physician-prescribed treatments, the proposed sepsis treatment recommendation model exhibited a similarity rate of 74.58182%. This indicates a significant alignment between the model’s recommended treatments and those prescribed by the medical professionals. Such a high degree of similarity underscores the potential utility and effectiveness of the model in aiding clinical decision making for sepsis treatment.

The following key aspects explain why our study claims superiority over the traditional reinforcement learning (RL) methods:

Sensitivity Analysis: The proposed approach leverages sensitivity analysis to optimize the sepsis treatment dosages. This involves making minor adjustments to the dosage levels based on predictions from our deep learning model. The objective is to fine-tune the treatment strategies, rendering them more patient-specific and adaptive to individual patients’ needs.

Data-Driven Approach: The proposed method focuses on providing data-driven treatment recommendations based on the predicted Sequential Organ Failure Assessment (SOFA) scores. This approach acknowledges the varying responses of individual patients to the treatment. These variations can be significant and require personalized consideration, which traditional RL methods may not readily provide.

Overcoming RL Limitations: In conventional RL, the creation of a virtual environment and agent to compute rewards for actions can be challenging in clinical settings. Our model overcomes this limitation by not relying on RL, making it more practical and applicable in real-world clinical scenarios.

More Sophisticated Treatment Strategies: The proposed method discretizes the dosages of Intravenous (IV) and Vasopressor treatments into 10,000 categories, allowing an extensive exploration of the potential treatment combinations. This approach provides a broader range of treatment options when compared to those of traditional RL, which often involves a limited set of predefined actions.

5. Conclusions

In this paper, the authors introduced a novel sepsis treatment recommendation approach utilizing sensitivity analysis. The methodology entailed the initial training of an ensemble deep learning model to forecast the subsequent SOFA score. Subsequently, sensitivity analysis was employed to derive optimal treatment regimens by making minor adjustments to the dosage levels. The experimental findings unequivocally demonstrate that the model consistently yields high-quality results when compared to those of the alternative methods. Looking ahead, this research trajectory includes extending this methodology to explore various diseases and leverage multi-modality data, encompassing radiomics, genomics, and clinical features, to discern optimal treatment strategies. This multi-faceted approach holds promise for advancing personalized medicine and improving patients’ outcomes.