Next Article in Journal
An Image Fusion Method of SAR and Multispectral Images Based on Non-Subsampled Shearlet Transform and Activity Measure
Next Article in Special Issue
Efficient Distributed Wireless Power Transfer System for Multiple Wearable Sensors through Textile Coil Array
Previous Article in Journal
Photoelectric Target Detection Algorithm Based on NVIDIA Jeston Nano
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Sensors
Volume 22
Issue 18
10.3390/s22187054
sensors-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Advantage of Vital Sign Monitoring Using a Wireless Wearable Device for Predicting Septic Shock in Febrile Patients in the Emergency Department: A Machine Learning-Based Analysis

by
Arom Choi
1,
Kyungsoo Chung
2,
Sung Phil Chung
1,
Kwanhyung Lee
3,
Heejung Hyun
3 and
Ji Hoon Kim
1,*
1
Department of Emergency Medicine, Yonsei University College of Medicine, 50 Yonsei-ro, Seodaemun-gu, Seoul 03722, Korea
2
Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Yonsei University College of Medicine, 50 Yonsei-ro, Seodaemun-gu, Seoul 03722, Korea
3
AITRICS, 28 Hyoryeong-ro 77-gil, Seocho-gu, Seoul 06627, Korea
*
Author to whom correspondence should be addressed.
Sensors 2022, 22(18), 7054; https://doi.org/10.3390/s22187054
Submission received: 25 July 2022 / Revised: 2 September 2022 / Accepted: 13 September 2022 / Published: 17 September 2022
(This article belongs to the Special Issue Recent Advances in Wireless Power and Data Transfer System for Biomedical/Wearable/Healthcare Devices)
Browse Figures
Versions Notes

Abstract

:
Intermittent manual measurement of vital signs may not rapidly predict sepsis development in febrile patients admitted to the emergency department (ED). We aimed to evaluate the predictive performance of a wireless monitoring device that continuously measures heart rate (HR) and respiratory rate (RR) and a machine learning analysis in febrile but stable patients in the ED. We analysed 468 patients (age, ≥18 years; training set, n = 277; validation set, n = 93; test set, n = 98) having fever (temperature >38 °C) and admitted to the isolation care unit of the ED. The AUROC of the fragmented model with device data was 0.858 (95% confidence interval [CI], 0.809–0.908), and that with manual data was 0.841 (95% CI, 0.789–0.893). The AUROC of the accumulated model with device data was 0.861 (95% CI, 0.811–0.910), and that with manual data was 0.853 (95% CI, 0.803–0.903). Fragmented and accumulated models with device data detected clinical deterioration in febrile patients at risk of septic shock 9 h and 5 h 30 min earlier, respectively, than those with manual data. Continuous vital sign monitoring using a wearable device could accurately predict clinical deterioration and reduce the time to recognise potential clinical deterioration in stable ED patients with fever.

1. Introduction

Patients who visit the emergency department (ED) for fever account for 5% of all patients and 15% of all older adult patients in the ED, and those with chronic diseases or aged ≥65 years have a 30-day mortality rate of 7–9% [1]. Sepsis is a serious complication in patients who present to the ED with febrile symptoms and can lead to a fatal clinical outcome [2]. Therefore, it is crucial to recognise the deterioration in these patients at the early stages of their disease courses and to promptly intervene to avoid poor outcomes [3,4]. Since changes in vital signs are important indicators of physiological decline, their immediate detection enables early recognition of clinical deterioration and intervention [5,6,7,8]. However, it is practically impossible for physicians to closely monitor and observe each patient who presents to the ED. Therefore, close patient monitoring is prioritised for patients who are unstable during triage or the first check-up. Accordingly, early detection of clinical deterioration may not be achieved by intermittent measurement of vital signs since resources are not continuously provided to patients initially deemed as not requiring close monitoring [9,10,11,12,13,14,15]. Furthermore, febrile patients have become more vulnerable to deterioration since the outbreak of the coronavirus disease 2019 (COVID-19) as bedside patient observation has been restricted to control the spread of the disease [16,17].
Recently, wireless wearable devices for continuous monitoring of heart rate (HR), respiratory rate (RR), body temperature, and patient location have been introduced worldwide. Previous studies found that these devices can improve the safety of patients who have difficulty accessing regular monitoring devices [18,19,20,21]. However, the reliability of these devices in relevant clinical environments has not been fully validated; thus, their practical use in clinical settings is limited [22,23,24]. Furthermore, only a few studies were conducted in the ED, and most studies focused on the use of these devices in patients admitted to wards [15,25].
Due to the tremendous advances in the development of machine learning and deep learning algorithms, the availability of large databases, and the increase in computational processing power, machine learning in the medical field has rapidly evolved over the past two decades, with a wide range of applications using several different algorithms such as natural language processing, data mining, clustering, and classification [26].
The size of data obtained using a continuous monitoring device is much larger than that obtained using the manual method in which the measurements are obtained intermittently. Machine learning-based analysis is being used in various clinical research studies that include large amounts of data since it is suitable to promptly manage large-scale datasets with various datatypes within a short time [27,28].
Although previous studies revealed that frequently monitored vital signs are useful for clinical decision making, the particular type and frequency of vital signs that should be monitored and the correlation between each parameter and poor outcomes are yet to be studied [29]. With the growing number of patients presenting to EDs and the enormous amount of data related to patient monitoring, conventional techniques are considered inadequate to process these data. Therefore, the efficiency of emergency medical practice can be maximised if a large amount of data generated during the emergency department stay can be immediately processed using a machine learning algorithm [30,31].
Therefore, in the present study, we aimed to evaluate the predictive performance of a wireless monitoring device for the continuous measurement of HR and RR in febrile but stable patients in the ED using machine learning-based analysis.

2. Materials and Methods

2.1. Study Design and Population

This was a retrospective study conducted using data collected prospectively at the ED of a tertiary teaching hospital from 20 July 2020 to 15 June 2021. We used an interventional design, which involved continuous monitoring of the signals of patients obtained using a wireless wearable device and intermittent manual measurements of the vital signs.
Patients aged ≥18 years who were stable but had a body temperature over 38 °C upon arrival and were admitted to the isolation care unit in the ED were enrolled in this study. The isolation care unit in the ED is for patients with suspected COVID-19. All patients who met the enrolment criteria were asked to participate in the study on arrival at the ED, and written informed consent was obtained if the patient agreed to participate in the study. This study was approved by the Institutional Review Board of Yonsei University College of Medicine, Severance Hospital (no. 1-2019-0058).

2.2. Description of the Wireless Wearable Device

Hicardi (MEZOO Co., Ltd., Wonju-si, Gangwon-do, Korea) is an 8 g, 42 × 30 × 7 mm, wireless, and wearable adhesive monitoring device certified as a medical device by the Ministry of Food and Drug Safety of Korea. This device is used to monitor and record single-lead electrocardiograms (ECGs), RR, and skin surface temperature, as well as patient location and activity (Figure S1). It comprises a reusable sensor module and a disposable adhesive patch that houses a two-point ECG electrode. The reusable sensor module contains a microprocessor, an ECG/respiration circuit, a temperature sensor, a triaxial accelerometer, a Bluetooth low-energy (BLE) transceiver, and a 50 mAh Li-ion polymer battery. This device can be attached to the chest and can continuously measure signals for up to 24 h. The sensor processes and transmits the signals through BLE to a mobile device with a smartphone application. This application can display ECG, HR, RR, patient posture, and geographical location in real time and transmit the data to a cloud-based monitoring server with a randomised research number. Patient information is not recorded on the mobile device to ensure anonymisation. This system complied with the international standards and guidelines of regulatory authorities regarding cybersecurity [32,33].
The ECG signal is recorded at a rate of 250 samples/s in 14 bits. The algorithm, which calculates the HR using ECG, is based on the automated detection of QRS complexes of ECG waveforms. RR is derived from the changes in respiration detected through the measurement of the electrical impedance of the patient’s thorax. This device with a real-time heartbeat detection algorithm showed 99.95% of the sensitivity, 99.95% of the positive predictivity, and 0.10% of the detection fail rate on the four different databases in the previous study [34].

2.3. Data Acquisition

ECG rhythm and respiratory signals were continuously monitored using the wearable wireless device and manually measured intermittently. After obtaining informed consent from the patient, the wireless wearable monitoring device was attached to the patient’s left sternal border. Measurement of vital signs, such as systolic blood pressure, diastolic blood pressure, HR, RR, and mental status, was performed manually and recorded every hour, from baseline to 6 h later, while the wearable device was attached to the patient. If the patient died, was discharged, or was hospitalised within 6 h from the time of enrolment in this study, the data collected up to that time were used.
The signals were uploaded to the cloud-based server according to the device number and analysed after data collection was completed. The data from the device were separated, encrypted, and stored in electronic medical records in the hospital legacy system [35]. The researchers checked the central monitor and verified the quality of the signal data during data collection to determine whether the ECG and respiratory waveforms were properly transmitted from the mobile device to the cloud server. Regarding the signal data from the wearable monitoring device, 250 Hz ECG waveform data with information about R-peak location and 25 Hz respiratory waveform data were used.

2.4. Signal Data Filtering Process

The raw data transmitted by the wearable device were obtained, and invalid data were removed to retrieve continuous vectors of vital signals. Vital sign data were sampled once per second with their timestamps. Artifacts were not removed before the signal data filtering process. For ECG data, one unit was set and sliced to process the outlier so that the QRS wave was at the centre of each unit. If a unit was 1.5 times longer than the median of the units, or if the maximum or minimum values of the units were twice as large or small as the other units, these units were excluded to minimise the noise of the signal. The median HR for each unit was calculated from the entire ECG data, and the units excluded as described above were processed using the carry forward method. For respiratory signal data, the units excluded from ECG data filtering were removed at first, and the number of sections between each peak of the respiratory signal wave was calculated to obtain the RR [36]. If the value of the respiratory wave was 0.5 times smaller or 1.5 times larger than the previous respiratory value, these units were excluded. The median RR for each unit was calculated from the entire respiratory data, and the units excluded as described above were processed using the carry forward method.

2.5. Development of the Prediction Model

We processed and utilised 14 different values for model development. Data regarding age, sex, body temperature, and oxygen saturation were recorded on arrival and included as the static values for model development (four values), and data regarding systolic blood pressure, diastolic blood pressure, HR, RR, and mental status obtained every hour were included as the dynamic values (five values); the data were measured either manually or using the wearable device. We then calculated the difference between the data measured hourly and the vital sign data measured initially (five values).
With these data, we created a data fragment to build two different models—a simple computed model that predicted outcomes using only one data fragment in a specific time unit, and the accumulated model that predicted outcomes by considering all previous data fragments comprehensively. For data fragmentation, data units of the 14 previously mentioned values (five static values, four dynamic values, and four difference values), measured every hour, were used as manual data, whereas data units of the same 14 values, with HR and RR replaced with signal data from the wearable device, measured every 5 min, were used as device data.
For the development of the model, we used a convolutional neural network (CNN) and a long-short term memory (LSTM) network for time-sensitive analysis. A CNN is a type of artificial neural network commonly used for the analysis of visual images, such as ECG signals [37]. LSTM is a structure with an artificial recurrent neural network that is used in deep learning algorithms. Compared with standard neural networks, LSTM is applied during unsegmented data handling owing to the property that processes entire sequences of data using feedback connections [38]. The technical structure of the model is shown in Figure 1 and Supplementary Table S1. Considering the average time to evaluate, diagnose, and decide disposition during the ED stay, we set the data acquisition time to 6 h to predict septic shock in febrile patients. Regarding the estimation of the time to predict deterioration, it was assumed and recorded that it took over 6 h to predict septic shock if the model failed to predict it within the first 6 h.

2.6. Outcomes

The primary outcome was clinically identified septic shock requiring treatment with vasopressors to maintain a mean arterial pressure of 65 mm Hg or more in the absence of hypovolemia after adequate fluid administration within 24 h from the time of enrolment [39].

2.7. Statistical Analysis

Statistical analyses were performed using R software version 3.4.4 for Windows (R foundation for statistical computing, Vienna, Austria). The results are presented as mean ± standard deviation (SD) or median (interquartile range) for continuous variables and as frequencies (%) for categorical variables. The patients were randomly allocated to a training set, a validation set, and a test set in a 6:2:2 ratio for machine learning and model development. The sensitivity and specificity of the models were examined. The discriminative ability of each model was assessed using the area under the receiver operating characteristic curve (AUROC) and the area under the precision–recall curve with 95% confidence interval (CI) values.

3. Results

3.1. Baseline Characteristics

A total of 1263 febrile patients were admitted to the isolation care unit of the ED for COVID-19 screening. We excluded 793 patients who did not provide informed consent. Two patients had missing wearable device data owing to technical issues. Thus, the data of 468 patients, comprising 277 patients in the training set, 93 in the validation set, and 98 in the test set, were eligible for analysis (Figure 2).
Table 1 shows the baseline characteristics of the enrolled patients.

3.2. Reliability of the Wireless Monitoring Device

Figure 3a shows the agreement between HR data measured using the wireless device and a standard monitoring device (reference standard). The mean difference in HR was −0.685 bpm, with a 95% level of agreement (−15.978–14.607 bpm). Figure 3b shows the agreement between RR data measured using the wireless device and a wall monitor. The mean difference was 1.039 bpm, with a high level of agreement (−5.884–7.963). Bland–Altman plots to compare the Hicardi with a standard measuring method are depicted in Supplementary Figure S2.

3.3. Predictive Performance

Of the 468 enrolled patients, 76 (16.2%) suffering from sepsis needed vasopressors to maintain at least 65 mmHg of mean arterial pressure. In the test set, the clinical condition of 16 of the 98 (16.3%) patients deteriorated, resulting in septic shock. The AUROC value of the fragmented model developed using device data was 0.858 (95% CI, 0.809–0.908), which was higher than that developed using manually recorded data (AUROC, 0.841; 95% CI, 0.789–0.893). The accumulated model developed using device data showed better predictive performance (AUROC, 0.861; 95% CI, 0.811–0.910) than that developed using manually recorded data (AUROC, 0.853; 95% CI, 0.803–0.903) (Table 2).

3.4. Time to Predict Deterioration

In the test set, both fragmented and accumulated models developed using device data accurately predicted septic shock within 6 h in one more patient than those developed using manual data. Regarding the time to predict septic shock when the threshold was set at maximum sensitivity with specificity over 0.9, compared with the model with manual data, the fragmented model with device data predicted septic shock at least 9 h earlier in total, and the accumulated model with device data predicted septic shock 5 h 30 min earlier at the minimum. The prediction time point data for all patients with positive outcomes are shown in Figure 4.

3.5. Feature Importance Scores of the Models

As a result, blood pressure (SBP, DBP), both the value of the current time point and the difference value between the present and the time of the visit, showed high feature importance scores in all four models. Additionally, accumulating the latest HR value was important for prediction since the current HR ranked higher in the model using the device data. On the other hand, in the model using manual data, the HR difference value ranked higher. The ranking of features with high importance scores showed a similar tendency in the accumulated and fragmented models. The feature importance scores of the models are shown in Figure 5.

4. Discussion

Patients in the ED who show signs of instability during the triage stage are assigned to a treatment area with a standard multi-parameter monitoring device used in intensive care units (ICU) and operating rooms [17]. Since the supply of resources in the ED is limited, it is not possible to assign all patients in the ED to this critical treatment area or to provide a standard monitoring device for each patient. In addition, unlike in the ICU, ED physicians cannot continuously manage the admitted patients for an extended period. Moreover, since patients in the ED are frequently moved out of the ED for radiologic evaluation or urgent procedures, the use of standard monitoring devices with limited mobility in the ED is not practical [40,41]. Additionally, patients in the ED are more likely to become unstable than patients in the general ward, even if their initial vital signs are stable; thus, frequent monitoring of patients in the ED is important. Currently, numerous wireless monitoring devices capable of continuously measuring vital signs have been introduced to compensate for the lack of mobility of standard monitoring devices [42,43]. According to the findings of previous studies, the performances of these devices are not inferior to those of standard monitors; in particular, the measurement of HR using wireless devices has been considerably reliable [22,23]. However, studies that assess the clinical utility of these devices in specific medical settings are rare [44]. Furthermore, several studies conducted to confirm the improvement in clinical outcomes with the use of wireless monitoring devices could not clearly demonstrate the superiority of such devices over the manual measurement of vital signs [24]. The present study demonstrated that continuous measurement of HR and RR using a wireless monitoring device enabled the accurate prediction of clinical deterioration and could reduce the time to predict septic shock in actual clinical practice. In addition, we minimised the contamination of the prediction modelling by the noise that inevitably occurred when using the real-time data by developing an algorithm after pre-processing using time series data processing technology [36]. It is known that clinical deterioration of patients can be predicted more precisely when vital signs are measured and recorded frequently and the changes at intervals are considered comprehensively [45,46,47,48,49]. The predictive model developed in the present study confirmed that even though only two vital signs were measured continuously, the predictive value, except sensitivity for clinical deterioration, was increased compared with that of the model developed using intermittent input. In summary, the present study quantitatively confirms the clinical hypothesis that physicians should make a medical decision considering changes in the patient’s condition by acquiring vital sign values more frequently.
Assessment of the risk of potential clinical deterioration in patients who are clinically stable on arrival to the ED is a crucial and challenging task for ED physicians [3,50,51]. In the frequently overcrowded ED, it is practically impossible for medical staff to continuously collect signal data of vital signs manually and use them in real time for the timely detection of clinical deterioration of patients. In addition, the utility of the wearable device can be guaranteed only when there is an algorithm that can accurately predict deterioration by rapidly processing real-time unstructured data generated from the device. Therefore, advanced technologies, such as wearable devices for patient monitoring and the machine learning analysis introduced in this study, can support the complex decision-making task of physicians in clinical settings through the quantification of atypical data, which is currently missing in practice.
Advanced devices that can facilitate patient monitoring have been introduced in previous studies [18,19,20,21]. However, the medical device itself does not change the clinical environment. These technologies can be applied in clinical practice only when the clinical benefit of the device’s performance is clearly identified in a specific clinical scenario [52]. The scenario of the present study meets the needs of ED physicians in clinical practice. Specifically, bedside evaluation of febrile patients who visit the ED has been limited since the outbreak of COVID-19 [16,17]. Accordingly, we utilised a real-world scenario to verify the clinical usefulness of this wireless monitoring device by improving the blind spots in monitoring that may occur owing to the limitations in bedside access by medical staff. In addition, the structure of the prediction model developed was designed to reflect the physician’s point of view and predict clinical deterioration in febrile patients. Since vital signs reflect core information that identifies patients progressing to septic shock [53,54], we used serial values as input information for the model. In particular, the machine learning algorithm used for detecting the earliest time to recognise deterioration was trained to maximise sensitivity in predicting poor outcomes while maintaining high specificity. This was performed considering the characteristics of the ED, where it is critical to not miss deteriorating patients and to initiate treatment within a short period.
The present study had some limitations that should be considered when interpreting its results. Firstly, of the vital sign indicators, only the continuous signal data for HR and RR were used for analysis. The performance of the predictive model can be improved when the continuous signal data of additional indicators are obtained. Secondly, although the data were collected prospectively, the analysis was performed retrospectively; thus, the predictive model did not lead to physician intervention in clinical practice. Therefore, future prospective research is required to confirm whether our predictive model supports the clinical decisions of ED physicians and improves clinical outcomes.

5. Conclusions

The present study is a pilot trial in which a wearable device continuously captured patients’ vital signs that occurred in the clinical field but could not be detected and converted into a useful data source for clinical practice. It showed that continuous monitoring of vital signs using a wearable device could predict clinical deterioration accurately and reduce the time to recognise potential clinical deterioration in stable ED patients with fever. Our results support the application of a wearable device in clinical settings to decrease safety risks due to limited ED resources. Similar studies are needed to secure the use of these digital technologies in various clinical settings in the future.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/s22187054/s1, Table S1: Technical structure of the prediction modelling; Figure S1: The wireless wearable monitoring device and the smartphone application used in the study; Figure S2: Bland–Altman plots to compare the Hicardi with a standard measuring method.

Author Contributions

Conceptualisation, A.C., K.C. and J.H.K.; methodology, K.L. and H.H.; software, K.L. and H.H.; validation, H.H., A.C., and J.H.K.; formal analysis, K.L. and H.H.; investigation, A.C.; resources, K.C.; data curation, A.C.; writing—original draft preparation, A.C. and J.H.K.; writing—review and editing, S.P.C. and J.H.K.; visualisation, A.C., K.L., and H.H.; supervision, K.C.; project administration, J.H.K.; funding acquisition, K.C. and J.H.K. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the Severance Hospital Research fund for Clinical excellence (grant numbers C-2019-0005, C-2022-0010).

Institutional Review Board Statement

The study was conducted in accordance with the Declaration of Helsinki and approved by the Institutional Review Board of Yonsei University College of Medicine, Severance Hospital (no. 1-2019-0058).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

The data presented in this study are available on request from the corresponding author. The data are not publicly available due to the Personal Information Protection Act in Korea.

Acknowledgments

The authors thank Hun Shim, Sung Pil Cho, and Junghwan Park, from MEZOO Co., Ltd., for their technical support.

Conflicts of Interest

The authors declare no conflict of interests. The funder had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.

References

  1. Lee, S.J.; Choi, A.; Ryoo, H.W.; Pak, Y.S.; Kim, H.C.; Kim, J.H. Changes in Clinical Characteristics among Febrile Patients Visiting the Emergency Department before and after the COVID-19 Outbreak. Yonsei Med. J. 2021, 62, 1136–1144. [Google Scholar] [CrossRef] [PubMed]
  2. Yang, M.; Liu, C.; Wang, X.; Li, Y.; Gao, H.; Liu, X.; Li, J. An Explainable Artificial Intelligence Predictor for Early Detection of Sepsis. Crit. Care Med. 2020, 48, e1091–e1096. [Google Scholar] [CrossRef] [PubMed]
  3. Cho, K.J.; Kwon, O.; Kwon, J.M.; Lee, Y.; Park, H.; Jeon, K.H.; Kim, K.H.; Park, J.; Oh, B.H. Detecting Patient Deterioration Using Artificial Intelligence in a Rapid Response System. Crit. Care Med. 2020, 48, e285–e289. [Google Scholar] [CrossRef] [PubMed]
  4. Nemati, S.; Holder, A.; Razmi, F.; Stanley, M.D.; Clifford, G.D.; Buchman, T.G. An Interpretable Machine Learning Model for Accurate Prediction of Sepsis in the ICU. Crit. Care Med. 2018, 46, 547–553. [Google Scholar] [CrossRef] [PubMed]
  5. Buist, M.; Bernard, S.; Nguyen, T.V.; Moore, G.; Anderson, J. Association between clinically abnormal observations and subsequent in-hospital mortality: A prospective study. Resuscitation 2004, 62, 137–141. [Google Scholar] [CrossRef] [PubMed]
  6. Cuthbertson, B.H.; Boroujerdi, M.; McKie, L.; Aucott, L.; Prescott, G. Can physiological variables and early warning scoring systems allow early recognition of the deteriorating surgical patient? Crit. Care Med. 2007, 35, 402–409. [Google Scholar] [CrossRef]
  7. Desautels, T.; Calvert, J.; Hoffman, J.; Jay, M.; Kerem, Y.; Shieh, L.; Shimabukuro, D.; Chettipally, U.; Feldman, M.D.; Barton, C.; et al. Prediction of Sepsis in the Intensive Care Unit With Minimal Electronic Health Record Data: A Machine Learning Approach. JMIR Med. Inform. 2016, 4, e28. [Google Scholar] [CrossRef]
  8. Mayaud, L.; Lai, P.S.; Clifford, G.D.; Tarassenko, L.; Celi, L.A.; Annane, D. Dynamic data during hypotensive episode improves mortality predictions among patients with sepsis and hypotension. Crit. Care Med. 2013, 41, 954–962. [Google Scholar] [CrossRef]
  9. Beckett, D.; Gordon, C.; Paterson, R.; Chalkley, S.; Macleod, D.; Bell, D. Assessment of clinical risk in the out of hours hospital prior to the introduction of Hospital at Night. Acute Med. 2009, 8, 33–38. [Google Scholar] [CrossRef]
  10. Fuhrmann, L.; Lippert, A.; Perner, A.; Ostergaard, D. Incidence, staff awareness and mortality of patients at risk on general wards. Resuscitation 2008, 77, 325–330. [Google Scholar] [CrossRef]
  11. Calzavacca, P.; Licari, E.; Tee, A.; Egi, M.; Downey, A.; Quach, J.; Haase-Fielitz, A.; Haase, M.; Bellomo, R. The impact of Rapid Response System on delayed emergency team activation patient characteristics and outcomes—A follow-up study. Resuscitation 2010, 81, 31–35. [Google Scholar] [CrossRef] [PubMed]
  12. Taenzer, A.H.; Pyke, J.B.; McGrath, S.P. A review of current and emerging approaches to address failure-to-rescue. Anesthesiology 2011, 115, 421–431. [Google Scholar] [CrossRef] [PubMed]
  13. DeVita, M.A.; Smith, G.B.; Adam, S.K.; Adams-Pizarro, I.; Buist, M.; Bellomo, R.; Bonello, R.; Cerchiari, E.; Farlow, B.; Goldsmith, D.; et al. “Identifying the hospitalised patient in crisis”—A consensus conference on the afferent limb of rapid response systems. Resuscitation 2010, 81, 375–382. [Google Scholar] [CrossRef] [PubMed]
  14. Hammer, J.A.; Jones, T.L.; Brown, S.A. Rapid response teams and failure to rescue: One community’s experience. J. Nurs. Care Qual. 2012, 27, 352–358. [Google Scholar] [CrossRef]
  15. Weenk, M.; Koeneman, M.; van de Belt, T.H.; Engelen, L.; van Goor, H.; Bredie, S.J.H. Wireless and continuous monitoring of vital signs in patients at the general ward. Resuscitation 2019, 136, 47–53. [Google Scholar] [CrossRef] [PubMed]
  16. Wang, J.; Zong, L.; Zhang, J.; Sun, H.; Harold Walline, J.; Sun, P.; Xu, S.; Li, Y.; Wang, C.; Liu, J.; et al. Identifying the effects of an upgraded ‘fever clinic’ on COVID-19 control and the workload of emergency department: Retrospective study in a tertiary hospital in China. BMJ Open 2020, 10, e039177. [Google Scholar] [CrossRef] [PubMed]
  17. Choi, A.; Kim, H.Y.; Cho, A.; Noh, J.; Park, I.; Chung, H.S. Efficacy of a four-tier infection response system in the emergency department during the coronavirus disease-2019 outbreak. PLoS ONE 2021, 16, e0256116. [Google Scholar] [CrossRef]
  18. Breteler, M.J.M.; KleinJan, E.; Numan, L.; Ruurda, J.P.; Van Hillegersberg, R.; Leenen, L.P.H.; Hermans, M.; Kalkman, C.J.; Blokhuis, T.J. Are current wireless monitoring systems capable of detecting adverse events in high-risk surgical patients? A descriptive study. Injury 2020, 51 (Suppl. 2), S97–S105. [Google Scholar] [CrossRef]
  19. Downey, C.; Randell, R.; Brown, J.; Jayne, D.G. Continuous Versus Intermittent Vital Signs Monitoring Using a Wearable, Wireless Patch in Patients Admitted to Surgical Wards: Pilot Cluster Randomized Controlled Trial. J. Med. Internet Res. 2018, 20, e10802. [Google Scholar] [CrossRef]
  20. Sun, L.; Joshi, M.; Khan, S.N.; Ashrafian, H.; Darzi, A. Clinical impact of multi-parameter continuous non-invasive monitoring in hospital wards: A systematic review and meta-analysis. J. R. Soc. Med. 2020, 113, 217–224. [Google Scholar] [CrossRef]
  21. Khanna, A.K.; Hoppe, P.; Saugel, B. Automated continuous noninvasive ward monitoring: Future directions and challenges. Crit. Care 2019, 23, 194. [Google Scholar] [CrossRef] [PubMed]
  22. Breteler, M.J.M.; KleinJan, E.J.; Dohmen, D.A.J.; Leenen, L.P.H.; van Hillegersberg, R.; Ruurda, J.P.; van Loon, K.; Blokhuis, T.J.; Kalkman, C.J. Vital Signs Monitoring with Wearable Sensors in High-risk Surgical Patients: A Clinical Validation Study. Anesthesiology 2020, 132, 424–439. [Google Scholar] [CrossRef] [PubMed]
  23. Youssef Ali Amer, A.; Wouters, F.; Vranken, J.; de Korte-de Boer, D.; Smit-Fun, V.; Duflot, P.; Beaupain, M.H.; Vandervoort, P.; Luca, S.; Aerts, J.M.; et al. Vital Signs Prediction and Early Warning Score Calculation Based on Continuous Monitoring of Hospitalised Patients Using Wearable Technology. Sensors 2020, 20, 6593. [Google Scholar] [CrossRef] [PubMed]
  24. Leenen, J.P.L.; Leerentveld, C.; van Dijk, J.D.; van Westreenen, H.L.; Schoonhoven, L.; Patijn, G.A. Current Evidence for Continuous Vital Signs Monitoring by Wearable Wireless Devices in Hospitalized Adults: Systematic Review. J. Med. Internet Res. 2020, 22, e18636. [Google Scholar] [CrossRef]
  25. Weenk, M.; Bredie, S.J.; Koeneman, M.; Hesselink, G.; van Goor, H.; van de Belt, T.H. Continuous Monitoring of Vital Signs in the General Ward Using Wearable Devices: Randomized Controlled Trial. J. Med. Internet Res. 2020, 22, e15471. [Google Scholar] [CrossRef]
  26. Jordan, M.I.; Mitchell, T.M. Machine learning: Trends, perspectives, and prospects. Science 2015, 349, 255–260. [Google Scholar] [CrossRef]
  27. Muralitharan, S.; Nelson, W.; Di, S.; McGillion, M.; Devereaux, P.J.; Barr, N.G.; Petch, J. Machine Learning-Based Early Warning Systems for Clinical Deterioration: Systematic Scoping Review. J. Med. Internet Res. 2021, 23, e25187. [Google Scholar] [CrossRef]
  28. Mendo, I.R.; Marques, G.; de la Torre Diez, I.; Lopez-Coronado, M.; Martin-Rodriguez, F. Machine Learning in Medical Emergencies: A Systematic Review and Analysis. J. Med. Syst. 2021, 45, 88. [Google Scholar] [CrossRef]
  29. Brekke, I.J.; Puntervoll, L.H.; Pedersen, P.B.; Kellett, J.; Brabrand, M. The value of vital sign trends in predicting and monitoring clinical deterioration: A systematic review. PLoS ONE 2019, 14, e0210875. [Google Scholar] [CrossRef]
  30. Shafaf, N.; Malek, H. Applications of Machine Learning Approaches in Emergency Medicine; a Review Article. Arch. Acad. Emerg. Med. 2019, 7, 34. [Google Scholar]
  31. Stewart, J.; Sprivulis, P.; Dwivedi, G. Artificial intelligence and machine learning in emergency medicine. Emerg. Med. Australas. 2018, 30, 870–874. [Google Scholar] [CrossRef] [PubMed]
  32. U.S. Food and Drug Administration Center for Devices and Radiological Health. Content of Premarket Submissions for Management of Cybersecurity in Medical Devices; U.S. Food and Drug Administration Center for Devices and Radiological Health: Silver Spring, MD, USA, 2014. Available online: https://www.fda.gov/regulatory-information/search-fda-guidance-documents/content-premarket-submissions-management-cybersecurity-medical-devices (accessed on 19 February 2020).
  33. International Medical Device Regulators Forum. Principles and Practices for Medical Device Cybersecurity; International Medical Device Regulators Forum: Silver Spring, MD, USA, 2020; Available online: https://www.imdrf.org/sites/default/files/docs/imdrf/final/technical/imdrf-tech-200318-pp-mdc-n60.pdf (accessed on 19 February 2020).
  34. Song, M.H.; Cho, S.P.; Kim, W.; Lee, K.J. New real-time heartbeat detection method using the angle of a single-lead electrocardiogram. Comput. Biol. Med. 2015, 59, 73–79. [Google Scholar] [CrossRef] [PubMed]
  35. An Introductory Resource Guide for Implementing the HIPAA Security Rule; US Department of Commerce: Gaithersburg, MD, USA, 2008. Available online: https://csrc.nist.gov/publications/detail/sp/800-66/rev-1/final (accessed on 1 October 2008).
  36. Khodadad, D.; Nordebo, S.; Muller, B.; Waldmann, A.; Yerworth, R.; Becher, T.; Frerichs, I.; Sophocleous, L.; van Kaam, A.; Miedema, M.; et al. Optimized breath detection algorithm in electrical impedance tomography. Physiol. Meas. 2018, 39, 094001. [Google Scholar] [CrossRef] [PubMed]
  37. Yamashita, R.; Nishio, M.; Do, R.K.G.; Togashi, K. Convolutional neural networks: An overview and application in radiology. Insights Imaging 2018, 9, 611–629. [Google Scholar] [CrossRef]
  38. Muraki, R.; Teramoto, A.; Sugimoto, K.; Sugimoto, K.; Yamada, A.; Watanabe, E. Automated detection scheme for acute myocardial infarction using convolutional neural network and long short-term memory. PLoS ONE 2022, 17, e0264002. [Google Scholar] [CrossRef]
  39. Singer, M.; Deutschman, C.S.; Seymour, C.W.; Shankar-Hari, M.; Annane, D.; Bauer, M.; Bellomo, R.; Bernard, G.R.; Chiche, J.D.; Coopersmith, C.M.; et al. The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA 2016, 315, 801–810. [Google Scholar] [CrossRef]
  40. Darwish, A.; Hassanien, A.E. Wearable and implantable wireless sensor network solutions for healthcare monitoring. Sensors 2011, 11, 5561–5595. [Google Scholar] [CrossRef]
  41. Majumder, S.; Mondal, T.; Deen, M.J. Wearable Sensors for Remote Health Monitoring. Sensors 2017, 17, 130. [Google Scholar] [CrossRef]
  42. Michard, F.; Gan, T.J.; Kehlet, H. Digital innovations and emerging technologies for enhanced recovery programmes. Br. J. Anaesth. 2017, 119, 31–39. [Google Scholar] [CrossRef]
  43. Fung, E.; Jarvelin, M.R.; Doshi, R.N.; Shinbane, J.S.; Carlson, S.K.; Grazette, L.P.; Chang, P.M.; Sangha, R.S.; Huikuri, H.V.; Peters, N.S. Electrocardiographic patch devices and contemporary wireless cardiac monitoring. Front. Physiol. 2015, 6, 149. [Google Scholar] [CrossRef]
  44. Diomidous, M.; Dalamaga, M.; Nikolopoulos, M.; Tzortzis, E.; Stratigou, T.; Antonakos, G.; Karampela, I. Wireless Monitoring Through Wearable Devices in the ICU: Are We Close? Stud. Health Technol. Inform. 2020, 272, 175–178. [Google Scholar] [CrossRef]
  45. Downey, C.L.; Tahir, W.; Randell, R.; Brown, J.M.; Jayne, D.G. Strengths and limitations of early warning scores: A systematic review and narrative synthesis. Int. J. Nurs. Stud. 2017, 76, 106–119. [Google Scholar] [CrossRef] [PubMed]
  46. Clifton, L.; Clifton, D.A.; Pimentel, M.A.; Watkinson, P.J.; Tarassenko, L. Predictive monitoring of mobile patients by combining clinical observations with data from wearable sensors. IEEE J. Biomed. Health Inform. 2014, 18, 722–730. [Google Scholar] [CrossRef]
  47. Appelboom, G.; Camacho, E.; Abraham, M.E.; Bruce, S.S.; Dumont, E.L.; Zacharia, B.E.; D’Amico, R.; Slomian, J.; Reginster, J.Y.; Bruyere, O.; et al. Smart wearable body sensors for patient self-assessment and monitoring. Arch. Public Health 2014, 72, 28. [Google Scholar] [CrossRef] [PubMed]
  48. Credland, N.; Dyson, J.; Johnson, M.J. What are the patterns of compliance with Early Warning Track and Trigger Tools: A narrative review. Appl. Nurs. Res. 2018, 44, 39–47. [Google Scholar] [CrossRef] [PubMed]
  49. Sahandi, R.; Noroozi, S.; Roushan, G.; Heaslip, V.; Liu, Y. Wireless technology in the evolution of patient monitoring on general hospital wards. J. Med. Eng. Technol. 2010, 34, 51–63. [Google Scholar] [CrossRef]
  50. Kim, J.; Chang, H.; Kim, D.; Jang, D.H.; Park, I.; Kim, K. Machine learning for prediction of septic shock at initial triage in emergency department. J. Crit. Care 2020, 55, 163–170. [Google Scholar] [CrossRef]
  51. Hong, S.; Lee, S.; Lee, J.; Cha, W.C.; Kim, K. Prediction of Cardiac Arrest in the Emergency Department Based on Machine Learning and Sequential Characteristics: Model Development and Retrospective Clinical Validation Study. JMIR Med. Inform. 2020, 8, e15932. [Google Scholar] [CrossRef]
  52. Kim, J.H.; Han, S.G.; Cho, A.; Shin, H.J.; Baek, S.E. Effect of deep learning-based assistive technology use on chest radiograph interpretation by emergency department physicians: A prospective interventional simulation-based study. BMC Med. Inform. Decis. Mak. 2021, 21, 311. [Google Scholar] [CrossRef]
  53. Padilla, R.M.; Mayo, A.M. Clinical deterioration: A concept analysis. J. Clin. Nurs. 2018, 27, 1360–1368. [Google Scholar] [CrossRef]
  54. Churpek, M.M.; Yuen, T.C.; Edelson, D.P. Predicting clinical deterioration in the hospital: The impact of outcome selection. Resuscitation 2013, 84, 564–568. [Google Scholar] [CrossRef] [PubMed] [Green Version]
Sensors 22 07054 g001 550
Figure 1. Technical structure of the prediction modelling process.
Figure 1. Technical structure of the prediction modelling process.
Sensors 22 07054 g001
Sensors 22 07054 g002 550
Figure 2. Flowchart of patient selection for the study.
Figure 2. Flowchart of patient selection for the study.
Sensors 22 07054 g002
Sensors 22 07054 g003 550
Figure 3. Agreement between the measurements from the wireless monitoring device and manual measurements. (a) Agreement between heart rate measurements. (b) Agreement between respiratory rate measurements.
Figure 3. Agreement between the measurements from the wireless monitoring device and manual measurements. (a) Agreement between heart rate measurements. (b) Agreement between respiratory rate measurements.
Sensors 22 07054 g003
Sensors 22 07054 g004 550
Figure 4. Prediction timepoint data for all patients with positive outcomes. (a) Fragmented model. (b) Accumulated model.
Figure 4. Prediction timepoint data for all patients with positive outcomes. (a) Fragmented model. (b) Accumulated model.
Sensors 22 07054 g004
Sensors 22 07054 g005a 550Sensors 22 07054 g005b 550
Figure 5. Feature importance scores of the models. (a) Fragmented model with manual data (left), with device data (right). (b) Accumulated model with manual data (left), with device data (right). SBP, systolic blood pressure; DBP, diastolic blood pressure, HR, heart rate; RR, respiratory rate; GCS, Glasgow coma scale; BT, body temperature; Sat, oxygen saturation; diff, difference; seq, sequence.
Figure 5. Feature importance scores of the models. (a) Fragmented model with manual data (left), with device data (right). (b) Accumulated model with manual data (left), with device data (right). SBP, systolic blood pressure; DBP, diastolic blood pressure, HR, heart rate; RR, respiratory rate; GCS, Glasgow coma scale; BT, body temperature; Sat, oxygen saturation; diff, difference; seq, sequence.
Sensors 22 07054 g005aSensors 22 07054 g005b
Table
Table 1. Baseline characteristics of the enrolled patients.
Table 1. Baseline characteristics of the enrolled patients.
CharacteristicsValue
sex (male)202 (44.20)
age (years)56.87 ± 18.80
duration of device use (min) 300.27 ± 100.03
vital signs on arrival
systolic blood pressure (mmHg) 127.36 ± 22.06
diastolic blood pressure (mmHg)75.72 ± 11.46
heart rate (bpm)108.59 ± 18.52
respiratory rate (bpm)18.16 ± 2.71
body temperature (°C)38.64 ± 0.57
oxygen saturation (%) 96.77 ± 2.47
Glasgow Coma Scale score14.98 ± 0.23
use of vasopressors 85 (18.60)
Values are expressed as number (%) or mean ± standard deviation.
Table
Table 2. Predictive performance of each model.
Table 2. Predictive performance of each model.
ModelDataAUROC
(95% CI)		AUPRC
(95% CI)		Sensitivity
(95% CI)		Specificity
(95% CI)
fragmented
model		manual data0.841
(0.789–0.893)		0.699
(0.598–0.783)		0.731
(0.633–0.811)		0.836
(0.796–0.870)
device data0.858
(0.809–0.908)		0.761
(0.664–0.837)		0.710
(0.611–0.792)		0.936
(0.907–0.956)
accumulated
model		manual data0.853
(0.803–0.903)		0.679
(0.578–0.766)		0.710
(0.611–0.792)		0.841
(0.802–0.874)
device data0.861
(0.811–0.910)		0.689
(0.588–0.775)		0.699
(0.599–0.783)		0.880
(0.844–0.908)
AUROC, area under the receiver operating characteristic curve; AUPRC, area under the precision–recall curve; CI, confidence interval.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Choi, A.; Chung, K.; Chung, S.P.; Lee, K.; Hyun, H.; Kim, J.H. Advantage of Vital Sign Monitoring Using a Wireless Wearable Device for Predicting Septic Shock in Febrile Patients in the Emergency Department: A Machine Learning-Based Analysis. Sensors 2022, 22, 7054. https://doi.org/10.3390/s22187054

AMA Style

Choi A, Chung K, Chung SP, Lee K, Hyun H, Kim JH. Advantage of Vital Sign Monitoring Using a Wireless Wearable Device for Predicting Septic Shock in Febrile Patients in the Emergency Department: A Machine Learning-Based Analysis. Sensors. 2022; 22(18):7054. https://doi.org/10.3390/s22187054

Chicago/Turabian Style

Choi, Arom, Kyungsoo Chung, Sung Phil Chung, Kwanhyung Lee, Heejung Hyun, and Ji Hoon Kim. 2022. "Advantage of Vital Sign Monitoring Using a Wireless Wearable Device for Predicting Septic Shock in Febrile Patients in the Emergency Department: A Machine Learning-Based Analysis" Sensors 22, no. 18: 7054. https://doi.org/10.3390/s22187054

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop