Next Article in Journal
HSP-UNet: An Accuracy and Efficient Segmentation Method for Carbon Traces of Surface Discharge in the Oil-Immersed Transformer
Previous Article in Journal
Indeterministic Data Collection in UAV-Assisted Wide and Sparse Wireless Sensor Network
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Sensors
Volume 24
Issue 19
10.3390/s24196497
sensors-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Review

Discrepancies between Promised and Actual AI Capabilities in the Continuous Vital Sign Monitoring of In-Hospital Patients: A Review of the Current Evidence

by
Nikolaj Aagaard
1,*,
Eske K. Aasvang
2,3,† and
Christian S. Meyhoff
1,3,†
1
Department of Anaesthesia and Intensive Care, Copenhagen University Hospital—Bispebjerg and Frederiksberg, 2400 Copenhagen, Denmark
2
Department of Anaesthesia, Centre for Cancer and Organ Diseases, Copenhagen University Hospital—Rigshospitalet, 2100 Copenhagen, Denmark
3
Department of Clinical Medicine, University of Copenhagen, 2200 Copenhagen, Denmark
*
Author to whom correspondence should be addressed.
†
These authors contributed equally to this work.
Sensors 2024, 24(19), 6497; https://doi.org/10.3390/s24196497
Submission received: 3 September 2024 / Revised: 1 October 2024 / Accepted: 2 October 2024 / Published: 9 October 2024
(This article belongs to the Special Issue Intelligent Measurements and Interpretation of Wireless Systems for Continuous Vital Sign Monitoring)
Browse Figures
Review Reports Versions Notes

Abstract

:
Continuous vital sign monitoring (CVSM) with wireless sensors in general hospital wards can enhance patient care. An artificial intelligence (AI) layer is crucial to allow sensor data to be managed by clinical staff without over alerting from the sensors. With the aim of summarizing peer-reviewed evidence for AI support in CVSM sensors, we searched PubMed and Embase for studies on adult patients monitored with CVSM sensors in general wards. Peer-reviewed evidence and white papers on the official websites of CVSM solutions were also included. AI classification was based on standard definitions of simple AI, as systems with no memory or learning capabilities, and advanced AI, as systems with the ability to learn from past data to make decisions. Only studies evaluating CVSM algorithms for improving or predicting clinical outcomes (e.g., adverse events, intensive care unit admission, mortality) or optimizing alarm thresholds were included. We assessed the promised level of AI for each CVSM solution based on statements from the official product websites. In total, 467 studies were assessed; 113 were retrieved for full-text review, and 26 studies on four different CVSM solutions were included. Advanced AI levels were indicated on the websites of all four CVSM solutions. Five studies assessed algorithms with potential for applications as advanced AI algorithms in two of the CVSM solutions (50%), while 21 studies assessed algorithms with potential as simple AI in all four CVSM solutions (100%). Evidence on algorithms for advanced AI in CVSM is limited, revealing a discrepancy between promised AI levels and current algorithm capabilities.

1. Introduction

Delayed detection of patient deterioration in hospital wards significantly contributes to in-hospital mortality and intensive care unit (ICU) admissions [1,2]. As a consequence, routine manual vital sign monitoring at fixed intervals, such as the National Early Warning Score (NEWS) has been implemented across countries [3,4]. However, no documented impact of NEWS on morbidity and mortality has been observed [5,6]. This may be partly due to deterioration occurring between manual observations, where continuous vital sign monitoring (CVSM) with wireless sensors can enable the early detection of patient deterioration, which is important for timely treatment and prevention of further decline [7,8,9]. Clinical implementation of CVSM may reduce the risk of serious adverse events (SAEs) [10,11], as it has been documented to detect more vital sign deviations [12,13]. NEWS-based algorithms have demonstrated an inconsistent ability to predict subsequent SAEs when applied to CVSM [14,15]. Identifying the optimal thresholds for various SAE subtypes is thought to be crucial and potentially attainable through advanced data analysis and artificial intelligence (AI) [16]. Thus, AI implementation to sensor data may enhance sensitivity and specificity, ensuring that CVSM sensors are effective, and assist clinical staff without causing alarm fatigue or unintended adverse events (AE) from overtreatment and unnecessary diagnostic procedures [16]. AI encompasses a wide range of technologies designed to simulate or replicate human cognitive functions, like problem solving and decision making, and the foundation of AI algorithms varies widely in terms of data and analysis methods [17]. Despite significant advancements evidenced in publications, the implementation of these technologies in the medical field remains slow and complex [18,19] and a discrepancy between what is promised on CVSM producers websites and what is documented in the scientific literature may therefore exist. It is crucial to validate and report the clinical impact of AI algorithms used for CVSM sensors in peer-reviewed studies to ensure transparency and enable decision makers to determine when and what to implement [16]. We hypothesized that CVSM producers are promising AI advancements beyond what has been evaluated in peer-reviewed evidence. This review aims to summarize current research on CVSM sensors in hospitalized patients with potential for application as AI algorithms within state-of-the-art CVSM solutions.

2. Materials and Methods

2.1. Litterature Search

We conducted an electronic literature search in PubMed and Embase (Ovid). We searched for articles published up to 30 May 2024. No limits were set for publication date in PubMed, while studies were limited to those published after 2010 in the Embase search. The search was conducted using Medical Subject Headings in PubMed. No limitations were applied to the search in PubMed. In Embase, the following limits were applied: human, English language, study type (clinical trial, randomized controlled trial, controlled clinical trial, or multicenter study), adult (age 18–64 years), and elderly (age > 64 years). Based on title and abstract, the search results were screened and assessed for eligibility with Rayyan (http://rayyan.qcri.org) [20]. The complete search strategy, including terms, is detailed in Table 1. The official websites of the CVSM solutions, identified through the literature search, were also examined for peer-reviewed evidence and other relevant material, including white papers and reports [21,22,23,24,25]. If a CVSM study was excluded at full-text review, the official website of the solution was still evaluated, and any relevant evidence was retrieved and assessed accordingly. Peer-reviewed evidence from these websites was retrieved via PubMed or Embase and assessed. Non-peer-reviewed evidence was retrieved and evaluated in the same way as peer-reviewed evidence.

2.2. Study Selection

The inclusion criteria in this review were based on the following PICO elements:

2.2.1. Population

We included studies on adult patients (age ≥ 18 years), admitted to general medical or surgical wards. Studies focusing on ICUs, post-anesthesia care units, and other high-dependency units were excluded, as CVSM is already standard in these settings, and the potential AI algorithms would likely be different due to a higher staff–patient ratio in the handling of alerts. Studies on non-hospitalized patients were also excluded, as these patients are typically not monitored at all. Studies on invasive CVSM sensors were excluded as they are not typically practical or standard in general wards.

2.2.2. Intervention

CVSM was defined as non-invasive recordings of one or more vital signs continuously at a high sampling frequency (e.g., less than five minutes) with one or more wireless biosensors. This review included studies measuring traditional vital signs, including respiratory rate (RR), peripheral oxygen saturation (SpO2), heart rate (HR), temperature (Tp), and/or blood pressure (BP), with BP considered semi-continuous with intermittent measurements of less than an hour with the same wireless device applied throughout the monitoring period. Only sensors intended for medical use were included, i.e., studies on consumer-grade fitness trackers were excluded. Only studies of CVSM solutions claiming on their official websites to be supported by AI or algorithms that generate alarms for patient deterioration or complications were included.

2.2.3. Outcome

The outcome of interest was the classification of AI level as reported on product websites and reported in peer-reviewed articles. Several definitions and classifications of AI exist, and for this study, AI classification was based on evaluation of the statistical methods or algorithms used to predict complications or set alarms within the CVSM solution, with the level of AI determined by its ability to replicate human capabilities [26]. Simple AI (level 1) corresponds to reactive machines that have no memory or learning capabilities, thus restricting these systems or algorithms to automated responses for a limited number of parameters, specifically when crossing predefined or fixed vital sign thresholds [26]. Fixed vital sign thresholds that were adjusted during the study based on clinical staff feedback were also considered simple AI. Only studies evaluating thresholds for improving clinical outcomes, predicting complications or specific outcomes (e.g., ICU admission, mortality, readmission), and optimizing alarm thresholds were included. Studies quantifying the ability of CVSM sensors to detect deviations compared to routine monitoring or describing deviations from fixed thresholds without association to clinical outcomes were excluded, as they do not represent predictive algorithms with AI potential. If the study analyzed and described vital sign thresholds or patterns in specific cases prior to a complication, such as sepsis or myocardial infarction, it was graded as simple AI, provided that no adaptive algorithms were used for analysis.
Advanced AI (level 2) corresponds to limited memory models that can learn from past data to make decisions, thus they have adaptive capabilities and the ability to modify and adjust thresholds based on previous vital sign measurements. Studies utilizing machine learning (ML) methods, a subset of AI focused on developing algorithms and statistical models that learn from complex patterns and make data-based predictions [27], were classified as advanced AI. Only studies evaluating ML algorithms used to predict complications or set alarms were classified as advanced AI. ML algorithms used to improve the accuracy of vital sign measurements were not included as advanced AI. As the current most-advanced AI applications are encompassed by limited memory models, this level is considered advanced AI, and no levels beyond this were classified. The different levels of AI are illustrated in Figure 1.
Studies with the following designs were included: randomized controlled trials (RCT), cluster RCTs, and observational studies. Non-peer-reviewed materials, such as white papers, reports, conference articles, and abstracts were also included. All studies on CVSM in adult patients admitted to general hospital wards were retrieved for full-text review, as relevant outcomes may not have been reported in the abstracts.

2.3. Data Extraction

We extracted characteristics of each monitoring solution (country of origin, devices used, regulatory status (CE/FDA), study design, type of included patients, and vital signs measured) and the promised AI capability level reported on the official websites. The method of analysis in studies assessing the prediction of complications or filtering of alarms was extracted and the potential AI level was evaluated and graded. The performance metrics of the algorithm were also retrieved and reported, including quantification of predictive ability, measures for improving clinical outcomes, and the time spent outside thresholds for patients with complications compared to those without.

3. Results

In total, 391 studies were identified through a search of the databases. Including peer-reviewed evidence from official websites, a total of 467 studies were considered, with 113 studies retrieved for full-text review, including nine different CVSM solutions. In total, 26 peer-reviewed studies of four different CVSM solutions supported by AI (ViSi Mobile® (VM), WARD 24/7, Sensium Vitals® (SV), and Biobeat) met the inclusion criteria (Figure 2).
The characteristics of each monitoring solution are presented in Table 2. Evidence from the official website of Isansys Lifecare was not evaluated independently, as it had been investigated in adult patients admitted to general wards through its integration with WARD 24/7 [28]. Four CVSM solutions identified through the literature search were not evaluated, as they do not currently claim AI or predictive algorithm support, or no eligible studies were identified [29,30,31,32,33]. The promised levels of AI compared to the highest potential AI levels of algorithms assessed in peer-reviewed studies are presented in Table 3.
A total of five studies assessing algorithms with the potential for advanced AI application have been published on two CVSM solutions (50%): four from WARD 24/7 [34,35,36,37] and one from SV [38]. A total of 21 studies assessing algorithms with the potential for simple AI applications have been published on all CVSM solutions (100%): six from VM [11,39,40,41,42,43], seven from WARD 24/7 [13,14,15,44,45,46,47], six from SV [48,49,50,51,52,53], and two from Biobeat [54,55] (Figure 3). The methods and reported outcomes of the included studies varied too much for a meta-analysis. Specifically, performance metrics were reported differently across studies, and even within subgroups of studies using similar metrics, the algorithms significantly differed and were modified during the studies. Therefore, the findings were presented descriptively. The following sections present the evidence for each solution.

3.1. Visi Mobile ®

The VM Surveillance Monitoring System (Sotera Digital Health, Carlsbad, CA, USA) is a CE-marked and FDA-cleared wireless wrist-worn device for monitoring vital signs, utilizing ML algorithms designed to identify patient deterioration and promptly alert hospital staff, while minimizing irrelevant alarms [21]. VM sends information directly to the medical records and the hospital staff’s phones, when connected to the hospital’s network (Table 2 and Table 3) [21].

3.1.1. Improving Outcomes

A recent observational study monitored adult patients admitted to general surgical wards at a tertiary health care center (Atrium Health Wake Forest Baptist Medical Center) over a 2-year period (1 January 2018–31 December 2019). A propensity-matched analysis was performed and found that intermittent monitoring (n = 12,345) was associated with increased risk of mortality and ICU admission (odds ratio (OR) 3.42, 95% confidence interval (CI): 3.19–3.67), heart failure (OR 1.48, 95%CI: 1.21–1.81), myocardial infarction (OR 3.87, 95%CI: 2.71–5.71), and acute kidney injury (OR 1.32, 95%CI: 1.09–1.57) compared with CVSM (n = 7955) [11]. The odds of rapid response team (RRT) activation were similar in both groups (OR 0.86, 95%CI: 0.79–1.06) [11].
A recent before-and-after study of both surgical and medical patients from a university hospital in the Netherlands included 2303 and 2466 patients in the intervention and baseline cohort, respectively. Patients monitored with the VM system had significantly fewer unplanned ICU admissions with 3.4% vs. 2.3% (p = 0.03) and fewer RRT calls with 4.3% vs. 3.1% (p = 0.02). No differences in hospital length of stay or in-hospital mortality were observed [43]. Another before-and-after study found that the VM system, combined with modified early warning score, significantly reduced RRT activations from 9.6% in the baseline period (February to April 2016) to 4.8% in the intervention period (February to April 2018) (p < 0.01) [42]. An additional before-and-after study conducted from December 2015 to March 2016, in the orthopedic and trauma general care ward, found a significant reduction in complication incident rates of 34.3 per 1000 patient days and 9.6 per 1000 patient days for the preintervention and intervention group, respectively (p < 0.05) [41]. The frequencies of RRT calls were similar in the preintervention and intervention groups, while ICU transfers and failure to rescue were insignificantly reduced from 22 (5.4%) to 19 (4.5%) and 2 (0.46%) to 0, respectively [41].

3.1.2. Alarm Filtering

A prospective, observational pilot study in neurological and neurosurgical patients evaluated the tolerability of alarm thresholds based on clinical staff feedback on 736 patients included for analysis [39]. Predefined vital sign thresholds were filtered for artifacts, and the alarm thresholds were adjusted over four iterations, aiming for two alarms per patient per day. An acceptable rate of 2.3 alarms per patient per day was achieved in the final stage of implementation with modified thresholds and durations (Table 4). The study reported maintaining patient safety with fewer RRT calls, ICU transfers, and unplanned deaths compared to the five months before implementation of the VM system, with the latter two not being statistically significant [39]. A different study aimed to reduce alarm fatigue and optimize alarm settings in CVSM using data from the VM system, analyzing 94,575 h from 3430 patients stored in cloud databases. By simulating various alarm thresholds and delays with cloud computing tools based on data distribution, the authors manually assessed alarm rates to potentially help manage alarm burden [40]. A white paper described alarm development within the VM system using the cloud database. Client-specific alarm data are processed through various “what-if” configurations, resulting in an alarm performance table that includes parameters, delays, vital sign thresholds, and corresponding alarms per patient per day [56]. An example of this alarm modification form is presented in Supplementary Table S1.

3.2. WARD 24/7

Wireless Assessment of Respiratory and circulatory Distress (WARD) (WARD 247 ApS, Copenhagen, Denmark) is a CE-marked clinical support system (CSS) with continuous 24 h vital sign monitoring of high-risk medical and surgical patients using advanced clinically modeled algorithms to monitor patients with real-time alarms [22,23]. Vital sign data are collected with the clinically validated CE- marked and FDA-approved Isansys Lifetouch and Lifetemp (Isansys Lifecare, Oxfordshire, UK), Nonin WristOx 3150 (Nonin Medical Inc., Plymouth, MN, USA), Meditech BlueBP-05 (Meditech Ltd., Budapest, Hungary), and A&D TM-2441 (A&D COMPANY, Tokyo, Japan) (Table 2 and Table 3) [22,23].

3.2.1. Prediction of Serious Adverse Events and Deterioration

Studies with the WARD CSS have assessed the predictive ability of traditional vital signs for subsequent SAEs, defined according to the International Conference on Harmonization–Good Clinical Practice Guideline [57]. Two recent prospective observational trials, utilizing the WARD CSS assessed the predictive ability of predefined vital sign threshold, based on the NEWS, for subsequent SAEs in surgical and medical patients, respectively [14,15]. Both studies analyzed the duration and frequency of vital sign deviations and investigated the association between 24 h exposure periods and SAEs. Apart from bradypnea in the medical patients, both studies found no statistically significant association between the frequency or total duration of vital sign abnormalities and subsequent occurrence of SAEs [14,15]. For specific types of SAEs, one study in high-risk surgical and medical patients found a significant association between the accumulated duration of SpO2 < 85% and subsequent myocardial injury, with a 14.2 min longer deviation in those with upcoming myocardial injury compared to those without (95% CI: −4.7 to 33.1 min, p = 0.005) [44]. Other thresholds for SpO2 (<88% and <80%), HR (>110 and >130 beats/minute), and RR (>24 and >30 per minute) were also significantly associated with myocardial injury [44]. Another study in patients undergoing major gastrointestinal surgery found a higher detection rate of post-operative atrial fibrillation (6.5%, 95% CI: 4.5–9.4) based on CVSM compared to clinical staff observations (3.5%; 95% CI: 1.9–5.8) [47]. No significant association between preceding vital sign deviations and postoperative atrial fibrillation was identified [47]. A prospective observational study analyzed data from 90 patients with suspected COVID-19 infection. The study found that patients who died within 30 days or were admitted to the ICU had significantly longer durations outside the thresholds: RR > 21, RR > 24, SpO2 < 80%, SpO2 < 85%, and HR > 110 bpm [13]. A study examined the association between deviating vital signs prior to discharge and the risk of readmission. SpO2 < 88% for at least 10 min was observed in 66% of readmitted patients and 62% of non-readmitted patients (p = 0.62), while SpO2 < 85% occurred in 58% of readmitted and 52% of non-readmitted patients (p = 0.50) for at least 5 min. The sustained deviation of at least one vital sign was detected in 90% of readmitted and 85% of non-readmitted patients (p = 0.20). Deviating vital signs prior to discharge were frequent but not associated with increased readmission risk within 30 days [45].

3.2.2. Alarm Filtering

A recent study evaluated the effect of alarm filtering on data from 716 medical and surgical patients included from four prospective observational trials. Various NEWS thresholds were applied to trigger alarms, which were then filtered by artifact removal combined with duration-specific criteria and artifact removal only. This significantly reduced alarms per patient per day from a median of 137 [IQR 87–188] to 101 [IQR 56–160] with only artifact removal, and 19 [IQR 9–34] when combining artifact removal with a duration-specific criterium (Table 4) [46].

3.2.3. Future Possibilities

A recent study used Circadian Kernel density estimation to measure physiological stability [34]. This ML model was used to compare the final 24 h of monitoring for patients discharged from the hospital without experiencing any SAEs against the eight hours of data leading up to physiological deterioration, defined as occurrence of SAEs or total NEWS of either  ≥ 6, ≥ 8, or  ≥ 10 [34]. The model did not demonstrate a convincing ability to discriminate stable physiology from observations prior to SAEs with area-under-receiver-operating-characteristics curve (AUROC) values of 0.59–0.61, but effectively discriminated stable physiology from events of high NEWS with AUROC of 0.77–0.99, specifically with better discrimination for higher total NEWSs [34]. Another study evaluated four ML models for real-time SAE prediction [35]. The best performing model had an AUROC of 0.65 for patients with SAEs and a corresponding false-positive rate of 0.073 in patients without SAEs, as well as a balanced sensitivity and false-positive rate [35]. Three additional studies evaluated different ML models in a prospective observational study of patients undergoing major surgery. The first study included 453 patients and based on a support vector machine demonstrated high accuracy with an AUROC of 0.93 with a corresponding sensitivity of 80% and specificity of 93% for the prediction of any SAEs based on a 2 h prediction window and 10 h observation window [36]. Another study with 450 patients evaluated a SMOTE-enhanced XGBoost model for classifying nighttime vital signs as preceding an SAE or from patients without SAEs, achieving an AUROC of 0.65 [37].

3.3. Sensium Vitals ®

The SV patch (Sensium, Abingdon, UK) is a CE-marked and FDA-approved wireless CVSM solution, transmitting data to SV bridges in the ward [24]. These data can be integrated with electronic medical records, allowing for the input of additional patient data to calculate the NEWS [24]. SV smart algorithms continuously process and analyze patient data, generating targeted notifications of deterioration (Table 2 and Table 3) [24].

3.3.1. Prediction of Serious Adverse Events

An observational study of 31 patients monitored with different CVSM solutions, including SV, descriptively analyzed vital sign trends in patients from a surgical step-down unit to general wards. Twenty AEs occurred in 11 patients, and the most common were atrial fibrillation and pneumonia, with four instances of each. SV effectively detected atrial fibrillation as a sudden HR increase and respiratory insufficiency as increased RF with decreased SpO2 [50]. No overall CVSM thresholds for AE prediction were highlighted, and specific vital sign changes for each case were not consistently detailed with thresholds [50]. A case-series based on implementation of SV in five European hospitals, including both medical and surgical wards, found that SV identified the first signs of respiratory and circulatory deterioration in the selected nine cases of AE with four cases of (paroxysmal) atrial fibrillation, two of sepsis, and one each of pyrexia, cardiogenic pulmonary edema, and pulmonary embolisms [53]. Specific thresholds were not consistently detailed but included descriptions of vital sign patterns. Tachycardia was noted in all cases and tachypnea in some for detecting atrial fibrillation [53]. A case study of a patient admitted for pneumonia and monitored with SV detected deterioration three hours post-admission due to a sudden increase in RR from 22/min to 51/min, HR from 75 to 157 bpm, and temperature from 38.0 to 39.5 °C, leading to a clinical examination and a new diagnosis of respiratory failure with severe sepsis [52].

3.3.2. Improving Outcomes

A large RCT with 747 patients, of which 517 were in the control group and 230 in the intervention group, was terminated early (57% inclusion) due to COVID-19 [48]. Alarm limits were preset to HR < 40 or > 120, RR < 8 or >24, and Tp > 39.0 °C, and delayed for 14 min, before an audio alert was sent to nurses’ devices. New disability three months after surgery occurred in 43.7% of patients in the control group and 39.1% of patients in the intervention group, resulting in an absolute difference of 4.6%. Descriptive statistics showed that all groups of postoperative complications were detected equally or more often in the intervention group compared to the control group [48]. A feasibility RCT of surgical patients included 125 patients for analysis, with 60 patients in the intervention group. The control group experienced more complications across all Clavien–Dindo categories, notably in major complications with 15.5% (10 patients) in the control group versus 5.0% (3 patients) in the intervention group. The intervention group had fewer unplanned high-dependency unit and ICU admissions (5 vs. 1 patient), and a shorter average length of stay (11.6 days vs. 16.2 days) compared to the control group. The intervention group had higher readmission rates and longer delays in receiving antibiotics for sepsis cases [49]. Another pilot cluster RCT included 226 patients, with 140 randomized to CVSM with predefined thresholds. The study found that CVSM patients received antibiotics faster after sepsis (10.4 vs. 16.9 h), had shorter hospital stays (13.3 vs. 14.6 days), and had lower 30-day readmission rates (11.4% vs. 20.9%). None of the associations were statistically significant [51].

3.3.3. Alarm Filtering

An observational study of 39 surgical patients simulated various alarm strategies, including SV threshold-based alarm algorithms and six methods for personalized alarm thresholds based on individual or situational factors (Table 4). A performance score, calculated from the overall alarm rate and early sensitivity defined as true-positive alarms within 24 h before an AE, evaluated alternative alarm strategies against the original strategy, with a positive value indicating improvement [38]. The SV algorithm produced an average of 0.49 alarms per patient per day, with an early sensitivity of 39% and a false detection rate of 59%. Compared to the original algorithm, only one strategy assessing lowered nighttime HR and RR thresholds, resulted in a positive performance score. A combination of strategies gave the best results, boosting early sensitivity up to 61% and overall sensitivity to 72%, with a slight increase in the alarm rate to 0.59 alarms per patient per day [38].

3.4. BioBeat

Biobeat (Biobeat Technologies Ltd., Petah Tikva, Israel) developed an FDA-cleared and CE-marked wrist monitor for repeated use and a disposable chest monitor for in-hospital monitoring, enabling the real-time monitoring of 14 cardiopulmonary vital signs processed every five minutes, including all standard vital signs as well as pulse pressure, systemic vascular resistance, heart rate variability, stroke volume, cardiac output, and cardiac index [25]. Biobeat states on their website that the system uses next-generation health AI and ML on big data to provide actionable insights for patient care, enabling a data-driven approach to personalized medicine (Table 2 and Table 3) [25].

Prediction of Deterioration

An observational study included 521 participants: 361 high-risk patients from general wards and 160 healthy controls. A multi-parameter real-time warning score (MPRT-WS) based on retrospectively defined thresholds set by a specialist panel for nine parameters, such as HR, RR, SpO2, Tp, and BP, was compared to the NEWS for detecting patient deterioration (cardiopulmonary resuscitation, ICU admission, death, or based on ABCDE criteria) (Supplementary Table S2) [55]. Comparing risk levels using both tools, NEWS classified 92.6% as low, 6.1% as medium, and 1.3% as high-risk, while MRPT-WS classified 92.9% as low, 6.4% as medium, and 0.7% as high-risk (p = 0.863). Among the 39 hospitalized patients that deteriorated, 30 were classified as high-risk or urgent by MPRT-WS, while 6 were classified as high-risk using NEWS [55].
Another observational study in two general wards of a large tertiary medical center included 217 high-risk patients for analysis [54]. Various scores were calculated, including NEWS, ABCNO (Airway, Breathing, Circulation, Neurology, Other), and a “wish list” deterioration score defined by the clinical team. Deterioration occurred in 24 patients: NEWS was high in 16 (67%), the ABCNO score was high in 18 (75%), and the “wish list” scoring criteria were observed in all 24 (100%) [54]. Of 193 patients without clinical deterioration, NEWS provided early warning alerts in 150 (77.7%), ABCNO criteria were met in 162 (83.9%), and the “wish list” criteria generated alerts for all 193 patients (100%) [54], corresponding to a sensitivity of 67% and specificity of 22% for NEWS; a sensitivity of 75% and specificity of 16% for ABCNO; and a 100% sensitivity and 0% specificity for the “wish list” criteria [54].

4. Discussion

AI supported CVSM have significant potential to improve patient care and reduce complications, but the clinical impact of AI supported CVSM on morbidity and mortality is sparsely described in the scientific literature. While all CVSM solutions claimed utilization of advanced AI, the assessed algorithms and methods mostly have potential application as simple AI, and only five studies from two CVSM solutions evaluated ML methods or prediction models with advanced AI potential, and these predictive algorithms do not appear to have been implemented in the solutions yet. Thus, a discrepancy between the claimed AI levels and the potential for the actual algorithms evaluated in peer-reviewed evidence was observed. This limitation is only relevant to current versions of the systems, with more advanced methods under development.

4.1. Vital Sign Thresholds and Algorithms for Prediction of Clinical Outcomes

Most studies used binary models of simple criteria for predictions of SAEs, clinical deterioration, and alarm filtering [14,15,43,48,55]. NEWS-based or other predefined thresholds demonstrated overall limited predictive ability for SAEs and generally resulted in many irrelevant alarms when applied to CVSM [14,15,39,46,51,54,55]. The lack of association between the duration and frequency of vital sign deviations outside predefined thresholds indicates that implementation of AI solely based on predefined thresholds is likely inadequate. It is still worth considering whether CVSM may outperform NEWS with this AI level, given that NEWS has only been designed and validated for predicting 24 h mortality, not for predicting SAEs, mortality outside 24 h, or further clinical deterioration [58]. Furthermore, CVSM significantly reduces the clinical staff’s workload from manual NEWS monitoring and is considered safe with minimal device-related adverse effects [59,60].
Eisenkraft 2023 used the Biobeat system and detected 25,635 readings (21.8% of the total) as “Urgent.” The clinical relevance of the utilized algorithms should be interpreted carefully, even considering health care providers did not respond to the alerts [55]. The study used healthy controls as a normal baseline for analysis and escalation scores, which may not be feasible since some degree of vital sign deviations are expected in all high-risk patients during admission [55]. Another study reported a significant imbalance in sensitivity and specificity for all evaluated algorithms (NEWS, ABCNO, “wish list”). The “wish list” algorithm generated alarms for all patients, yielding clinically irrelevant results with 100% sensitivity and 0% specificity [54].
Using the SV measuring RR, HR, and Tp, two studies analyzed cases of deteriorating patients included from observational studies and described vital sign patterns before specific AEs, defined as either preventable or unpreventable complications that required intervention [50,53]. The changes in all cases were not consistently detailed with thresholds but mostly limited to descriptions like “tachycardia” and “sudden RR increases”. Despite visualizations in figures that illustrate vital sign changes, these do not clarify thresholds or specific changes for complication prediction [50,53]. Describing and visualizing vital sign patterns in specific cases is likely insufficient to directly use as evidence for developing AI algorithms [17,26,27].
Two studies evaluated ML methods for the prediction of SAEs and clinical deterioration based on HR, RR, SpO2, Tp, and BP measurements from the WARD 24/7 system. Both studies found AUROC for the prediction of subsequent SAEs corresponding to a low accuracy, likely limiting the clinical relevance [34,35,61]. The limited predictive ability of these models was potentially due to heterogeneity in the SAEs or the fact that some SAEs may not at all be preceded by a vital sign deviation. The underlying pathophysiology varies substantially for different SAEs, even within specific categories. This variation likely results in different vital sign patterns preceding each specific SAE, making it impossible for one model to accurately predict all SAEs or a combination of various SAEs in specific groups.

4.2. Continuous Vital Sign Monitoring for Improving Clincial Outcomes

Posthuma 2024 conducted a large RCT using SV in high-risk surgical patients evaluating predefined thresholds, and found a 4.6% absolute reduction in new disability three months after surgery in the CVSM group, but only descriptive statistics were used, so differences between control and intervention groups should be interpreted with caution [48]. A feasibility RCT also utilized the SV system and reported the improvement of certain outcomes, but the small sample size limited the power to reach a significant conclusion [49].
Three before-and-after studies utilizing VM for the CVSM of HR, RR, SpO2, Tp, and BP, reported improved outcomes, including reduced ICU admissions, RRT calls, and complications [41,42,43]. The outcomes were not consistently improved across studies, and most other examined outcomes, including hospital length of stay and in-hospital mortality, were not significantly reduced [41,42,43]. Establishing a causal relationship between the intervention and outcomes is challenging with a before-and-after study design [62]. One study did not specify the method of patient monitoring during the baseline period [42], and another did not clarify the definition of complications and it may differ significantly from the AE and SAE definitions [41]. Another study utilized the VM system and conducted a propensity-matched retrospective observational analysis that demonstrated increased odds for adverse outcomes in the intermittently monitored group (mortality or ICU admission, heart failure, myocardial infarction, and acute kidney injury), but this study design was vulnerable to bias and still requires validation in large RCTs [11].

4.3. Filtering Alarms

Kjærgaard (2023) and Weller (2018) both reduced the number of alarms per patient per day through filtering and clinical staff feedback, respectively [39,46]. Neither study detailed the clinical impact of these alarm adjustments in terms of sensitivity and specificity, which is an issue since the goal should be to generate relevant alarms, not simply decrease the number of alarms. Another study utilized the VM system and created a cloud computing tool allowing threshold testing and optimization that may help manage alarm burden [40]. It remains uncertain if ML algorithms are applied to this database, and what the clinical impact would be of adjusting alarm thresholds [40].
Rossum (2022) utilized six different ML methods based on CVSM of RR, HR, and Tp with SV for adaptive threshold generation and successfully improved the sensitivity for any AE with a minimal increase in alarms, specifically when integrating multiple models as compared to the standard threshold based SV algorithm [38]. Although the study only included 39 patients, these findings are consistent with previous research indicating the potential for AI to reduce irrelevant alarms and predict complications [12,34,63].

4.4. Strengths and Limitations

The primary strength of this review is the assessment of many peer-reviewed articles providing a comprehensive overview of the evidence for different algorithms used within four different state-of-the-art CVSM solutions. We were unable to find other studies assessing current evidence with potential application as AI in CVSM, highlighting a gap between promised AI levels and the capabilities of current algorithms.
This review also comes with several limitations. The first limitation is the significant variety in study designs, CVSM sensors, outcomes, and reported performance metrics for the algorithms, making data comparison across studies and CVSM solutions difficult and rendering a meta-analysis unfeasible. The second limitation is the subjective aspects of AI definitions. No definitive guidelines exist on AI levels or the implementation of algorithms in peer-reviewed research to CVSM sensors. This leaves the AI levels claimed on official websites, and the potential AI level of the methods assessed in peer-reviewed evidence open to interpretation. A third limitation is the potential existence of more recent studies, as most included studies were found on the official CVSM solutions websites, and the latest literature may not have been uploaded or published yet. We screened and included many studies, with only two identified through the literature search not uploaded on the websites. The focus solely on non-invasive CVSM technologies and exclusion of specific patient groups, such as those in ICU and non-hospitalized settings, limits the generalizability of the study findings to the very high- and the low-risk populations. The results from this review will likely change over the coming months and years, where hopefully more CVSM solutions report results of the clinical validation of AI algorithms.

4.5. Perspectives

Several CVSM solutions, including those described in this review, have been implemented and validated in clinical settings, but there is a notable absence of large RCTs [64,65]. The assessment of CVSM for improving clinical outcomes needs to be thoroughly validated in large RCTs. There is currently a discrepancy in the definition of complications. A uniform method for defining complications, including AE and SAE, is warranted to make results transparent and comparable. As knowledge of AI expands and larger databases are developed, more sophisticated ML models will probably enhance the predictive ability of CVSM. Future benefits of AI will likely include stable clinical evaluations of both traditional and novel vital signs, such as heart rate variability and perfusion index, including personalized thresholds based on advanced data analyses of extensive databases [16,66,67], as well as integration with demographic data, blood samples, fluid therapy, and other clinically relevant factors [34,68].
All CVSM solutions evaluated in this review describe the application of medical evidence-based algorithms or AI to reduce alarms and improve patient outcomes [21,22,23,24,25]. This is currently only sparsely reflected in the number of peer-previewed publications, and the methods applied in the commercially available systems are still at a simple AI level. In addition, the experimental ML methods have generally presented a low accuracy for prediction of complications [34,35,61]. There are major limitations affecting how quickly AI can be applied to the commercial solutions, as both the conduction of large studies, analysis of data, and peer-review process take a long time and are associated with large financial expenses. The entire regulatory process of validating the effectiveness and safety of the algorithms is also very rigorous, complex, and time-consuming. Transparently disclosing detailed information about the AI and ML algorithms used to set alarm thresholds is important to ensure an understanding of assumptions and prevent AI biases from the data being unrepresentative to the applied population, which could lead to reduced effectiveness and potential adverse outcomes [69]. This further complicates their implementation, as the algorithms need to be validated in patient populations with different demographic characteristics to minimize AI bias [69].

5. Conclusions

Current evidence on CVSM primarily suggests its potential application as simple AI, which generates alarms based on fixed vital sign thresholds. All four CVSM solutions (100%) assessed algorithms with simple AI potential and only two CVSM solutions (50%) assessed hypothetical models with potential for application as advanced AI. A discrepancy exists between promised AI levels and the capabilities of algorithms assessed in the peer-reviewed evidence. Transparent reporting on the application of AI and ML algorithms in CVSM sensors is warranted.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/s24196497/s1, Table S1: Alarm modification within the Visi Mobile System, Table S2: Algorithms used in the National Early Warning Score and the new multi-parameter real-time warning score.

Author Contributions

Conceptualization, N.A. and C.S.M.; Methodology N.A. and C.S.M.; investigation, N.A.; writing—original draft preparation, NA.; writing—review and editing, N.A., C.S.M. and E.K.A.; visualization, N.A.; supervision, C.S.M. and E.K.A. All authors have read and agreed to the published version of the manuscript.

Funding

This project was supported by Novo Nordisk Fonden (NNF21SA0069370).

Data Availability Statement

No new data were created or analyzed in this study. Data sharing is not applicable to this article.

Conflicts of Interest

Christian S. Meyhoff and Eske K. Aasvang have founded a start-up company, WARD247 ApS, with the aim of pursuing the regulatory and commercial activities of the WARD-project. WARD247 ApS has obtained license agreement for any WARD-project software and patents. One patent has been filed: “Wireless Assessment of Respiratory and circulatory Distress (WARD)—clinical support system (CSS)—an automated clinical support system to improve patient safety and outcomes”. None of the above entities have influence on the study design, conduct, analysis, or reporting.

References

  1. McQuillan, P.; Pilkington, S.; Allan, A.; Taylor, B.; Short, A.; Morgan, G.; Nielsen, M.; Barrett, D.; Smith, G.; Collins, C.H. Confidential Inquiry into Quality of Care before Admission to Intensive Care. BMJ 1998, 316, 1853–1858. [Google Scholar] [CrossRef] [PubMed]
  2. McGloin, H.; Adam, S.K.; Singer, M. Unexpected Deaths and Referrals to Intensive Care of Patients on General Wards. Are Some Cases Potentially Avoidable? J. R. Coll. Physicians Lond. 1999, 33, 255–259. [Google Scholar] [PubMed]
  3. Smith, M.E.B.; Chiovaro, J.C.; O’Neil, M.; Kansagara, D.; Quiñones, A.R.; Freeman, M.; Motu’apuaka, M.L.; Slatore, C.G. Early Warning System Scores for Clinical Deterioration in Hospitalized Patients: A Systematic Review. Ann. Am. Thorac. Soc. 2014, 11, 1454–1465. [Google Scholar] [CrossRef] [PubMed]
  4. Gerry, S.; Bonnici, T.; Birks, J.; Kirtley, S.; Virdee, P.S.; Watkinson, P.J.; Collins, G.S. Early Warning Scores for Detecting Deterioration in Adult Hospital Patients: Systematic Review and Critical Appraisal of Methodology. BMJ 2020, 369, m1501. [Google Scholar] [CrossRef] [PubMed]
  5. Tarassenko, L.; Hann, A.; Young, D. Integrated Monitoring and Analysis for Early Warning of Patient Deterioration. Br. J. Anaesth. 2006, 97, 64–68. [Google Scholar] [CrossRef]
  6. Pedersen, N.E.; Rasmussen, L.S.; Petersen, J.A.; Gerds, T.A.; Østergaard, D.; Lippert, A. A Critical Assessment of Early Warning Score Records in 168,000 Patients. J. Clin. Monit. Comput. 2018, 32, 109–116. [Google Scholar] [CrossRef]
  7. Saab, R.; Wu, B.P.; Rivas, E.; Chiu, A.; Lozovoskiy, S.; Ma, C.; Yang, D.; Turan, A.; Sessler, D.I. Failure to Detect Ward Hypoxaemia and Hypotension: Contributions of Insufficient Assessment Frequency and Patient Arousal during Nursing Assessments. Br. J. Anaesth. 2021, 127, 760–768. [Google Scholar] [CrossRef]
  8. Sun, Z.; Sessler, D.I.; Dalton, J.E.; Devereaux, P.J.; Shahinyan, A.; Naylor, A.J.; Hutcherson, M.T.; Finnegan, P.S.; Tandon, V.; Darvish-Kazem, S.; et al. Postoperative Hypoxemia Is Common and Persistent: A Prospective Blinded Observational Study. Anesth. Analg. 2015, 121, 709–715. [Google Scholar] [CrossRef] [PubMed]
  9. De Vries, E.N.; Ramrattan, M.A.; Smorenburg, S.M.; Gouma, D.J.; Boermeester, M.A. The Incidence and Nature of In-Hospital Adverse Events: A Systematic Review. Qual. Saf. Health Care 2008, 17, 216–223. [Google Scholar] [CrossRef]
  10. Sun, L.; Joshi, M.; Khan, S.N.; Ashrafian, H.; Darzi, A. Clinical Impact of Multi-Parameter Continuous Non-Invasive Monitoring in Hospital Wards: A Systematic Review and Meta-Analysis. J. R. Soc. Med. 2020, 113, 217–224. [Google Scholar] [CrossRef]
  11. Rowland, B.A.; Motamedi, V.; Michard, F.; Saha, A.K.; Khanna, A.K. Impact of Continuous and Wireless Monitoring of Vital Signs on Clinical Outcomes: A Propensity-Matched Observational Study of Surgical Ward Patients. Br. J. Anaesth. 2024, 132, 519–527. [Google Scholar] [CrossRef] [PubMed]
  12. Peelen, R.V.; Eddahchouri, Y.; Koeneman, M.; Melis, R.; van Goor, H.; Bredie, S.J.H. Comparing Continuous with Periodic Vital Sign Scoring for Clinical Deterioration Using a Patient Data Model. J. Med. Syst. 2023, 47, 60. [Google Scholar] [CrossRef] [PubMed]
  13. Grønbaek, K.K.; Rasmussen, S.M.; Langer, N.H.; Vincentz, M.; Oxbøll, A.-B.; Søgaard, M.; Awada, H.N.; Jensen, T.O.; Jensen, M.T.; Sørensen, H.B.D.; et al. Continuous Monitoring Is Superior to Manual Measurements in Detecting Vital Sign Deviations in Patients with COVID-19. Acta Anaesthesiol. Scand. 2023, 67, 640–648. [Google Scholar] [CrossRef] [PubMed]
  14. Elvekjaer, M.; Rasmussen, S.M.; Grønbæk, K.K.; Porsbjerg, C.M.; Jensen, J.-U.; Haahr-Raunkjær, C.; Mølgaard, J.; Søgaard, M.; Sørensen, H.B.D.; Aasvang, E.K.; et al. Clinical Impact of Vital Sign Abnormalities in Patients Admitted with Acute Exacerbation of Chronic Obstructive Pulmonary Disease: An Observational Study Using Continuous Wireless Monitoring. Intern. Emerg. Med. 2022, 17, 1689–1698. [Google Scholar] [CrossRef] [PubMed]
  15. Haahr-Raunkjaer, C.; Mølgaard, J.; Elvekjaer, M.; Rasmussen, S.M.; Achiam, M.P.; Jorgensen, L.N.; Søgaard, M.I.V.; Grønbaek, K.K.; Oxbøll, A.-B.; Sørensen, H.B.D.; et al. Continuous Monitoring of Vital Sign Abnormalities; Association to Clinical Complications in 500 Postoperative Patients. Acta Anaesthesiol. Scand. 2022, 66, 552–562. [Google Scholar] [CrossRef]
  16. Michard, F.; Gan, T.J.; Kehlet, H. Digital Innovations and Emerging Technologies for Enhanced Recovery Programmes. Br. J. Anaesth. 2017, 119, 31–39. [Google Scholar] [CrossRef]
  17. Haug, C.J.; Drazen, J.M. Artificial Intelligence and Machine Learning in Clinical Medicine, 2023. N. Engl. J. Med. 2023, 388, 1201–1208. [Google Scholar] [CrossRef]
  18. Steerling, E.; Siira, E.; Nilsen, P.; Svedberg, P.; Nygren, J. Implementing AI in Healthcare-the Relevance of Trust: A Scoping Review. Front. Health Serv. 2023, 3, 1211150. [Google Scholar] [CrossRef]
  19. Ghassemi, M.; Naumann, T.; Schulam, P.; Beam, A.L.; Chen, I.Y.; Ranganath, R. A Review of Challenges and Opportunities in Machine Learning for Health. AMIA Jt. Summits Transl. Sci. Proc. 2020, 2020, 191–200. [Google Scholar]
  20. Ouzzani, M.; Hammady, H.; Fedorowicz, Z.; Elmagarmid, A. Rayyan-a Web and Mobile App for Systematic Reviews. Syst. Rev. 2016, 5, 210. [Google Scholar] [CrossRef]
  21. Patient Vital Signs Monitoring System|Sotera Digital Health. Available online: https://soteradigitalhealth.com (accessed on 9 February 2024).
  22. Home—WARD Project. Available online: https://ward247.org/ (accessed on 27 May 2024).
  23. Ward 24|7. Available online: https://ward247.com/ (accessed on 27 May 2024).
  24. Sensium®—Connected Care. Available online: https://www.tsc-cc.com/products/sensium/ (accessed on 30 May 2024).
  25. Biobeat Medical—Smart Vital Signs Monitoring. Available online: https://www.bio-beat.com/ (accessed on 30 May 2024).
  26. Joshi, N. 7 Types Of Artificial Intelligence. Available online: https://www.forbes.com/sites/cognitiveworld/2019/06/19/7-types-of-artificial-intelligence/ (accessed on 21 March 2024).
  27. Deo, R.C. Machine Learning in Medicine. Circulation 2015, 132, 1920–1930. [Google Scholar] [CrossRef] [PubMed]
  28. Welcome to Isansys. Available online: https://www.isansys.com/ (accessed on 20 June 2024).
  29. Covidien Products|Medical Surgical Portfolio|Medtronic. Available online: https://www.medtronic.com/covidien/en-us/products.html (accessed on 30 May 2024).
  30. Home—Polso—Chronic Disease Management. Anywhere. Available online: https://polsohealth.com/ (accessed on 30 May 2024).
  31. Home—VitalConnect. Available online: https://vitalconnect.com/ (accessed on 10 June 2024).
  32. Healthdot|Philips. Available online: https://www.philips.com/a-w/about/dutch-design-week-2020/healthdot.cs (accessed on 30 May 2024).
  33. Thermia. Available online: https://thermia.io/ithermonitor (accessed on 30 May 2024).
  34. Rasmussen, S.S.; Grønbæk, K.K.; Mølgaard, J.; Haahr-Raunkjær, C.; Meyhoff, C.S.; Aasvang, E.K.; Sørensen, H.B.D. Quantifying Physiological Stability in the General Ward Using Continuous Vital Signs Monitoring: The Circadian Kernel Density Estimator. J. Clin. Monit. Comput. 2023, 37, 1607–1617. [Google Scholar] [CrossRef] [PubMed]
  35. Kristinsson, Æ.Ö.; Gu, Y.; Rasmussen, S.M.; Mølgaard, J.; Haahr-Raunkjær, C.; Meyhoff, C.S.; Aasvang, E.K.; Sørensen, H.B.D. Prediction of Serious Outcomes Based on Continuous Vital Sign Monitoring of High-Risk Patients. Comput. Biol. Med. 2022, 147, 105559. [Google Scholar] [CrossRef] [PubMed]
  36. Gu, Y.; Rasmussen, S.M.; Molgaard, J.; Haahr-Raunkjar, C.; Meyhoff, C.S.; Aasvang, E.K.; Sorensen, H.B.D. Prediction of Severe Adverse Event from Vital Signs for Post-Operative Patients. Annu. Int. Conf. IEEE Eng. Med. Biol. Soc. 2021, 2021, 971–974. [Google Scholar] [CrossRef]
  37. Mayer, L.; Rasmussen, S.M.; Molgaard, J.; Gu, Y.; Aasvang, E.K.; Meyhoff, C.S.; Sorensen, H.B.D. Prediction of Serious Adverse Events from Nighttime Vital Signs Values. Annu. Int. Conf. IEEE Eng. Med. Biol. Soc. 2022, 2022, 2631–2634. [Google Scholar] [CrossRef]
  38. Van Rossum, M.C.; Vlaskamp, L.B.; Posthuma, L.M.; Visscher, M.J.; Breteler, M.J.M.; Hermens, H.J.; Kalkman, C.J.; Preckel, B. Adaptive Threshold-Based Alarm Strategies for Continuous Vital Signs Monitoring. J. Clin. Monit. Comput. 2022, 36, 407–417. [Google Scholar] [CrossRef]
  39. Weller, R.S.; Foard, K.L.; Harwood, T.N. Evaluation of a Wireless, Portable, Wearable Multi-Parameter Vital Signs Monitor in Hospitalized Neurological and Neurosurgical Patients. J. Clin. Monit. Comput. 2018, 32, 945–951. [Google Scholar] [CrossRef]
  40. Welch, J.; Kanter, B.; Skora, B.; McCombie, S.; Henry, I.; McCombie, D.; Kennedy, R.; Soller, B. Multi-Parameter Vital Sign Database to Assist in Alarm Optimization for General Care Units. J. Clin. Monit. Comput. 2016, 30, 895–900. [Google Scholar] [CrossRef]
  41. Verrillo, S.C.; Cvach, M.; Hudson, K.W.; Winters, B.D. Using Continuous Vital Sign Monitoring to Detect Early Deterioration in Adult Postoperative Inpatients. J. Nurs. Care Qual. 2019, 34, 107–113. [Google Scholar] [CrossRef]
  42. Stellpflug, C.; Pierson, L.; Roloff, D.; Mosman, E.; Gross, T.; Marsh, S.; Willis, V.; Gabrielson, D. Continuous Physiological Monitoring Improves Patient Outcomes. Am. J. Nurs. 2021, 121, 40–46. [Google Scholar] [CrossRef]
  43. Eddahchouri, Y.; Peelen, R.V.; Koeneman, M.; Touw, H.R.W.; van Goor, H.; Bredie, S.J.H. Effect of Continuous Wireless Vital Sign Monitoring on Unplanned ICU Admissions and Rapid Response Team Calls: A before-and-after Study. Br. J. Anaesth. 2022, 128, 857–863. [Google Scholar] [CrossRef] [PubMed]
  44. Loft, F.C.; Rasmussen, S.M.; Elvekjaer, M.; Haahr-Raunkjaer, C.; Sørensen, H.B.D.; Aasvang, E.K.; Meyhoff, C.S. WARD-Project Group Continuously Monitored Vital Signs for Detection of Myocardial Injury in High-Risk Patients—An Observational Study. Acta Anaesthesiol. Scand. 2022, 66, 674–683. [Google Scholar] [CrossRef] [PubMed]
  45. Songthawornpong, N.; Elvekjaer, M.; Mølgaard, J.; Rasmussen, S.M.; Meyhoff, C.S.; Aasvang, E.K.; Eriksen, V.R. Deviating Vital Signs in Continuous Monitoring Prior to Discharge and Risk of Readmission: An Observational Study. Intern. Emerg. Med. 2023, 18, 1453–1461. [Google Scholar] [CrossRef] [PubMed]
  46. Kjærgaard, K.; Mølgaard, J.; Rasmussen, S.M.; Meyhoff, C.S.; Aasvang, E.K. The Effect of Technical Filtering and Clinical Criteria on Alert Rates from Continuous Vital Sign Monitoring in the General Ward. Hosp Pract (1995) 2023, 51, 295–302. [Google Scholar] [CrossRef] [PubMed]
  47. Jokinen, J.D.V.; Carlsson, C.J.; Rasmussen, S.M.; Nielsen, O.W.; Winkel, B.G.; Jorgensen, L.N.; Achiam, M.P.; Mølgaard, J.; Sørensen, H.B.D.; Aasvang, E.K.; et al. Wireless Single-Lead ECG Monitoring to Detect New-Onset Postoperative Atrial Fibrillation in Patients After Major Noncardiac Surgery: A Prospective Observational Study. Anesth. Analg. 2022, 135, 100–109. [Google Scholar] [CrossRef]
  48. Posthuma, L.M.; Breteler, M.J.M.; Lirk, P.B.; Nieveen van Dijkum, E.J.; Visscher, M.J.; Breel, J.S.; Wensing, C.A.G.L.; Schenk, J.; Vlaskamp, L.B.; van Rossum, M.C.; et al. Surveillance of High-Risk Early Postsurgical Patients for Real-Time Detection of Complications Using Wireless Monitoring (SHEPHERD Study): Results of a Randomized Multicenter Stepped Wedge Cluster Trial. Front. Med. 2023, 10, 1295499. [Google Scholar] [CrossRef]
  49. Downey, C.L.; Croft, J.; Ainsworth, G.; Buckley, H.; Shinkins, B.; Randell, R.; Brown, J.M.; Jayne, D.G. Trial of Remote Continuous versus Intermittent NEWS Monitoring after Major Surgery (TRaCINg): A Feasibility Randomised Controlled Trial. Pilot. Feasibility Stud. 2020, 6, 183. [Google Scholar] [CrossRef]
  50. Breteler, M.J.M.; KleinJan, E.; Numan, L.; Ruurda, J.P.; Van Hillegersberg, R.; Leenen, L.P.H.; Hermans, M.; Kalkman, C.J.; Blokhuis, T.J. Are Current Wireless Monitoring Systems Capable of Detecting Adverse Events in High-Risk Surgical Patients? A Descriptive Study. Injury 2020, 51 (Suppl. 2), S97–S105. [Google Scholar] [CrossRef]
  51. Downey, C.; Randell, R.; Brown, J.; Jayne, D.G. Continuous Versus Intermittent Vital Signs Monitoring Using a Wearable, Wireless Patch in Patients Admitted to Surgical Wards: Pilot Cluster Randomized Controlled Trial. J. Med. Internet Res. 2018, 20, e10802. [Google Scholar] [CrossRef]
  52. Ghazali, D.A.; Stephan, O.; Choquet, C.; Antoniol, S.; Casalino, E. Early Diagnosis of Acute Respiratory Failure Using an E-Health Application in Patients Requiring Oxygen Therapy. Am. J. Emerg. Med. 2019, 37, 1370–1372. [Google Scholar] [CrossRef]
  53. Posthuma, L.M.; Downey, C.; Visscher, M.J.; Ghazali, D.A.; Joshi, M.; Ashrafian, H.; Khan, S.; Darzi, A.; Goldstone, J.; Preckel, B. Remote Wireless Vital Signs Monitoring on the Ward for Early Detection of Deteriorating Patients: A Case Series. Int. J. Nurs. Stud. 2020, 104, 103515. [Google Scholar] [CrossRef] [PubMed]
  54. Itelman, E.; Shlomai, G.; Leibowitz, A.; Weinstein, S.; Yakir, M.; Tamir, I.; Sagiv, M.; Muhsen, A.; Perelman, M.; Kant, D.; et al. Assessing the Usability of a Novel Wearable Remote Patient Monitoring Device for the Early Detection of In-Hospital Patient Deterioration: Observational Study. JMIR Form. Res. 2022, 6, e36066. [Google Scholar] [CrossRef] [PubMed]
  55. Eisenkraft, A.; Goldstein, N.; Merin, R.; Fons, M.; Ishay, A.B.; Nachman, D.; Gepner, Y. Developing a Real-Time Detection Tool and an Early Warning Score Using a Continuous Wearable Multi-Parameter Monitor. Front. Physiol. 2023, 14, 1138647. [Google Scholar] [CrossRef] [PubMed]
  56. Valentin, J. Addressing the Alarm Burden. Available online: https://cdn2.hubspot.net/hubfs/5599582/SoteraWireless-April2020/PDF/Addressing-the-Alarm-Burden.pdf?__hstc=233292865.0c35ef94d9fa63c4ca1f49bdc300fb2a.1708338975578.1716285333525.1716969776382.7&__hssc=233292865.1.1716969776382&__hsfp=4242973585&hsCtaTrac (accessed on 6 June 2024).
  57. ICH E6 (R2) Good Clinical Practice—Scientific Guideline|European Medicines Agency. Available online: https://www.ema.europa.eu/en/ich-e6-r2-good-clinical-practice-scientific-guideline (accessed on 5 February 2024).
  58. Holland, M.; Kellett, J. A Systematic Review of the Discrimination and Absolute Mortality Predicted by the National Early Warning Scores According to Different Cut-off Values and Prediction Windows. Eur. J. Intern. Med. 2022, 98, 15–26. [Google Scholar] [CrossRef] [PubMed]
  59. Aagaard, N.; Larsen, A.T.; Aasvang, E.K.; Meyhoff, C.S. The Impact of Continuous Wireless Monitoring on Adverse Device Effects in Medical and Surgical Wards: A Review of Current Evidence. J. Clin. Monit. Comput. 2023, 37, 7–17. [Google Scholar] [CrossRef] [PubMed]
  60. Sigvardt, E.; Grønbaek, K.K.; Jepsen, M.L.; Søgaard, M.; Haahr, L.; Inácio, A.; Aasvang, E.K.; Meyhoff, C.S. Workload Associated with Manual Assessment of Vital Signs as Compared with Continuous Wireless Monitoring. Acta Anaesthesiol. Scand. 2024, 68, 274–279. [Google Scholar] [CrossRef]
  61. Swets, J.A. Measuring the Accuracy of Diagnostic Systems. Science 1988, 240, 1285–1293. [Google Scholar] [CrossRef]
  62. Goodacre, S. Uncontrolled Before-after Studies: Discouraged by Cochrane and the EMJ. Emerg. Med. J. 2015, 32, 507–508. [Google Scholar] [CrossRef]
  63. Anderson, H.R.; Borgen, A.C.; Christnacht, R.; Ng, J.; Weller, J.G.; Davison, H.N.; Noseworthy, P.A.; Olson, R.; O’Laughlin, D.; Disrud, L.; et al. Stats on the Desats: Alarm Fatigue and the Implications for Patient Safety. BMJ Open Qual. 2023, 12, e002262. [Google Scholar] [CrossRef]
  64. Blythe, R.; Parsons, R.; White, N.M.; Cook, D.; McPhail, S. A Scoping Review of Real-Time Automated Clinical Deterioration Alerts and Evidence of Impacts on Hospitalised Patient Outcomes. BMJ Qual. Saf. 2022, 31, 725–734. [Google Scholar] [CrossRef]
  65. Areia, C.; Biggs, C.; Santos, M.; Thurley, N.; Gerry, S.; Tarassenko, L.; Watkinson, P.; Vollam, S. The Impact of Wearable Continuous Vital Sign Monitoring on Deterioration Detection and Clinical Outcomes in Hospitalised Patients: A Systematic Review and Meta-Analysis. Crit. Care 2021, 25, 351. [Google Scholar] [CrossRef] [PubMed]
  66. Aagaard, N.; Olsen, M.H.; Rasmussen, O.W.; Grønbaek, K.K.; Mølgaard, J.; Haahr-Raunkjaer, C.; Elvekjaer, M.; Aasvang, E.K.; Meyhoff, C.S. Prognostic Value of Heart Rate Variability for Risk of Serious Adverse Events in Continuously Monitored Hospital Patients. J. Clin. Monit. Comput. 2024. [Google Scholar] [CrossRef] [PubMed]
  67. Sun, X.; He, H.; Xu, M.; Long, Y. Peripheral Perfusion Index of Pulse Oximetry in Adult Patients: A Narrative Review. Eur. J. Med. Res. 2024, 29, 457. [Google Scholar] [CrossRef] [PubMed]
  68. Ren, W.; Jin, N.; Wang, T. An Interdigital Conductance Sensor for Measuring Liquid Film Thickness in Inclined Gas–Liquid Two-Phase Flow. IEEE Trans. Instrum. Meas. 2024, 73, 1–9. [Google Scholar] [CrossRef]
  69. Jahandideh, S.; Ozavci, G.; Sahle, B.W.; Kouzani, A.Z.; Magrabi, F.; Bucknall, T. Evaluation of Machine Learning-Based Models for Prediction of Clinical Deterioration: A Systematic Literature Review. Int. J. Med. Inform. 2023, 175, 105084. [Google Scholar] [CrossRef]
Sensors 24 06497 g001
Figure 1. Levels of artificial intelligence. Legends: Created in BioRender. Aagaard, N. (2024) BioRender.com/q01h863. Based on standardized AI definitions [26]. Pictures representing level II–V were created using DALL·E 3 (OpenAI, San Francisco, CA, USA).
Figure 1. Levels of artificial intelligence. Legends: Created in BioRender. Aagaard, N. (2024) BioRender.com/q01h863. Based on standardized AI definitions [26]. Pictures representing level II–V were created using DALL·E 3 (OpenAI, San Francisco, CA, USA).
Sensors 24 06497 g001
Sensors 24 06497 g002
Figure 2. Prisma flow diagram showing the selection process of included articles.
Figure 2. Prisma flow diagram showing the selection process of included articles.
Sensors 24 06497 g002
Sensors 24 06497 g003
Figure 3. Number of peer-reviewed studies assessing algorithms for potential simple and advanced AI applications.
Figure 3. Number of peer-reviewed studies assessing algorithms for potential simple and advanced AI applications.
Sensors 24 06497 g003
Table 1. Search report.
Table 1. Search report.
Search ReportCurrent Evidence on Continuous Vital Sign Monitoring of In-Hospital Patients
DatabasePubMed and Embase
Date conducted30 May 2024
Search terms in PubMedContinuous * monitor * OR “physiologic monitoring” OR “physiological monitoring” OR real-time monitoring OR automat * monitor * OR “surveillance monitoring” OR “signal processing” AND (randomized controlled trial [Publication Type] OR observational study [Publication Type] OR prospective observational study [Publication Type] OR cluster randomized controlled trial [Publication Type]) AND (Blood Pressure [MeSH Terms] OR Oxygen Saturation [MeSH Terms] OR Body Temperature [MeSH Terms] OR Heart Rate [MeSH Terms] OR Respiratory Rate [MeSH Terms] OR Vital Signs [MeSH Terms]) AND wireless
Search terms in EmbaseContinous vital sign monitoring {Including Related Terms}
Limits in EmbaseLimit to (human and english language and embase and (clinical trial or randomized controlled trial or controlled clinical trial or multicenter study) and yr = “2010-Current” and article and (adult <18 to 64 years> or aged <65+ years>))
Hits391
Included10
Table 2. Characteristics of continuous vital sign monitoring systems.
Table 2. Characteristics of continuous vital sign monitoring systems.
CompanyDevicesCountryCE-MarkFDA-Approved or ClearedPopulation in Published ArticlesStudy Types in Peer-Reviewed EvidenceVital Signs Measured
SurgicalMedicalRCTOBSB-ARRSpO2HRBPTP
Sotera Digital HealthVisi Mobile®Carlsbad, California,
United States		xxxxxxxxxxxx
WARD 247 ApSIsansys Lifetouch and Lifetemp, Nonin WristOx, 3150 Meditech BlueBP-05, or A&D TM-2441Copenhagen, Denmarkx xx x xxxxx
The Surgical Company,
Connected Care		Sensium Vitals®Abingdon, United Kingdomxxxxxx x x x
Biobeat Technologies Ltd.Biobeat wrist monitor and chest monitorPetah Tikva, Israelxxxx x xxxxx
Legends: FDA: Food and Drug Administration, RCT: randomized controlled trial, OBS: observational studies, B-A: before-and-after studies, RR: respiratory rate, SpO2: peripheral oxygen saturation, HR: heart rate, BP: blood pressure, TP: temperature (skin or axillary).
Table 3. Comparison of artificial intelligence levels stated on official websites versus levels found in peer-reviewed evidence.
Table 3. Comparison of artificial intelligence levels stated on official websites versus levels found in peer-reviewed evidence.
System and SourceCitation of AI Level on Official WebsiteStated AI Level on Official WebsiteComparison of AI-Levels Stated on Websites versus Published in Peer-Reviewed JournalsPotential AI Level in Peer-Reviewed EvidenceDescription of Methods Utilized in Peer-Reviewed EvidenceNumber of Studies Using Algorithms with Potential for Promised AI Level
Visi Mobile®“ViSi Mobile improves patient safety by utilizing machine learning from millions of hours of patient data to recognize patient deterioration while minimizing alarm burden and maximizing clinical workflow”Advanced AI
(Level II)		>Simple AI
(Level I)		Studies using predefined thresholds with adjustment based on clinical staff feedback to reduce total alarms.0
WARD24/7.org“Wireless Assessment of Respiratory and circulatory Distress (WARD) offers continuous 24/7 observation of high-risk patients with real-time alarms through innovative machine learning algorithms in a Clinical Support System (CSS)”Advanced AI
(Level II)		=Advanced AI
(Level II)		Advanced ML algorithms to predict deterioration and SAEs. Not clearly stated if implemented to the clinical support system yet.4
WARD24/7.com“Advanced clinically modelled algorithms interpret standard vital sign patterns from standard wireless sensors and detect deterioration earlier in real-time”Advanced AI
(Level II)		=Advanced AI
(Level II)		Advanced ML algorithms to predict deterioration and SAEs. Not clearly stated if implemented to the clinical support system yet.4
Sensium Vitals®“Sensium’s monitoring and smart algorithms continuously process and analyse all patient data, generating targeted notifications of deterioration, efficiently bringing the nurse to the deteriorating patient.”Advanced AI
(Level II)		=Advanced AI
(Level II)		Adaptive thresholds-based alarm strategies, and a combination of these strategies. Not clearly stated if implemented to the system yet.1
Biobeat“Biobeat’s solution uses health-AI and ML on big-data in order to provide actionable insights on patient care. More than just analyzing the data, Biobeat also generates it, using our proprietary sensor for continuous monitoring of vital signs unique to Biobeat”Advanced AI
(Level II)		>Simple AI
(Level I)		Studies using predefined thresholds based on a modification of NEWS or fixed thresholds retrospectively set by a specialist panel0
Legends: Equal and greater than signs illustrate comparison of AI-levels stated on websites and published in peer-reviewed journals. ML: machine learning; SAEs: serious adverse events; ECG: electrocardiographic; AI: artificial intelligence; N.A.: not available.
Table 4. Overview of alarms based on system and study.
Table 4. Overview of alarms based on system and study.
AlarmThresholdsDuration before Alarm
Visi Mobile [39]
BradypneaRR < 4 breaths/min2 min
TachypneaRR > 35 breaths/min2 min
DesaturationSpO2 < 85%1.5 min
TachycardiaHR > 150 beats/min15 s
BradycardiaHR < 39 beats/min15 s
HypotensionMAP < 58 mmHg1 min
HypertensionSYS BP > 200 mmHg4 min
WARD 24/7 [46]
BradypneaRR ≤ 5 breaths/min and HR ≥ 20 beats/min1 min
TachypneaRR ≥ 24 breaths/min5 min
DesaturationSpO2 < 92%60 min
SpO2 < 88%10 min
SpO2 < 85%5 min
SpO2 < 80%1 min
Sinus tachycardiaHR ≥ 111 beats/min60 min
HR > 130 beats/min30 min
BradycardiaHR < 30 beats/min1 min
HR = 30–40 beats/min5 min
HypotensionSYS BP <70 mmHg1 min
SYS BP <91 mmHg30 min
HypertensionSYS BP ≥180 mmHg60 min
SYS BP ≥220 mmHg1 min
Sensium Vitals [38]
Alarm strategySpecificationParameter setting
Threshold individualizationAlarm thresholds were defined using the cumulative density function (CDF), for each vital sign based on the first 24-h monitoring. Lower and upper alarm vital sign thresholds corresponds to the lower (CDFlow) and upper (CDFhigh) percentiles of the individual CDF. Default alarm thresholds were used for the first 24 h.- CDFlow: 0.1%; CDFhigh: 99.9%
- CDFlow: 0.5%; CDFhigh: 99.5%
- CDFlow: 1%; CDFhigh: 99%
Postoperative elevation of upper thresholdsThe standard upper alarm threshold is increased by a fixed percentage (POincrease) i.e., postoperative increase factor) for the first four days after surgery.- POincrease: 5% for HR/RR; 1% for T
- POincrease: 10% for HR/RR; 2.5% for T
- POincrease: 25% for HR/RR; 5% for T
Increase annunciation delay intervalThe length of the annunciation delay interval (Linterval) i.e., minimum number of successive abnormal measurements needed for generation of an alarm (default: 7 measurements, i.e., 14 min interval) is increased.- Linterval: 12 measurements
- Linterval: 17 measurements
- Linterval: 22 measurements
Daytime elevation of upper HR/RR thresholdsThe standard upper HR and RR threshold is increased by a fixed percentage (DTincrease)i.e., daytime increase factor) during daytime (8 a.m. to 10 p.m.).- DTincrease: 5% for HR; 15% for RR
- DTincrease: 10% for HR; 25% for RR
- DTincrease: 25% for HR; 35% for RR
Nighttime reduction of lower HR/RR thresholdsThe standard lower HR and RR threshold is decreased by a fixed percentage (NTincrease) i.e., nighttime decrease factor) during nighttime (10 p.m. to 8 a.m.).- NTdecrease: 5% for HR; 15% for RR
- NTdecrease: 10% for HR; 25% for RR
- NTdecrease: 25% for HR; 35% for RR
Slope-based alarmsAn alarm is generated only in case the slope of the linear regression line calculated over a past time interval (Tslope) exceeds a preset threshold: HR slope: ± 15 bpm over Tslope; RR slope: ± 10 brpm over Tslope; T slope: ± 1 °C over Tslope.- Tslope: 4 H
- Tslope: 8 H
- Tslope: 12 H
Legends: RR: respiratory rate; SpO2: peripheral oxygen saturation; HR: heart rate; BP: blood pressure; TP: temperature; SYS: systolic; Min: minutes; Sec: seconds; H: hours; mmHg: millimeter of mercury.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Aagaard, N.; Aasvang, E.K.; Meyhoff, C.S. Discrepancies between Promised and Actual AI Capabilities in the Continuous Vital Sign Monitoring of In-Hospital Patients: A Review of the Current Evidence. Sensors 2024, 24, 6497. https://doi.org/10.3390/s24196497

AMA Style

Aagaard N, Aasvang EK, Meyhoff CS. Discrepancies between Promised and Actual AI Capabilities in the Continuous Vital Sign Monitoring of In-Hospital Patients: A Review of the Current Evidence. Sensors. 2024; 24(19):6497. https://doi.org/10.3390/s24196497

Chicago/Turabian Style

Aagaard, Nikolaj, Eske K. Aasvang, and Christian S. Meyhoff. 2024. "Discrepancies between Promised and Actual AI Capabilities in the Continuous Vital Sign Monitoring of In-Hospital Patients: A Review of the Current Evidence" Sensors 24, no. 19: 6497. https://doi.org/10.3390/s24196497

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop