Next Article in Journal
Road Speed Prediction Scheme by Analyzing Road Environment Data
Previous Article in Journal
Blockchain-Based Authentication and Trust Management Mechanism for Smart Cities
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Sensors
Volume 22
Issue 7
10.3390/s22072605
sensors-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Performance Evaluation of a Smart Bed Technology against Polysomnography

by
Farzad Siyahjani
*,
Gary Garcia Molina
,
Shawn Barr
and
Faisal Mushtaq
Sleep Number® Labs, San Jose, CA 95113, USA
*
Author to whom correspondence should be addressed.
Sensors 2022, 22(7), 2605; https://doi.org/10.3390/s22072605
Submission received: 18 February 2022 / Revised: 22 March 2022 / Accepted: 24 March 2022 / Published: 29 March 2022
(This article belongs to the Section Intelligent Sensors)
Browse Figures
Versions Notes

Abstract

:
The Sleep Number smart bed uses embedded ballistocardiography, together with network connectivity, signal processing, and machine learning, to detect heart rate (HR), breathing rate (BR), and sleep vs. wake states. This study evaluated the performance of the smart bed relative to polysomnography (PSG) in estimating epoch-by-epoch HR, BR, sleep vs. wake, mean overnight HR and BR, and summary sleep variables. Forty-five participants (aged 22–64 years; 55% women) slept one night on the smart bed with standard PSG. Smart bed data were compared to PSG by Bland–Altman analysis and Pearson correlation for epoch-by-epoch HR and epoch-by-epoch BR. Agreement in sleep vs. wake classification was quantified using Cohen’s kappa, ROC analysis, sensitivity, specificity, accuracy, and precision. Epoch-by-epoch HR and BR were highly correlated with PSG (HR: r = 0.81, |bias| = 0.23 beats/min; BR: r = 0.71, |bias| = 0.08 breaths/min), as were estimations of mean overnight HR and BR (HR: r = 0.94, |bias| = 0.15 beats/min; BR: r = 0.96, |bias| = 0.09 breaths/min). Calculated agreement for sleep vs. wake detection included kappa (prevalence and bias-adjusted) = 0.74 ± 0.11, AUC = 0.86, sensitivity = 0.94 ± 0.05, specificity = 0.48 ± 0.18, accuracy = 0.86 ± 0.11, and precision = 0.90 ± 0.06. For all-night summary variables, agreement was moderate to strong. Overall, the findings suggest that the Sleep Number smart bed may provide reliable metrics to unobtrusively characterize human sleep under real life-conditions.

1. Introduction

Increased awareness of the importance of sleep, together with the advancement of consumer electronics technology, has resulted in the emergence of consumer sleep-tracking devices [1] that combine outputs from sensors with apps [2,3,4] and proprietary software [3]. These products use one or a combination of physiological signals to evaluate sleep: motion and movement (e.g., using wrist-worn accelerometers [3] or bedside radar detection [5]), brain activity signals (e.g., using dry-electrode headbands [6,7]), heart rate (HR; e.g., using chest belts [1]), breathing patterns (e.g., using chest patches [1,3]), cardiac activity (e.g., using optical pulse sensors [3]), and skin temperature (e.g., using wearable thermometers [1]).
The Sleep Number smart bed evaluates the sleeper’s bed presence, body movements, heartbeat, and breathing patterns in real time to determine the duration and quality of sleep. Similar to bed surfaces in sleep laboratories or specialized medical care units [8,9,10,11] and to under-mattress monitoring devices used by consumers [8], the smart bed sensing technology uses high-resolution full-body ballistocardiography (BCG) readings from an inflatable air bladder within the bed mattress, underlying the entire body of a single sleeper. The use of BCG as a passive-sensing technology enables the measurement of motion, position changes, breathing, and small movements within the body such as those generated by the ejection of blood with each heartbeat [12,13]. The smart bed sensors automatically begin data collection as soon as the user enters the bed and stop once the user leaves. Smart bed users receive information about estimated sleep timing, duration, and quality via a consumer smart phone app on a daily basis. Processed data are sent to the cloud and stored for further analysis.
Most sleep-tracking products require the sleeper to carry, wear, or attach a device, monitor battery charge, and/or manually mark “time in bed” to measure sleep behavior. Movement alone has limited specificity in tracking sleep patterns [14]. Newer sleep-tracking devices incorporate information from cardiovascular signals [15,16] in addition to triaxial accelerometers, which have contributed to increased accuracy in sleep/wake classification [17,18]. Despite these improvements, sleep-tracking devices have some disadvantages, including inaccuracy due to single location of signal sensing, inconvenience, susceptibility to environmental disturbances (e.g., external vibrations), device misplacement or displacement during sleep, sensor damage, limited battery life, and the need for manual intervention to mark sleep-related events such as sleep intent or bedtime. For these reasons, 34% of users abandon their device within 9 weeks [19]. In addition, a limited number of wearables and nearables have been clinically validated against standard sleep monitoring technologies [20].
Polysomnography (PSG) is the gold standard for sleep monitoring in clinical, field, and home settings. PSG includes an electroencephalogram, electrocardiogram (ECG), electromyogram, electrooculogram, and the measurement of breathing signals. PSG also represents the gold standard by which consumer sleep technologies are validated. Although its use in longitudinally monitoring sleep in real-life (ecologically valid) conditions is possible, it is expensive, time consuming, and imposes a high burden on the user.
The smart bed platform offers a potential alternative for tracking sleep-dependent cardiorespiratory function and sleep versus wake states without the need for a wearable device or PSG. As a first step to validating the use of the smart bed platform in sleep research, the present study performed three analyses evaluating the performance of the smart bed against PSG. First, the smart bed was assessed for its ability to measure all-night HR and breathing rate (BR). HR and BR data generated by the smart bed were compared to HR and BR data measured by the PSG system’s ECG and flow channels, respectively. Comparisons were made at the epoch-by-epoch level using the Bland–Altman method and Pearson correlation. Second, epoch-by-epoch comparisons were made to assess the smart bed’s sensitivity for detecting sleep, specificity of detecting wake, accuracy, precision, Cohen’s kappa value, prevalence and bias-adjusted kappa, and area under the receiver operating characteristic curve (AUC). Finally, all-night summary metrics including sleep efficiency (SE), sleep onset latency (SOL), total sleep time (TST), and wake after sleep onset (WASO) were compared between the smart bed estimates relative to PSG using Bland–Altman metrics.

2. Materials and Methods

2.1. Participants

Forty-five adults who were self-described ‘normal sleepers’ volunteered for the study and participated in an overnight laboratory PSG at the Lakeland Sleep Center, Minneapolis, MN, between February 2016 and March 2016. Demographic data are provided in Table 1. All participants were over 18 years of age and provided written consent to participate in product research with the Sleep Number Corporation. To be included in the study, each participant was required to spend one full night in the sleep laboratory between 21 February 2016, and 31 March 2016, and to provide their body mass index (BMI). No exclusion criteria were applied for participant recruitment, and the participants were free of medical, neurological, or psychiatric illnesses based on a health status questionnaire. The study was designed for product research by the Sleep Number Corporation, and it was reviewed by the University of Chicago Institutional Review Board (IRB) for the analysis of de-identified smart bed and PSG data. The IRB gave the study exempt status.
Before participant recruitment, the target sample size was calculated using confidence interval analysis for correlations, as outlined by Moinester and Gottfried [21]. For this analysis, confidence intervals for the correlations between smart bed-measured HR or BR versus PSG were used. This approach showed that the sample size needed for a correlation of 0.85 and confidence interval width of 0.1 was 36. This figure was rounded up to 40, which became the target threshold for participant recruitment. As described in the Participants section, no exclusion criteria were applied, and there was no bias in the selection of participants; hence, all 45 participants were included for further analysis.

2.2. Protocol

Participants underwent one night of simultaneous PSG and smart bed recordings in a laboratory setting. All studies were performed while the participants were sleeping alone on the same model of queen-size (60 in × 80 in) smart bed. Smart bed data were collected every second and uploaded in real time to a secure cloud database. At the end of each recording, PSG data from lights out to lights on were extracted and uploaded to a secure database for further processing. Timing of lights off and lights on were based on each participant’s usual sleep schedule; thus, time in bed varied from one participant to the next. All participants’ data were anonymized.

2.3. Overnight Polysomnography

A standard montage was used for the PSG recordings, including 6-lead electroencephalogram signals (F3-A2, C3-A2, O1-A2, F4-A1, C4-A1, O2-A1), electrooculogram, electromyogram, electrocardiogram, thoraco-abdominal effort via respiratory impedance plethysmography, oronasal airflow using thermistor and nasal pressure base flow measurement, SpO2 with pulse oximetry, and body position (SomnoStar Sleep System, Vyaire Medical, Mettawa, IL, USA). PSG sleep stages were independently scored by three independent, registered sleep technicians based on the American Academy of Sleep Medicine guidelines, and the final sleep stage for each epoch was chosen as the stage scored by at least two of the three technicians. In the case of disagreement among the three scorers, a sleep-staging expert (G.G.M.) made the final determination. The resulting consensus scores were used in the evaluation of the smart bed sleep/wake detection.

2.4. Sleep Monitoring with Smart Bed Technology

Smart bed technology is built on an advanced Internet of Things [22] architecture that incorporates devices embedded with sensors, software, network connectivity, and other necessary electronics, enabling the collection and exchange of data in real-time. The bed’s embedded software employs advanced biomedical signal processing and machine-learning algorithms to determine physiological biomarkers. Processed data are sent to the cloud every second via WiFi, where they are stored and further analyzed to generate summary data for the night and sleep metrics. In the event of Wi-Fi or network failure, the device can store up to 4 h of data and send it once the connection is re-established.
The Sleep Number smart bed uses high-resolution, full-body BCG sensing that collects data at a rate of 1000 samples/second. Figure 1 shows an example of a BCG recording from the smart bed during a motionless, 16 s-long, sleep segment. This figure illustrates the decomposed breathing and cardiac components of the BCG signal. The smart bed algorithms determine the instantaneous HR and BR, which can track changes in HR and BR over time, creating longitudinal profiles of HR and BR. The data captured by the smart bed detect changes in HR and BR as the body progresses from wakefulness through stages of sleep [23,24,25,26]. The smart bed’s deep neural network (DNN) was developed in a separate study on over 1000 h of synchronized high-resolution BCG and PSG data to score 30 s epochs as wake vs. sleep. The smart bed DNN was trained to map the cardiac and respiratory signals and motion data embedded within the high-resolution BCG signal to PSG-derived sleep/wake states. These states were determined for each PSG-defined epoch. The overall approach is similar to that of other DNNs that have successfully detected different sleep stages using PSG signals [27,28,29,30,31,32,33,34].

2.5. Data Curation and Cleaning and PSG-Smart Bed Synchronization

No data from any given participant were excluded due to the identification by PSG of an undiagnosed sleep disorder. Any epoch with PSG artifacts that led to missing HR or BR detection was excluded from the analysis. From the remaining epochs, any epoch for which the smart bed system did not produce a HR or BR reading was noted as a “not covered” epoch. All remaining epochs were included in the analysis. Details on epoch coverage are provided in Appendix A.
Smart bed data were synchronized with PSG data using the time stamps associated with the two processor clocks. However, an additional manual alignment step was performed to correct for any possible drift between the internal clocks of the PSG and smart bed systems. In this manual step, large-amplitude movements were used as the basis for determining the time shift between PSG and smart bed data collection, since these movements were clearly visible in both the PSG and smart bed outputs. Movement was detectable in both systems because the BCG sensor output from the smart bed was electronically forked, with one line serving as an input to the PSG via a DC channel.
Lastly, an exploratory analysis using cross-correlation calculations was conducted on the overnight temporal profiles of HR and BR assessed from PSG as compared to the corresponding smart bed signals. After applying a 15 min smoothing window, these profiles were aligned to the PSG-based hypnogram to determine whether the expected physiological changes in HR and BR occurred in parallel in the PSG-based profile versus in the smart-bed-based profile. An individual example is illustrated in Figure 2. In the case of this representative subject, the cross correlation between PSG HR and smart bed HR was 0.6195 (p < 0.0001), and the cross correlation between PSG HR and smart bed HR was 0.490 (p < 0.0001), consistent with the visual concordance apparent in Figure 2. This exploratory analysis was used to identify conditions where dissociations between PSG and smart bed signals may be more likely to occur.

2.6. Epoch-by-Epoch Analysis

Following data alignment and exclusion of epoch with PSG artefact or/and with no coverage from the smart bed technology, epoch-by-epoch comparisons were made between the smart bed and PSG to assess the ability of the smart bed to detect HR and BR and consequently sleep/wake states. Epoch values for HR and BR were obtained by computing the mean of the instantaneous HR or BR values detected within each 30 s epoch.

2.7. Analysis of All-Night Summary Variables

All-night summary variables, including mean HR, mean BR, sleep onset latency (SOL), wake after sleep onset (WASO), total sleep time (TST), and sleep efficiency (SE) were compared between the smart bed and PSG. The reference BR was calculated from the PSG’s flow and effort signals. The channel with the dominant physiological pattern was visually examined and manually selected. Afterward, local maxima for every breathing cycle were detected using a peak detection algorithm. The reference HR was calculated from the PSG ECG signal by applying a QRS peak detection algorithm [35]. Detected R peaks were visually inspected and manually removed or adjusted as needed. After artifact removal, mean overnight values for PSG-based HR and BR were calculated by averaging the HR and BR values across all epochs that were scored. Additionally, HR and BR were characterized for each PSG-defined sleep stage, as described in Appendix A (Table A1).
SOL was defined as the time (in minutes) from lights off to the first epoch scored as any stage other than wake. WASO was computed as the total wake time (in minutes) between sleep onset and sleep offset. TST was computed as the total number of minutes scored as sleep. SE was computed as the ratio between TST and total time in bed (PSG lights off to lights on).

2.8. Statistical Analysis

HR and BR analyses were performed using Matlab version R2020 (MathWorks, Natick, MA, USA), and generalized linear model analyses were performed using Matlab version 2019b. All other analyses were performed using Python version 3.8 (Scikit-learn 0.22, NumPy 1.13) and R, version 4.1.1. For each participant, epoch-by-epoch and summary all-night PSG and smart bed measures for HR and BR were matched and compared using the Bland–Altman method. The Bland–Altman method was also used to compare PSG and smart bed values for epoch-by-epoch sleep/wake detection and for all-night summary variables.
Proportional bias and heteroscedasticity were tested visually and using linear regression, as outlined by Menghini et al. [36]. Additionally, the differences in measurement between the smart bed and PSG were tested for normality using the Shapiro–Wilk test.
Bland–Altman [37] plots were generated to assess the level of agreement (LOA) between PSG and equivalent smart bed measures using the guideline proposed by Haghayegh et al. [38]. In the case of negligible proportional bias and lack of heteroscedasticity, comparisons of a single measurement per subject, mean difference (Bias), lower and upper agreement limits (mean difference ± 1.96 × standard deviation [SD]) with their corresponding 95% confidence intervals (CIs) between PSG and smart bed measures were calculated. For repeated measures of epoch-by-epoch analyses, the within- and between-subject variance in measurement difference was obtained and used to compute the bias and LOA along with their corresponding CI [38]. A positive bias indicated that the smart bed platform underestimated the PSG-derived measure, and a negative bias indicated that it overestimated the PSG-derived measure. Pearson correlation coefficients and 95% CI also were calculated.
For epoch-by-epoch sleep/wake detection, Cohen’s kappa, a chance-adjusted measure of agreement [39], was computed for each subject, and the mean and standard deviation were calculated for the entire sample. Because the high imbalance between sleep epochs (74% of all epochs) and wake epochs (26% of all epochs) can result in the underestimation of the true kappa value [39] for overnight profiles, we estimated an adjusted kappa value that accounts for prevalence and bias, as previously described [39]. The following metrics were computed: accuracy (percentage of epochs classified correctly); balanced accuracy (average of the wake and sleep epochs classified correctly); sensitivity (percentage of sleep epochs classified correctly); specificity (percentage of wake epochs classified correctly); precision (proportion of correctly detected sleep epochs among all detected sleep epochs); signal processing d′ (difference in SD between theoretical signal and noise distributions, which is indicative of the model’s ability to detect signals); and AUC.

3. Results

3.1. Dataset Characteristics

The demographic characteristics of the participants and summary PSG variables are shown in Table 1.

3.2. Analysis of Epoch-by-Epoch and All-Night BR and HR

A comparison of smart bed and PSG epoch-by-epoch HR and BR and mean HR and BR measurements during the entire sleep period is presented in Figure 3. Epoch-by-epoch and all-night average differences in HR, as measured by the smart bed and PSG, were tested for the presence of a proportional bias or heteroscedasticity and for normality. No significant dependency was observed, and the differences were normally distributed. After the removal of epochs containing PSG-related artifacts, the smart bed had a coverage rate of 93.6% for HR.
The Bland–Altman analysis for epoch-by-epoch HR (in beats per minute) showed strong agreement between the smart bed and PSG (bias = −0.23 (95% CI: −1.26, 0.78); lower LOA = −6.65 (95% CI: −8.33, −4.97); upper LOA = 6.19 (95% CI: 4.48, 7.85); Figure 3; Table A2) and a strong correlation (r = 0.81 (95% CI: 0.81, 0.82)). The smart bed’s accuracy for detecting HR was comparable to PSG over different sleep stages (see Appendix A). For all-night HR, the Bland–Altman analysis showed a clinically insignificant bias between mean HR from the smart bed versus PSG (bias = −0.15 (95% CI: −1.08, 0.77); lower LOA = −6.21 (95% CI: −7.85, −4.57); upper LOA = 5.90 (95% CI: 4.26, 7.54); Table A2), and a strong correlation was observed between the mean smart bed- and PSG-derived HR (r = 0.94 (95% CI: 0.91, 0.96)).
Epoch-by-epoch and all-night average differences in BR (in breaths per minute) as measured by the smart bed or PSG were similarly tested for the presence of a proportional bias or heteroscedasticity, and for normality. No significant dependency was observed, and the differences were normally distributed. The smart bed had a BR reading for 99.4% of the epochs for which there was also a PSG BR reading (i.e., a 99.4% coverage rate). Comparisons of the smart bed and PSG epoch-by-epoch and all-night average BR measurements in all participants are presented in Figure 3. Bland–Altman comparison of epoch-by-epoch BR between the smart bed and PSG showed insignificant bias, with narrow upper and lower LOAs (bias = 0.08 (95% CI: −0.25, 0.41); lower LOA = −2.02 (95% CI: −2.36, −1.68); upper LOA = 2.19 (95% CI: 1.85, 2.53; Table A3). Likewise, Bland–Altman comparison of all-night BR between the smart bed and PSG showed insignificant bias, with narrow upper and lower LOAs in mean BR (bias = 0.09 (95% CI: −0.03, 0.21); lower LOA = −0.69 (95% CI: −0.90, −0.48); upper LOA = 0.87 (95% CI: 0.66, 1.08; Table A3). Additionally, a strong correlation was observed between the mean smart bed- and PSG-derived BR over the entire night (r = 0.96 (95% CI: 0.94, 0.98)).

3.3. Analysis of Epoch-by-Epoch and All-Night Sleep/Wake Detection

The entire analysis included 39,855 epochs after data curation and cleaning. Performance measures for the smart bed’s ability to predict sleep and wake states correctly are provided in Table 2. The receiver operating characteristic curve (ROC) for smart bed prediction of sleep and wake states is shown in Figure 4.

3.4. Analysis of All-Night Summary Variables

Smart bed-collected physiological and motion signals were processed with the DNN to derive sleep/wake epochs and subsequently sleep metrics such as SOL, WASO, TST, and SE. These variables were compared to consensus PSG sleep metrics. A comparison of smart bed and PSG measurements for SOL, WASO, TST, and SE is reported in Table 3, and Bland–Altman analysis plots are shown in Figure 5.
The Bland–Altman analysis of SOL (in minutes) showed proportional bias and heteroscedasticity, where the difference between the measurements increased as the PSG SOL value increased. In addition, the differences were not normally distributed (Shapiro–Wilk test, p < 0.05); consequently, they were log-transformed for further analysis. We used basic bootstrapping methods to calculate confidence intervals and linear regression for modeling both the bias and LOAs for plotting and reporting of the Bland–Altman metrics [36]. There was strong agreement between SOL measurements from the smart bed and PSG (bias = 3.0 (95% CI: −1.9 to 7.7); LOAs = bias ± 2.5 [−3.4 + 0.9 × ref]).
The Bland–Altman analysis for WASO (in minutes) showed moderate heteroscedasticity, and the differences were normally distributed. There was moderate agreement between WASO measurements from the smart bed and PSG (bias = 15.28 − 0.49 × ref; LOAs = bias ± 2.46 [16.26 + 0.06 × ref]). Differences in TST (in minutes) were normally distributed and proportional to the size of measurement (r = −0.35 (95% CI: −0.52 to −0.13)). The bias (150.61 − 0.35 × ref) and LOAs (bias ± 61.78) were represented as a function of the size of measurement. Differences in SE (expressed as a percentage) were normally distributed, and the Bland–Altman analysis showed moderate heteroscedasticity and agreement between SE (%) measurements from the smart bed and PSG (bias = 0.48–0.51 × ref; LOAs = bias ± 0.12; Figure 5).

3.5. Influence of Demographic and Health Factors on PSG vs. Smart Bed Temporal Concordance

We further examined the role of BMI, sex, age, and AHI on the overall temporal concordance between PSG and smart bed estimations using the coefficient of cross correlation between the corresponding profiles. For HR, the mean ± SD of the cross-correlation was 0.4498 ± 0.2371 (p < 0.001), a positive but low correlation. For BR, the mean ± SD of the cross correlation was significantly lower than for HR (0.3629 ± 0.2033; p = 0.0312).
In a multiple linear regression with the cross-correlation value for HR as the dependent variable and sex, age, and BMI as covariates, the association with age was not significant (p = 0.4628). There was a significant impact of sex, with women having lower concordance between PSG and smart bed estimations (mean ± SE: 0.3945 ± 0.0460 versus 0.5393 ± 0.0507 in men, p = 0.0471). Further, increasing BMI was significantly associated with lower cross correlation (p = 0.0436). For BR, there were no significant impacts of sex, age, or BMI.
Six of the 45 participants had a negative cross correlation between PSG-based and smart bed-based HR or/and BR, suggestive of substantial discrepancy between temporal variations of PSG versus smart bed signals. Four of these six participants had undiagnosed but clinically significant sleep disorders (obstructive sleep apnea with AHI > 15; periodic leg movement with PLM > 10): subject 6—AHI of 23.9 and PLM of 17.3; subject 11—AHI of 68.9; subject 15—PLM of 46.5; subject 30—PLM of 80.9. This would have excluded them from the study if reported.

4. Discussion

We evaluated the performance of the Sleep Number smart bed system in measuring cardiorespiratory endpoints characteristic of sleep and wakefulness, sleep vs. wake status, and summary sleep variables using PSG as the gold standard for comparison. Our dedicated study conformed with multiple recommendations of the guidelines for validation of wearables for sleep outcomes proposed by Depner et al. [20], including recordings obtained under laboratory conditions, manual PSG scoring according to AASM rules, careful synchronization of PSG versus smart bed signals, use of epoch-by-epoch data, use of Bland–Altmann metrics, and use of sensitivity and specificity measures.
The results showed that the measurements of overnight HR and BR generated by the smart bed had biases of <1 beat per minute and <1 breath per minute, respectively. These biases are sufficiently small that they are unlikely to have clinical significance.
With an AUC value of 0.86, the smart bed platform differentiated sleep and wake states reasonably well. The smart bed’s performance was comparable to that of actigraphy found in studies comparing actigraphy to PSG. For example, the smart bed’s accuracy for sleep/wake detection was 0.86 ± 0.11, similar to the accuracy reported for the ActiGraph Link using the Cole-Kripke algorithm (0.891 ± 0.046), the Sadeh algorithm (0.880 ± 0.054), or the Spectrum Plus algorithm (0.904 ± 0.050) [31]. The smart bed’s balanced accuracy was 0.75 ± 0.12, comparable to the ActiGraph Link with Cole-Kripke algorithm (0.752 ± 0.070), ActiGraph Link with Sadeh algorithm (0.779 ± 0.071), and Actiwatch Spectrum Plus algorithm (0.674 ± 0.065) [31]. The kappa value for the smart bed was 0.45 ± 0.17, similar to the kappa values reported for the ActiGraph Link using the Cole–Kripke algorithm (0.482 ± 0.120), the Sadeh algorithm (0.487 ± 0.146), or the Spectrum Plus algorithm (0.424 ± 0.121) [31].
Adjusting for the higher prevalence of sleep compared to wake during overnight measurements, the bias- and prevalence-adjusted kappa value was 0.74 ± 0.11, showing a marked improvement in agreement. Similar to typical actigraphy, due to the prevalence of sleep epochs (of which approximately 74% were sleep in this study), the machine learning model tended to have higher sensitivity for detecting sleep and lower specificity for detecting wake. Specifically, the smart bed’s sensitivity was 0.94 ± 0.05, similar to the sensitivities reported for ActiGraph Link using the Cole–Kripke algorithm (0.936 ± 0.050), the Sadeh algorithm (0.912 ± 0.064), or the Spectrum Plus algorithm (0.982 ± 0.012; [31]). Likewise, its specificity for detecting wake was 0.48 ± 0.18, similar to the specificities reported for ActiGraph Link using the Cole–Kripke algorithm (0.568 ± 0.163), the Sadeh algorithm (0.647 ± 0.163), and the Spectrum Plus algorithm (0.366 ± 0.136) [31].
The performance of the smart bed was also comparable to that of a bed sensor using BCG via pressure-sensitive foil electrodes that detects heartbeat intervals compared with PSG-derived ECG [40,41,42]. For all-night HR, the smart bed’s bias (in beats per minute) was −0.15 (95% CI: −1.08, 0.77), with a lower LOA of −6.21 (95% CI: −7.85, −4.57) and an upper LOA of 5.90 (95% CI: 4.26, 7.54), comparable to the Emfit QS, which had a bias of 0.03 (95% CI: −0.77, 0.83), a lower LOA of −4.37 (95% CI: −5.82, −4.19), and an upper LOA of 4.43 (95% CI: 4.25, 5.88) [8]. Similarly, the smart bed’s bias for all-night BR (in breaths per minute) was 0.09 (95% CI: −0.03, 0.21), with a lower LOA of −0.69 (95% CI: −0.90, −0.48) and an upper LOA of 0.87 (95% CI: 0.66, 1.08). By comparison, the Emfit QS system’s bias for BR was −0.14 (95% CI: −0.57, 0.29), with a lower LOA of −2.49 (95% CI: −2.81, −2.40) and an upper LOA of 2.22 (95% CI: 2.12, 2.53) [8].
All-night-level summary data permit comparison of all-night sleep metrics between the smart bed platform and PSG. Bland–Altman comparisons for an SOL of less than 20 min indicate that the smart bed provided a relatively accurate estimate; however, as SOL increased, the accuracy of detecting SOL decreased. Given that the smart bed does not have direct access to brain activity (which is the gold standard signal to track sleep onset [43]), this is a reasonable level of agreement with PSG.
Bland–Altman comparisons also showed that the smart bed had reasonable accuracy at lower WASOs, but as wake time increased, the platform tended to overestimate WASO. Thus, the smart bed estimated WASO with moderate bias and level of agreement with PSG values. A similar phenomenon was observed for SE; the smart bed was less accurate for estimating SE in studies with lower SE, but its accuracy improved in studies with higher SE. Likewise, the smart bed estimation of sleep was less accurate for studies in which PSG detected shorter sleep durations. Overall, this indicates that the smart bed platform was able to generate a more reliable estimate of TST and other sleep metrics when participants spent less time awake in bed.
In this validation study, we studied a sample of 45 participants who reported having no acute health problems. Higher BMI appeared to negatively influence the accuracy of HR smart bed estimation, which had better concordance with PSG in men than in women. AHI did not significantly influence the accuracy of HR and BR measurements, mainly owing to the small number of patients with AHI > 5.
The validation data presented in this study suggest that the smart bed may provide reliable, longitudinal estimation of sleep quality in an ecologically valid environment, enabling access to sleep metrics in a large population for longer periods of time than is currently possible with PSG [44,45].
The study was limited by a relatively small sample size (N = 45) and a single night of simultaneous PSG and smart bed recording in the laboratory. The sample was fairly homogenous in terms of age and degree of adiposity; individuals with previously diagnosed sleep disorders were not included.
Future studies are needed to better characterize the accuracy of the smart bed, with particular regard to longitudinal repeatability and reliability, using data collected over multiple nights for each participant. Further validation is also required to assess the smart bed as a potential screening tool for sleep disorders. Importantly, more research is needed to understand how AHI, periodic leg movement during sleep (PLMS), BMI, and gender influence smart beds’ HR measurements.

5. Conclusions

Sleep-dependent changes in cardiorespiratory activity can be reliably detected by the Sleep Number smart bed, and the smart bed platform differentiated between sleep and wake states with performance similar to actigraphy or pressure-sensitive foil electrode bed sensors. All-night summary data analyses indicated that the smart bed provided relatively accurate detection of SOL, WASO, TST, and SE. Future work will build on these results by focusing on sleep-stage detection using the smart bed platform.

Author Contributions

Conceptualization: All authors; Methodology: F.S.; Software: F.S. and S.B.; Validation: F.S.; Formal Analysis: F.S., G.G.M. and S.B.; Investigation: F.S.; Resources: G.G.M.; Data Curation: F.S.; Writing—Original Draft Preparation: F.S., G.G.M. and S.B.; Writing—Review and Editing: F.S., G.G.M. and F.M.; Visualization: F.S.; Supervision: F.M.; Project Administration: F.M.; Funding Acquisition: F.M. All authors have read and agreed to the published version of the manuscript.

Funding

The study was funded privately by Sleep Number Corporation and all authors are employees of Sleep Number. There was no grant funding or associated grant number.

Institutional Review Board Statement

The study was reviewed by the University of Chicago Institutional Review Board and was given exempt status.

Informed Consent Statement

Informed consent was obtained from all participants involved in the study.

Data Availability Statement

Data are not publicly available due to limitations in the consent form.

Acknowledgments

The authors would like to thank Omid Sayadi for his assistance with study planning and data review process, Pete Bils for his help with participant enrollment and managing the sleep lab relations, Gueorgui Martynov, who provided support and expertise with the PSG data acquisition, and Cristina Jocson for her assistance with the study and data review process. Eve Van Cauter and Orfeu Buxton provided a critical review of several drafts of this article. Van Cauter is a member of the Scientific Advisory Board of the Sleep Number Corporation and receives a yearly honorarium; she is not a stockholder of the Sleep Number Corporation. Buxton (consultant) has no formal affiliation with Sleep Number Corporation. Medical writing support for this manuscript was provided by Sandra Page, of Oxford PharmaGenesis Inc., Newtown, PA, USA, and was funded by the Sleep Number Corporation.

Conflicts of Interest

All authors are employees of Sleep Number Corporation, the manufacturer of the Sleep Number smart bed, and are responsible for the design, execution, analysis, and conclusions of the study presented in this article, as well as the decision to publish the results. All authors have patents or patent applications associated with the technology described in this paper.

Appendix A

Profiles of HR and BR with Regard to Rapid Eye Movement (REM) and Non-REM (NREM) Sleep Stages

We utilized data collected from a 36-year-old woman, with a BMI of 21.6 and AHI of 0.1 (indicating a lack of sleep-disordered breathing) to assess BR and HR during REM and NREM sleep stages. Expected behavior was captured by both PSG and the smart bed. As sleep progressed from onset through the stages of NREM sleep, BR increased slightly and became more regular; these expected physiological changes were detected equally by PSG and smart bed, as shown by the decreased average standard deviation of BR through stages of N1, N2, and N3, irrespective of the method of recording (Table A1). Similar results were seen for HR, where the wake stage had the highest mean and standard deviation in HR relative to all other sleep stages (Table A1).
Table
Table A1. Mean epoch-by-epoch HR and BR during different sleep stages. The standard deviation of instantaneous HR and BR were computed on a 15 min, centered, moving window and are grouped and averaged based on PSG sleep stages, representing the 15 min HR and BR variability.
Table A1. Mean epoch-by-epoch HR and BR during different sleep stages. The standard deviation of instantaneous HR and BR were computed on a 15 min, centered, moving window and are grouped and averaged based on PSG sleep stages, representing the 15 min HR and BR variability.
Heart Rate, bpmBreathing Rate, bpm
Mean (SD)Smart BedPSGSmart BedPSG
Wake70.58 (2.52)71.16 (3.78)14.56 (1.48)15.05 (1.10)
Light sleep67.91 (2.16)67.11 (2.87)14.96 (0.98)15.05 (0.90)
Deep sleep68.11 (1.42)67.24 (1.75)15.76 (0.66)15.66 (0.75)
REM67.85 (2.51)67.89 (2.94)16.00 (1.34)15.97 (1.12)
All epochs67.96 (2.04)67.72 (2.71)15.22 (1.04)15.31 (0.93)
bpm, beats per minute (HR) or breaths per minute (BR); BR, breathing rate; HR, heart rate; PSG, polysomnography; REM, rapid eye movement; SD, standard deviation.
During REM sleep, the breathing pattern was more variable than in NREM, with an overall increase in BR, as expected. REM sleep was characterized by a pronounced increase in HR and HR variability; this increase was captured by both PSG and the smart bed and was consistent with the mean data over the entire population (Table A1). These results suggest an increase in variability in BR and HR during the transition to REM sleep. Furthermore, both methods detected an increase in BR and HR variability through the transition to wake.
The breakdown of the epoch-by-epoch Bland–Altman results for different sleep stages, from lights off to lights on, are illustrated in Figure A1. A minor bias and lower correlation are observed during the wake period, as would be expected due to less accurate measurements of physiological signals expected during wake time due to movements. However, as suggested by the low bias and LOA across different sleep stages (Table A1), the smart bed was able to track BR variations throughout the night.
Sensors 22 02605 g0a1 550
Figure A1. Comparison of the epoch-by-epoch BR and HR measurements by the smart bed versus PSG during different sleep stages. The difference between the smart bed’s versus PSG’s simultaneously measured values for 30 s epochs from all 45 participants was plotted as a function of the reference (PSG) measurement. Consistent insignificant bias and narrow LOA indicated that the smart bed platform was able to track physiological variations during sleep. In these epoch-by-epoch analyses (686–1036 epochs per subject), calculations of bias and LOA were performed as previously described [38]. LOA, level of agreement; PSG, polysomnography; REM, rapid eye movement.
Figure A1. Comparison of the epoch-by-epoch BR and HR measurements by the smart bed versus PSG during different sleep stages. The difference between the smart bed’s versus PSG’s simultaneously measured values for 30 s epochs from all 45 participants was plotted as a function of the reference (PSG) measurement. Consistent insignificant bias and narrow LOA indicated that the smart bed platform was able to track physiological variations during sleep. In these epoch-by-epoch analyses (686–1036 epochs per subject), calculations of bias and LOA were performed as previously described [38]. LOA, level of agreement; PSG, polysomnography; REM, rapid eye movement.
Sensors 22 02605 g0a1
The breakdown of the epoch-by-epoch Bland–Altman graphs for HR according to the sleep stage from lights out to lights on are provided in Table A2. REM and deep sleep stages have the highest correlation values (r = 0.83 (95% CI: 0.82–0.84) and r = 0.83 (95% CI: 0.83–0.84), respectively), reflecting higher body quiescence during these stages. In contrast, the wake and light-sleep stages had the lowest correlation values (r = 0.79 (95% CI: 0.78–0.80) and r = 0.81 (95% CI: 0.81–0.82), respectively), mostly reflecting movements during the wake and light-sleep stages. Bland–Altman analysis of HR (bpm) across all epochs showed strong agreement between smart bed and PSG (bias = −0.23 (95% CI: −1.26 to 0.78); lower LOA = −6.65 (95% CI: −8.33 to −4.97); upper LOA = 6.19 (95% CI: 4.48–7.85)) and a strong correlation (r = 0.81 (95% CI: 0.81–0.82)).
Table
Table A2. Summary of correlation and Bland–Altman metrics for HR for all participants. The number of epochs refers to epochs that were included in the analysis of each group. No coverage refers to the percentages of epochs for which the smart bed did not produce a HR reading, but a PSG HR reading was available. A positive (or negative) bias indicated that the smart bed underestimated (or overestimated) the PSG-derived measure.
Table A2. Summary of correlation and Bland–Altman metrics for HR for all participants. The number of epochs refers to epochs that were included in the analysis of each group. No coverage refers to the percentages of epochs for which the smart bed did not produce a HR reading, but a PSG HR reading was available. A positive (or negative) bias indicated that the smart bed underestimated (or overestimated) the PSG-derived measure.
DifferencesCorrelation
Mean [95% CI]ΔHR (bpm)Limits of Agreement (bpm)rNumber of Epochs (Percent No Coverage)
Over entire night−0.15 [−1.08, 0.77]−6.21 [−7.85, −4.57]–5.90 [4.26, 7.54]0.94 [0.91–0.96]-
Wake0.57 [−0.5, 1.65]−6.28 [−8.04, −4.53]–7.43 [5.68, 9.19]0.79 [0.78–0.80]5891 (5.9%)
Light sleep−0.36 [−1.52, 0.79]−7.76 [−9.67, −5.86]–7.03 [5.13, 8.94]0.81 [0.81–0.82]19,555 (7.1%)
Deep sleep−0.87 [−1.87, 0.13]−7.18 [−8.80, −5.56]–5.45 [3.83, 7.07]0.83 [0.83–0.84]5870 (0.40%)
REM0.06 [−0.98, 1.09]−6.48 [−8.16, −4.8]–6.59 [4.91, 8.28]0.83 [0.82–0.84]5934 (7.31%)
All epochs−0.23 [−1.26, 0.78]−6.65 [−8.33, −4.97]–6.19 [4.48, 7.85]0.81 [0.81–0.82]37,250 (6.40%)
bpm, beats per minute; HR, heart rate; PSG, polysomnography; REM, rapid eye movement.
All-epoch Bland–Altman analysis for BR (bpm) showed a strong agreement between the smart bed and PSG (bias = 0.08) and a positive correlation (r = 0.71; Table A3). The light-sleep and deep-sleep stages had the highest correlation values with BR (r = 0.75 and r = 0.77, respectively), reflecting higher body quiescence (i.e., less motion) during these stages. In contrast, the wake and REM stages had the lowest correlation values (r = 0.5 and r = 0.72, respectively), primarily due to movements during the wake and REM stages.
Table
Table A3. Summary of correlation and Bland–Altman metrics for BR for all participants. The number of epochs refers to epochs that were included in the analysis of each group. No coverage refers to the percentages of epochs for which the smart bed did not produce a BR reading, but a PSG BR reading was available. A positive (or negative) bias indicated that the smart bed underestimated (or overestimated) the PSG-derived measure.
Table A3. Summary of correlation and Bland–Altman metrics for BR for all participants. The number of epochs refers to epochs that were included in the analysis of each group. No coverage refers to the percentages of epochs for which the smart bed did not produce a BR reading, but a PSG BR reading was available. A positive (or negative) bias indicated that the smart bed underestimated (or overestimated) the PSG-derived measure.
DifferencesCorrelation
Mean [95% CI]ΔBR (bpm)Limits of Agreement (bpm)rNumber of Epochs (Percent No Coverage)
Over entire night0.09 [−0.03, 0.21]−0.69 [−0.90, −0.48]–0.87 [0.66, 1.08]0.96 [0.94–0.98]-
Wake0.49 [0.11, 0.88]−1.97 [−2.45, −1.49]–2.96 [2.47, 3.44]0.5 [0.48–0.52]6238 (1.3%)
Light sleep0.05 [−0.29, 0.4]−2.14 [−2.5, −1.78]–2.25 [1.89, 2.61]0.75 [0.75–0.76]20,919 (0.6%)
Deep sleep−0.10 [−0.47, 0.27]−2.46 [−2.87, −2.05]–2.26 [1.85, 2.67]0.77 [0.75–0.78]6105 (0.0%)
REM−0.02 [−0.41, 0.36]−2.45 [−2.9, −2.0]–2.4 [1.96, 2.85]0.72 [0.71–0.73]6368 (0.0%)
All epochs0.08 [−0.25, 0.41]−2.02 [−2.36, −1.68]–2.19 [1.85, 2.53]0.71 [0.70–0.71]39,630 (0.6%)
bpm, breaths per minute; BR, breathing rate; PSG, polysomnography; REM, rapid eye movement.
During sleep, the smart bed showed noticeable increases in BR, HR variability, and BR variability in REM sleep, and HR and BR variability decreases in deep NREM. The capability of the smart bed in tracking physiological variations throughout different sleep stages suggest that smart bed technology could be used for detecting sleep stages; this will be the focus of future studies.

References

  1. Ko, P.R.; Kientz, J.A.; Choe, E.K.; Kay, M.; Landis, C.A.; Watson, N.F. Consumer sleep technologies: A review of the landscape. J. Clin. Sleep Med. 2015, 11, 1455–1461. [Google Scholar] [CrossRef] [PubMed]
  2. Fietze, I. Sleep applications to assess sleep quality. Sleep Med. Clin. 2016, 11, 461–468. [Google Scholar] [CrossRef] [PubMed]
  3. Kolla, B.; Mansukhani, S.; Mansukhani, M. Consumer sleep tracking devices: A review of mechanisms, validity and utility. Expert Rev. Med. Devices 2016, 13, 497–506. [Google Scholar] [CrossRef] [PubMed]
  4. Stippig, A.; Hubers, U.; Emerich, M. Apps in sleep medicine. Sleep Breath 2015, 19, 411–417. [Google Scholar] [CrossRef]
  5. Toften, S.; Pallesen, S.; Hrozanova, M.; Moen, F.; Gronli, J. Validation of sleep stage classificiation using non-contact radar technology and machine learning (Somnofy®). Sleep Med. 2020, 75, 54–61. [Google Scholar] [CrossRef]
  6. Arnal, P.J.; Thorey, V.; Debellemaniere, E.; Ballard, M.E.; Bou Hernandez, A.; Guillot, A.; Jourde, H.; Harris, M.; Guillard, M.; van Beers, P.; et al. The Dreem Headband compared to polysomnography for electroencephalographic signal acquisition and sleep staging. Sleep 2020, 43, zsaa097. [Google Scholar] [CrossRef]
  7. Garcia-Molina, G.; Tsoneva, T.; Jasko, J.; Steele, B.; Aquino, A.; Baher, K.; Pastoor, S.; Pfundtner, S.; Ostrowski, L.; Miller, B.; et al. Closed-loop system to enhance slow-wave activity. J. Neural Eng. 2018, 15, 066018. [Google Scholar] [CrossRef]
  8. Ranta, J.; Aittokoski, T.; Tenhunen, M.; Alasaukko-Oja, M. EMFIT QS heart rate and respiration rate validation. Biomed. Phys. Eng. Express 2019, 5, 025016. [Google Scholar] [CrossRef]
  9. Zimlichman, E.; Szyper-Kravitz, M.; Shinar, Z.; Klap, T.; Levkovich, S.; Unterman, A.; Rozenblum, R.; Rothschild, J.M.; Amital, H.; Shoenfeld, Y. Early recognition of acutely deteriorating patients in non-intensive care units: Assessment of an innovative monitoring technology. J. Hosp. Med. 2012, 7, 628–633. [Google Scholar] [CrossRef] [Green Version]
  10. Zink, M.D.; Bruser, C.; Stuben, B.O.; Napp, A.; Stohr, R.; Leonhardt, S.; Marx, N.; Mischke, K.; Schulz, J.B.; Schiefer, J. Unobtrusive nocturnal heartbeat monitoring by a ballistocardiographic sensor in patients with sleep disordered breathing. Sci. Rep. 2017, 7, 13175. [Google Scholar] [CrossRef] [Green Version]
  11. Migliorini, M.; Kortelainen, J.M.; Pärkkä, J.; Tenhunen, M.; Himanen, S.L.; Bianchi, A.M. Monitoring nocturnal heart rate with bed sensor. Methods Inf. Med. 2014, 53, 308–313. [Google Scholar] [PubMed] [Green Version]
  12. Giovangrandi, L.; Inan, O.T.; Wiard, R.M.; Etemadi, M.; Kovacs, G.T.A. Ballistocardiography-A Method Worth Revisiting. In Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Boston, MA, USA, 30 August–4 September 2011; pp. 4279–4282. [Google Scholar]
  13. Gubner, R.; Rodstein, M.; Ungerleider, H. Ballistocardiography: An appraisal of technic, physiologic principles, and clinical value. Circulation 1953, 7, 268–286. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  14. Marino, M.; Li, Y.; Rueschman, M.N.; Winkelman, J.W.; Ellenbogen, J.M.; Solet, J.M.; Dulin, H.; Berkman, L.F.; Buxton, O.M. Measuring sleep: Accuracy, sensitivity, and specificity of wrist actigraphy compared to polysomnography. Sleep 2013, 36, 1747–1755. [Google Scholar] [CrossRef] [PubMed]
  15. Muzet, A.; Werner, S.; Fuchs, G.; Roth, T.; Saoud, J.B.; Viola, A.U.; Schaffhauser, J.Y.; Luthringer, R. Assessing sleep architecture and continuity measures through the analysis of heart rate and wrist movement recordings in healthy subjects: Comparison with results based on polysomnography. Sleep Med. 2016, 21, 47–56. [Google Scholar] [CrossRef] [Green Version]
  16. Waldeck, M.; Lambert, M. Heart rate during sleep: Implications for monitoring training status. J. Sports Sci. Med. 2003, 2, 133–138. [Google Scholar]
  17. De Zambotti, M.; Goldstone, A.; Claudatos, S.; Colrain, I.; Baker, F.C. A validation study of Fitbit Charge™ compared with polysomnography in adults. Chronobiol. Int. 2018, 35, 465–476. [Google Scholar] [CrossRef]
  18. De Zambotti, M.; Rosas, L.; Colrain, I.M.; Baker, F.C. The sleep of the ring: Comparison of the OURA sleep tracker against polysomnography. Behav. Sleep Med. 2019, 17, 124–136. [Google Scholar] [CrossRef]
  19. Lee, H.; Lee, Y. A look at wearable abandonment. In Proceedings of the 2017 18th IEEE International Conference on Mobile Data Management (MDM), Daejeon, Korea, 29 May–1 June 2017; pp. 392–393. [Google Scholar]
  20. Depner, C.M.; Cheng, P.C.; Devine, J.K.; Khosla, S.; de Zambotti, M.; Robillard, R.; Vakulin, A.; Drummond, S.P.A. Wearable technologies for developing sleep and circadian biomarkers: A summary of workshop discussions. Sleep 2020, 43, zsz254. [Google Scholar] [CrossRef] [Green Version]
  21. Moinester, M.A.; Gottfried, R. Sample size estimation for correlations with pre-specified confidence interval. Quant. Methods Psychol. 2014, 10, 124–130. [Google Scholar] [CrossRef]
  22. Greengard, S. The Internet of Things; MIT Press: Cambridge, MA, USA, 2015; p. 253. [Google Scholar]
  23. Boudreau, P.; Yeh, W.H.; Dumont, G.A.; Boivin, D.B. Circadian variation of heart rate variability across sleep stages. Sleep 2013, 36, 1919–1928. [Google Scholar] [CrossRef] [Green Version]
  24. De Zambotti, M.; Trinder, J.; Silvani, A.; Colrain, I.M.; Baker, F.C. Dynamic coupling between the central and autonomic nervous systems during sleep: A review. Neurosci. Biobehav. Rev. 2018, 90, 84–103. [Google Scholar] [CrossRef] [PubMed]
  25. Douglas, N.; White, D.; Pickett, C.; Weil, J.; Zwillich, C. Respiration during sleep in normal man. Thorax 1982, 37, 840–844. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  26. Gutierrez, G.; Williams, J.; Alrehaili, G.A.; McLean, A.; Pirouz, R.; Amdur, R.; Jain, V.; Ahari, J.; Bawa, A.; Kimbro, S. Respiratory rate variability in sleeping adults without obstructive sleep apnea. Physiol. Rep. 2016, 4, e12949. [Google Scholar] [CrossRef] [PubMed]
  27. Cen, L.; Yu, Z.; Tang, Y.; Shi, W.; FKluge, T.; Ser, W. Deep learning method for sleep stage classification. In Proceedings of the International Conference on Neural Information Processing, ICONIP 2017, Guangzhou, China, 14–18 November 2017; Liu, D., Xie, S.X., Li, Y., Zhao, D., El-Alfy, E., Eds.; Springer: Guangzhou, China, 2017. [Google Scholar]
  28. Li, Q.; Li, Q.; Liu, C.; Shashikumar, S.P.; Nemati, S.; Clifford, G.D. Deep learning in the cross-time frequency domain for sleep staging from a single-lead electrocardiogram. Physiol. Meas. 2018, 39, 124005. [Google Scholar] [CrossRef]
  29. Mousavi, S.; Afghah, F.; Acharya, U.R. SleepEEGNet: Automated sleep stage scoring with sequence to sequence deep learning approach. PLoS ONE 2019, 14, e0216456. [Google Scholar] [CrossRef] [Green Version]
  30. Phan, H.; Andreotti, F.; Cooray, N.; Chen, O.Y.; De Vos, M. Joint classification and prediction CNN framework for automatic sleep stage classification. IEEE Trans. Biomed. Eng. 2019, 66, 1285–1296. [Google Scholar] [CrossRef]
  31. Roberts, D.M.; Schade, M.M.; Mathew, G.M.; Gartenberg, D.; Buxton, O.M. Detecting sleep using heart rate and motion data from multisensor consumer-grade wearables, relative to wrist actigraphy and polysomnography. Sleep 2020, 43, zsaa045. [Google Scholar] [CrossRef]
  32. Sun, H.; Ganglberger, W.; Panneerselvam, E.; Leone, M.J.; Quadri, S.A.; Goparaju, B.; Tesh, R.A.; Akeju, O.; Thomas, R.J.; Westover, M.B. Sleep staging from electrocardiography and respiration with deep learning. Sleep 2020, 43, zsz306. [Google Scholar] [CrossRef] [Green Version]
  33. Yildirim, O.; Baloglu, U.B.; Acharya, U.R. A deep learning model for automated sleep stages classification using PSG signals. Int. J. Env. Res. Public Health 2019, 16, 599. [Google Scholar] [CrossRef] [Green Version]
  34. Zhang, Y.; Yang, Z.; Lan, K.; Liu, X.; Zhang, Z.; Li, P.; Cao, D.; Zheng, J.; Pan, J. Sleep Stage Classification Using Bidirectional LSTM in Wearable Multi-sensor Systems. In Proceedings of the IEEE INFOCOM 2019 Workshops, IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), Paris, France, 29 April–2 May 2019; pp. 443–448. [Google Scholar]
  35. Martinez, J.P.; Almeida, R.; Olmos, S.; Rocha, A.P.; Laguna, P. A wavelet-based ECG delineator: Evaluation on standard databases. IEEE Trans. Biomed. Eng. 2004, 51, 570–581. [Google Scholar] [CrossRef]
  36. Menghini, L.; Cellini, N.; Goldstone, A.; Baker, F.C.; de Zambotti, M. A standardized framework for testing the performance of sleep-tracking technology: Step-by-step guidelines and open-source code. Sleep 2021, 44, zsaa170. [Google Scholar] [CrossRef] [PubMed]
  37. Bland, J.M.; Altman, D. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986, 1, 307–310. [Google Scholar] [CrossRef]
  38. Haghayegh, S.; Kang, H.-A.; Khoshnevis, S.; Smolensky, M.; Diller, K. A comprehensive guideline for Bland-Altman and intra class correlation calculations to properly compare two methods of measurement and interpret findings. Physiol. Meas. 2020, 41, 055012. [Google Scholar] [CrossRef]
  39. Byrt, T.; Bishop, J.; Carlin, J. Bias, prevalence and kappa. J. Clin. Epidemiol. 1993, 46, 423–429. [Google Scholar] [CrossRef]
  40. Kortelainen, J.M.; Mendez, M.O.; Bianchi, A.M.; Matteucci, M.; Cerutti, S. Sleep staging based on signals acquired through bed sensor. IEEE Trans. Inf. Technol. Biomed. 2010, 14, 776–785. [Google Scholar] [CrossRef]
  41. Mendez, M.O.; Matteucci, M.; Cerutti, S.; Bianchi, A.M.; Kortelainen, J.M. Automatic detection of sleep macrostructure based on bed sensors. Annu. Int. Conf. IEEE Eng. Med. Biol. Soc. 2009, 2009, 5555–5558. [Google Scholar]
  42. Migliorini, M.; Bianchi, A.M.; Nistico, D.; Kortelainen, J.; Arce-Santana, E.; Cerutti, S.; Mendez, M.O. Automatic sleep staging based on ballistocardiographic signals recorded through bed sensors. Annu. Int. Conf. IEEE Eng. Med. Biol. Soc. 2010, 2010, 3273–3276. [Google Scholar]
  43. Hori, T.; Hayashi, M.; Morikawa, T. Topographical EEG changes and the hypnagogic experience. In Sleep Onset: Normal and Abnormal Processes; American Psychological Association: Washington, DC, USA, 1994; pp. 237–253. [Google Scholar]
  44. Kelly, J.M.; Strecker, R.E.; Bianchi, M.T. Recent developments in home sleep-monitoring devices. ISRN Neurol. 2012, 2012, 768794. [Google Scholar] [CrossRef] [Green Version]
  45. Meltzer, L.J.; Hiruma, L.S.; Avis, K.; Montgomery-Downs, H.; Valentin, J. Comparison of a commercial accelerometer with polysomnography and actigraphy in children and adolescents. Sleep 2015, 38, 1323–1330. [Google Scholar] [CrossRef] [Green Version]
Sensors 22 02605 g001 550
Figure 1. The smart bed setup, ballistocardiograph, and subsequent output. A BCG recording using the smart bed technology (black, top row of middle panel) during a motionless, 16 s recording segment decomposed into the respiratory pattern (blue, second row of middle panel) and heart activity (red, bottom row of middle panel). The dots denote detected individual heartbeats. Vertical axes are pressure readings in pascal units scaled for better visualization. BCG, ballistocardiography; SE, sleep efficiency; SOL, sleep onset latency; TST, total sleep time; WASO, wake after sleep onset.
Figure 1. The smart bed setup, ballistocardiograph, and subsequent output. A BCG recording using the smart bed technology (black, top row of middle panel) during a motionless, 16 s recording segment decomposed into the respiratory pattern (blue, second row of middle panel) and heart activity (red, bottom row of middle panel). The dots denote detected individual heartbeats. Vertical axes are pressure readings in pascal units scaled for better visualization. BCG, ballistocardiography; SE, sleep efficiency; SOL, sleep onset latency; TST, total sleep time; WASO, wake after sleep onset.
Sensors 22 02605 g001
Sensors 22 02605 g002 550
Figure 2. Example hypnogram from PSG and concurrent physiological data from PSG and the smart bed. Hypnogram data from a 36-year-old woman with BMI = 21.6 kg/m2 (A) and simultaneous profiles of PSG HR (B), smart bed HR (C), PSG BR (D), smart bed BR (E), and smart bed movement (F). Variability in overnight HR and BR was calculated at each time point as the standard deviation of values recorded over a running 15 min window (shaded grey bands). A 15 min smoothing window was applied to the HR and BR profiles to remove sporadic spikes and highlight the overall trends. BMI, body mass index; bpm, beats/breaths per minute; BR, breathing rate; HR, heart rate; PSG, polysomnography.
Figure 2. Example hypnogram from PSG and concurrent physiological data from PSG and the smart bed. Hypnogram data from a 36-year-old woman with BMI = 21.6 kg/m2 (A) and simultaneous profiles of PSG HR (B), smart bed HR (C), PSG BR (D), smart bed BR (E), and smart bed movement (F). Variability in overnight HR and BR was calculated at each time point as the standard deviation of values recorded over a running 15 min window (shaded grey bands). A 15 min smoothing window was applied to the HR and BR profiles to remove sporadic spikes and highlight the overall trends. BMI, body mass index; bpm, beats/breaths per minute; BR, breathing rate; HR, heart rate; PSG, polysomnography.
Sensors 22 02605 g002
Sensors 22 02605 g003 550
Figure 3. Bland–Altman comparison of all-night and epoch-by-epoch mean BR and HR measurements by smart bed versus PSG. Panels (A,B): Bland–Altman analysis of mean all-night measurements of BR and HR. The difference in biosignal measurement was plotted as a function of the reference PSG measurement. These analyses indicated a strong agreement of measurements (insignificant bias and narrow LOA) between the smart bed and PSG for both BR and HR. Panels (C,D): Bland–Altman comparison of epoch-by-epoch BR and HR from the smart bed and PSG. Epoch-by-epoch graphs plot the difference between simultaneously measured values for 30 s epochs from all 45 participants where there were measurements from both systems as a function of the reference (PSG) measurement. In this epoch-by-epoch analysis (minimum of 686 epochs to maximum of 1036 epochs per subject), calculations of bias and LOA were performed as previously described [38]. Epoch-by-epoch analysis indicated a strong agreement on instantaneous biosignals captured by smart bed and PSG. Solid and dotted red lines represent the bias and the upper and lower LOA; shaded areas represent the respective 95% confidence intervals. BR, breathing rate; HR, heart rate; LOA, level of agreement; PSG, polysomnography.
Figure 3. Bland–Altman comparison of all-night and epoch-by-epoch mean BR and HR measurements by smart bed versus PSG. Panels (A,B): Bland–Altman analysis of mean all-night measurements of BR and HR. The difference in biosignal measurement was plotted as a function of the reference PSG measurement. These analyses indicated a strong agreement of measurements (insignificant bias and narrow LOA) between the smart bed and PSG for both BR and HR. Panels (C,D): Bland–Altman comparison of epoch-by-epoch BR and HR from the smart bed and PSG. Epoch-by-epoch graphs plot the difference between simultaneously measured values for 30 s epochs from all 45 participants where there were measurements from both systems as a function of the reference (PSG) measurement. In this epoch-by-epoch analysis (minimum of 686 epochs to maximum of 1036 epochs per subject), calculations of bias and LOA were performed as previously described [38]. Epoch-by-epoch analysis indicated a strong agreement on instantaneous biosignals captured by smart bed and PSG. Solid and dotted red lines represent the bias and the upper and lower LOA; shaded areas represent the respective 95% confidence intervals. BR, breathing rate; HR, heart rate; LOA, level of agreement; PSG, polysomnography.
Sensors 22 02605 g003
Sensors 22 02605 g004 550
Figure 4. ROC for smart bed prediction of sleep and wake states relative to PSG. The ROC and AUC were assessed as accuracy measures invariant of the prevalence of sleep epochs. The dashed line shows the test with no discriminatory ability (AUC = 0.5). AUC, area under the curve; PSG, polysomnography; ROC, receiver operating characteristic curve.
Figure 4. ROC for smart bed prediction of sleep and wake states relative to PSG. The ROC and AUC were assessed as accuracy measures invariant of the prevalence of sleep epochs. The dashed line shows the test with no discriminatory ability (AUC = 0.5). AUC, area under the curve; PSG, polysomnography; ROC, receiver operating characteristic curve.
Sensors 22 02605 g004
Sensors 22 02605 g005 550
Figure 5. Bland–Altman comparisons of smart bed- versus PSG-derived WASO (A); TST (B); SOL (C); and SE (D). The density diagram on the right side of each plot represents the distribution of the differences for each variable. The solid lines are the bias and LOA. The dotted lines represent the corresponding confidence intervals for bias and LOA. LOA, level of agreement; PSG, polysomnography; SE, sleep efficiency; SOL, sleep onset latency; TST, total sleep time; WASO, wake after sleep onset.
Figure 5. Bland–Altman comparisons of smart bed- versus PSG-derived WASO (A); TST (B); SOL (C); and SE (D). The density diagram on the right side of each plot represents the distribution of the differences for each variable. The solid lines are the bias and LOA. The dotted lines represent the corresponding confidence intervals for bias and LOA. LOA, level of agreement; PSG, polysomnography; SE, sleep efficiency; SOL, sleep onset latency; TST, total sleep time; WASO, wake after sleep onset.
Sensors 22 02605 g005
Table
Table 1. Demographics of the study cohort and PSG-derived summary measures.
Table 1. Demographics of the study cohort and PSG-derived summary measures.
VariableAll Participants (N = 45)
Male sex, n (%)20 (44.4)
Mean age, years (SD)41.2 (10.5)
Mean BMI, kg/m2 (SD)25.9 (4.4)
Mean lights out, hh:mm (SD)22:16 (00:29)
Mean lights on, hh:mm (SD)05:55 (00:32)
Mean TST, min (SD)388.7 (46.3)
Mean WASO, min (SD)54.2 (33.9)
Mean SE, % (SD)87 (7)
Mean SOL, min (SD)16.4 (13.6)
Mean BR, bpm * (SD)14.9 (2.2)
Mean HR, bpm * (SD)66.4 (9.9)
Mean AHI, events/h (SD)6.53 (15.1)
AHI, apnea-hypopnea index; BMI, body mass index; bpm, breaths per minute or beats per minute; BR, breathing rate; HR, heart rate; SD, standard deviation; SE, sleep efficiency; SOL, sleep onset latency; TST, total sleep time; WASO, wake after sleep onset. * bpm refers to breaths per minute for BR and beats per minute for HR.
Table
Table 2. Performance of the smart bed’s machine learning algorithm for sleep/wake detection.
Table 2. Performance of the smart bed’s machine learning algorithm for sleep/wake detection.
Performance MeasureSmart Bed Machine Learning Algorithm
AUC0.86
Accuracy (mean ± SD)0.86 ± 0.11
Balanced accuracy (mean ± SD)0.75 ± 0.12
d′ (mean ± SD)1.47 ± 0.38
Kappa0.45 ± 0.17
Adjusted kappa (mean ± SD)0.74 ± 0.11
Precision (mean ± SD)0.90 ± 0.06
Sensitivity (mean ± SD)0.94 ± 0.05
Specificity (mean ± SD)0.48 ± 0.18
AUC, area under the receiver operating curve; SD, standard deviation.
Table
Table 3. Comparison of smart bed- and PSG-derived overnight sleep variables.
Table 3. Comparison of smart bed- and PSG-derived overnight sleep variables.
SOL (min)WASO (min)TST (min)SE (%)
Smart bed (SD)13.4 (11.4)76.8 (43.8)369.1 (52.7)0.8 (0.1)
PSG (SD)16.4 (13.7)54.2 (33.9)388.7 (46.4)0.9 (0.1)
Bias3.0 (16.8)15.3 − 0.5 × ref150.6 − 0.4 × ref0.5 − 0.5 × ref
Bias CI[−1.9, 7.7]b0 = [1.1, 27.6],
b1 = [−0.7, −0.3]		b0 = [63.0, 213.9],
b1 = [−0.5, −0.1]		b0 = [0.3, 0.6],
b1 = [−0.6, −0.3]
Lower LOAbias − 2.5 (−3.4 + 0.9 × ref)bias − 2.5 (16.3 + 0.1 × ref)bias − 61.8bias − 0.1
Upper LOAbias + 2.5 (−3.4 + 0.9 × ref)bias + 2.5 (16.3 + 0.1 × ref)bias + 61.8bias + 0.1
LOA CIc0 = [−22.9, −10.8],
c1 = [1.4, 2.2]		c0 = [4.2, 19.4],
c1 = [0.0, 0.2]		bias ± [52.8, 74.1]bias ± [0.1, 0.1]
Sleep metrics were tested for the proportional bias and heteroscedasticity with respect to the reference (PSG) measurement. In the case of proportional bias, the bias coefficients were provided as bias = b0 + b1 × (ref). In the case of heteroscedasticity, the LOA was provided as coefficients of the linear regression model under normality assumption as 95% LOA = bias ± 2.46 (c0 + c1 × (ref)). CI, confidence interval; LOA, level of agreement; PSG, polysomnography; ref, reference measurement; SD, standard deviation; SE, sleep efficiency; SOL, sleep onset latency; TST, total sleep time; WASO; wake after sleep onset.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Siyahjani, F.; Garcia Molina, G.; Barr, S.; Mushtaq, F. Performance Evaluation of a Smart Bed Technology against Polysomnography. Sensors 2022, 22, 2605. https://doi.org/10.3390/s22072605

AMA Style

Siyahjani F, Garcia Molina G, Barr S, Mushtaq F. Performance Evaluation of a Smart Bed Technology against Polysomnography. Sensors. 2022; 22(7):2605. https://doi.org/10.3390/s22072605

Chicago/Turabian Style

Siyahjani, Farzad, Gary Garcia Molina, Shawn Barr, and Faisal Mushtaq. 2022. "Performance Evaluation of a Smart Bed Technology against Polysomnography" Sensors 22, no. 7: 2605. https://doi.org/10.3390/s22072605

APA Style

Siyahjani, F., Garcia Molina, G., Barr, S., & Mushtaq, F. (2022). Performance Evaluation of a Smart Bed Technology against Polysomnography. Sensors, 22(7), 2605. https://doi.org/10.3390/s22072605

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop