Next Article in Journal
Digitalization of an Industrial Process for Bearing Production
Previous Article in Journal
SiCRNN: A Siamese Approach for Sleep Apnea Identification via Tracheal Microphone Signals
Previous Article in Special Issue
Near-Infrared Spectroscopy for Neonatal Sleep Classification
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Real-Time Postural Disturbance Detection Through Sensor Fusion of EEG and Motion Data Using Machine Learning

by
Zhuo Wang
1,2,*,†,
Avia Noah
3,4,
Valentina Graci
2,5,
Emily A. Keshner
3,
Madeline Griffith
2,
Thomas Seacrist
2,
John Burns III
2,
Ohad Gal
3 and
Allon Guez
1,3
1
Department of Electrical and Computer Engineering, Drexel University, Philadelphia, PA 19104, USA
2
Center for Injury Research and Prevention, Children’s Hospital of Philadelphia, Philadelphia, PA 19104, USA
3
GraceFall, Inc., Penn Valley, PA 19702, USA
4
Faculty of Engineering, Ruppin Academic Center, Emek-Hefer, Monash 40250, Israel
5
School of Biomedical Engineering, Science, and Health System, Drexel University, Philadelphia, PA 19104, USA
*
Author to whom correspondence should be addressed.
Current address: Department of Electrical and Computer Engineering, Kennesaw State University, Marietta, GA 30060, USA.
Sensors 2024, 24(23), 7779; https://doi.org/10.3390/s24237779
Submission received: 4 November 2024 / Revised: 29 November 2024 / Accepted: 2 December 2024 / Published: 5 December 2024
(This article belongs to the Special Issue Sensors and Sensing Technologies for Neuroscience)

Abstract

:
Millions of people around the globe are impacted by falls annually, making it a significant public health concern. Falls are particularly challenging to detect in real time, as they often occur suddenly and with little warning, highlighting the need for innovative detection methods. This study aimed to assist in the advancement of an accurate and efficient fall detection system using electroencephalogram (EEG) data to recognize the reaction to a postural disturbance. We employed a state-space-based system identification approach to extract features from EEG signals indicative of reactions to postural perturbations and compared its performance with those of traditional autoregressive (AR) and Shannon entropy (SE) methods. Using EEG epochs starting from 80 ms after the onset of the event yielded improved performance compared with epochs that started from the onset. The classifier trained on the EEG data achieved promising results, with a sensitivity of up to 90.9%, a specificity of up to 97.3%, and an accuracy of up to 95.2%. Additionally, a real-time algorithm was developed to integrate the EEG and accelerometer data, which enabled accurate fall detection in under 400 ms and achieved an over 99% accuracy in detecting unexpected falls. This research highlights the potential of using EEG data in conjunction with other sensors for developing more accurate and efficient fall detection systems, which can improve the safety and quality of life for elderly adults and other vulnerable individuals.

1. Background and Aim

Falls are a major public health problem worldwide. Falls can result in injuries such as fractures, head injuries, and lacerations, which can lead to hospitalization, disability, and even death [1]. According to the World Health Organization [2], an estimated 646,000 falls result in death globally each year, making it the second-leading cause of unintentional injury death after road traffic accidents. Falls also result in non-fatal injuries, with an estimated 37.3 million severe falls requiring medical attention each year [2]. The financial burden of falls is also significant. In the United States alone, falls result in approximately USD 50 billion in direct medical costs annually [3], and this figure is expected to rise to over USD 101 billion with an aging population by the year 2030 [4]. Falls also have indirect costs, including lost productivity, reduced quality of life, and caregiver burden [5].
Elderly adults have an increased likelihood of falls due to age-related changes in physical and cognitive function [6]. In the United States, it is estimated that approximately 27–30% of people aged 65 and elderly fell annually over the past ten years [7]. In addition to the elderly population, other groups that are prone to falling include adults with neurological disorders, such as Parkinson’s disease, Alzheimer’s disease, and epilepsy seizures, and individuals with visual impairments and athletes. With up to 70% of people with Parkinson’s experiencing at least one fall per year, the leading causes of falls in Parkinson’s disease include gait and balance disturbances, freezing of gait, and medication side effects [8]. People experiencing an epilepsy seizure are more likely to experience a loss of awareness and muscle control, which may cause a collapse to the ground [9]. Similarly, studies reported that individuals with visual impairments are three to four times more likely to experience falls than those without visual impairments. Factors that contribute to falls in this population include decreased visual acuity, contrast sensitivity, and depth perception [10]. Finally, otherwise healthy athletes in sports such as gymnastics, ice skating, and snowboarding are at risk of falls and associated injuries. In these sports, falls can occur due to factors such as muscle imbalances, core stability deficits, and poor neuromuscular control [11].
Fortunately, with the advancement of technologies, the number of proposed fall detection systems and devices has increased dramatically. Wu et al. [12] developed a fall detection system that collected acceleration and rotation angle information from a subjects’ waist and identified a threshold for the sum of acceleration that would detect falls. The system proposed in [13] utilized the accelerometer of a smartphone and incorporated GPS-based real-time location tracking for prompt emergency intervention. These innovations and similar devices rely on inertial sensors to trigger fall detection, allowing for the identification of key time points in the motion of a fall, from onset to completion. Despite these advancements, distinguishing between an actual loss of stability and non-fall activities remains challenging, leading to a high false alarm rate.
Attempts have been made to increase a fall detection device’s reliability through sensor fusion systems with cameras, radar, and acoustic systems. For instance, ref. [14] utilized artificial vision methods to monitor the presence of individuals in a room and detect falls by analyzing changes in their posture or lack of movement. Radar technologies were also used in studies [15] to detect falls by monitoring a change in motion frequencies. The authors of [16] proposed an acoustic fall detection system that utilized a circular microphone array to capture and analyze sounds for automatic fall detection. However, these methods are constrained by the active space within the location where the systems are installed. Moreover, the challenge of distinguishing real falls from activities of daily living (ADLs) persists [17], and this poses a challenge in finding a practical threshold between the missed fall detection rate and high false alarm rate, rendering these devices inefficient.
Recent research [18,19] suggested that wearable EEG devices may provide a solution to the challenge of accurately detecting real falls. An electroencephalogram, or EEG, is a noninvasive method of measuring brain activity that is commonly used in applications such as epilepsy seizure and sleep monitoring [20]. One type of cortical activation measured by EEG is the event-related potential (ERP), which can be elicited with balance perturbations [21]. This brain activity, known as perturbation-evoked potentials (PEPs), can be identified through a single electrode site [22], and are widely distributed over the frontal, central, and parietal cortexes. Additionally, correlation analysis demonstrated a good match between mastoid and cortical EEGs through well-known EEG paradigms [23]. Furthermore, advancements such as the development of dry-ear EEG sensors with flexible earpieces [24] demonstrate the potential for more convenient and cost-effective brain–computer interface applications. For practical application in real-world scenarios, integrating EEG signals with acceleration data from motion sensors enables earlier and more accurate fall prediction compared with existing solutions, thereby helping to prevent injuries.
Based on these findings, we previously conducted an experiment to differentiate PEPs into predictable and unpredictable postural disturbances using EEG recordings in young and elderly adults [25]. The Root-Mean-Square (RMS) values of EEG epochs from cortical and mastoid channels were analyzed using the three-way Repeated Measures Analysis of Variance (ANOVA) to evaluate the effects of the perturbation type. Our results show that the predictability of a postural disturbance modified the EEG activity related to balance adjustments and that this EEG alteration could be identified using a single electrode site placed over the mastoid. It is noteworthy that the labeling method used in our previous study was based on the index and direction of the perturbations rather than the EEG activity.
In the current study, we expanded upon our previous research [25] by implementing a supervised manual data labeling method that categorized the disturbances based on the EEG responses rather than the perturbation index or direction. This labeling method provided a cortical-activity-focused perspective on balance adjustments, potentially enhancing the detection of events that may have been overlooked by the previous labeling method.
A state-space-based system identification method was applied to evaluate the impact of varying EEG epoch lengths based on the classification performance. This approach was proved effective in prior studies, including sleep stage classification [26] and seizure detection [27]. Its efficacy was further demonstrated through comparisons with widely used approaches, such as autoregressive (AR) modeling, Shannon entropy (SE), and support vector machine (SVM) classifiers [28,29,30,31]. In addition, a real-time algorithm integrating EEG and accelerometer data underscores the system’s potential for rapid and accurate fall events classification, offering practical advantages for safety applications.
The objective of this study was to develop and evaluate a novel approach for real-time fall detection through the supervised classification of PEPs from ongoing mastoid EEG recordings. Our proposed method incorporates advanced feature extraction and machine learning techniques, as well as a comprehensive evaluation of various epoch lengths. Additionally, a real-time detection algorithm is introduced, enabling the rapid and accurate classification of fall events while allowing sufficient time for communication and protective measures, such as airbag deployment. By improving the accuracy and reliability of fall detection systems, particularly in distinguishing between actual falls and activities of daily living (ADLs), this work ultimately aimed to support the development of fall detection technologies that are effective and accessible to the elderly and other vulnerable populations, thereby enhancing their safety and overall quality of life.

2. Method

The protocol of this study was reviewed and approved by the Institutional Review Board of the Children’s Hospital of Philadelphia, and the participants were compensated for their participation.

2.1. Participants

Recruitment was conducted through ResearchMatch.org, a platform that connects individuals with research studies based on their health profiles. This study included 40 participants, comprising 20 young adults (mean age: 24.6 ± 5.9 years; height: 172.2 ± 6.9 cm; weight: 69.6 ± 10.6 kg) and 20 older adults (mean age: 66.9 ± 5.8 years; height: 169.4 ± 8.4 cm; weight: 68.8 ± 10.3 kg). Each group was evenly divided by gender, with 10 males and 10 females.
The exclusion criteria included a weight exceeding 195 lbs; cognitive impairments; use of walking aids; a history of spine, pelvis, or lower extremity fractures within the past five years; pregnancy; uncorrected vision or hearing issues; significant foot deformities or amputations; hip or knee replacements; peripheral neuropathy; medications affecting postural control; and a fear of falling or amusement park rides.

2.2. Procedures

All subjects wore a safety harness and were asked to maintain an upright stance on a dynamic platform measuring 60 cm by 90 cm, with their feet placed shoulder-width apart and eyes open (Figure 1A). The experimental protocol called for each subject to experience 34 balance perturbations, consisting of 29 posterior and 5 anterior translations of the platform of 20 cm displacement at a rate of 100 cm/s and acceleration of 0.2 g, for a total of 1360 translations. However, due to technical or subject-related issues, 40 translations across the group were not executed, yielding a total of 1320 completed translations.
During the experiment, active electrodes over an EasyCap (Brain Vision, Morrisville, NC, USA) were located according to the international 10–20 system at positions Fz, Cz, F7, F8, C3, C4, P7, P8, T7, and T8 (Figure 1B). Additionally, 3 channels from cEEGrids (TMSI, Oldenzaal, Netherlands) were patched over each mastoid at L5, L6, L7, R5, R6, and R7 (Figure 1C–E). Reference electrodes were placed at FCz and AFz. Raw EEG data were sampled at 1000 Hz and filtered offline with a 2nd-order Butterworth bandpass filter (2.5 Hz–30 Hz) and a 60 Hz notch filter. The platform acceleration was sampled at 150 Hz (Delsys, Inc., Natick, MA, USA). Because the platform’s movement was restricted to the antero-posterior direction, only 1D acceleration data along this axis were included in the dataset. Synchronization of the EEG and acceleration data, recorded by different devices, was achieved using transistor–transistor logic (TTL). Both the EEG and acceleration data were smoothed using the Savitzky–Golay filter (Matlab R2023b).

2.3. Data Analysis

To ensure uniformity across the subjects, detrending and baseline correction were applied to all event epochs. This was achieved by calculating the average and standard deviation of the EEG magnitude from 300 ms to 100 ms prior to the event onset and subtracting any linear trend or DC shifts. Subsequently, all EEG epochs within a channel were normalized by scaling them proportionally to the maximum EEG magnitude of the subject’s first event epoch, which was used as a reference point.
For this study, we focused on a single mastoid electrode site at R6 located at the center of the right mastoid bone. The EEG responses from each event were manually identified and categorized as either “Predicted” or “Unpredicted”. In total, 885 event EEGs were labeled as “Predicted”, while the remaining 435 were labeled as “Unpredicted”. Figure 2 illustrates the EEG data collected during two unpredictable events compared with ten predictable disturbances.
The onset of each event was defined as the initial acceleration of the dynamic platform and was determined using the platform accelerometer (Figure 3). We conducted experiments with different EEG epoch lengths, including 350 ms, 500 ms, 750 ms, and 1000 ms after the onset of the event, and evaluated the impact of selecting EEG epochs from 80 ms after the onset to the same cut-offs. EEG epochs were transferred into the state-space feature with 5th-order dynamic system state space matrices. The feature was then trained with the ensemble bagged trees classifier and 10-time 10-fold cross-validation.

2.4. State-Space Modeling

State-space-based system identification is a method that allows us to model and estimate the behavior of dynamic systems using measurement of the system’s input and output signals. It is important to note that unlike other systems, an EEG does not have any direct input signal; the EEG response reflects the cortical activity resulting from internal or external stimuli. Thus, we used an output-only system identification approach. This approach involves defining the state equation as
x T + 1 = A x T + K e T
where x T is the vector of internal states of the system at time T , A is the state matrix that defines the dynamic behavior of the system, K is the steady-state Kalman gain, and e T is the zero-mean white noise. K e T together represent the process noise that affects the system’s internal states. The output equation is defined by
y T = C x T + e T
where y T is the output EEG signal at time T and C is the output matrix that maps the internal states of the system to its output signals. The block diagram of (1) and (2) combined as a dynamic state-space model is shown in Figure 4. Finally, the estimation of the state matrices A , C , and K of the state-space dynamic system can be achieved using the n4sid method, which is described in detail in [32].
A = a 11 a 1 m a m 1 a m m , C = c 1 c m , K = k 1 k m T
After estimating the state matrices A , C , and K , a feature vector can be created by concatenating each element of the matrices. In this case, a 5th-order state-space model was used, which resulted in a feature vector of 35 elements (1 × 35).
f e a t u r e T = a 11 a m m , c 1 c m , k 1 k m

2.5. Comparative Analysis

As previously mentioned, in our comparative analysis, we assessed the performance of our state-space method against a widely used EEG classification approach that incorporated AR coefficients and Shannon entropy features. Specifically, we utilized a 4th-order AR model combined with level 4 Shannon entropy values to construct a 1 × 20 feature vector that comprised 4 AR coefficients and 16 Shannon entropy values.

2.6. Real-Time Analysis

The real-time classifier was trained on data from 70% of the subjects (28 individuals), using 34 fall events labeled as “Expected” and “Unexpected”, along with 34 no-fall events labeled as “Expected”. For each event, EEG data (Ch. R6) and acceleration data were processed using a window size of 256 ms, with an 80 ms delay from the “onset” value. The selection of a short time window was driven by the necessity to predict falls within 400 ms, providing adequate time for communication, computation, and airbag deployment. The signals were filtered and normalized as previously described, and state-space features were extracted using a system order of 3 to facilitate rapid computation. The normalized EEG and acceleration threshold was defined as the minimum value among all maximum values of the “Unexpected” training data.
The algorithmic workflow for real-time fall detection is illustrated in Figure 5. A 256 ms window was initially selected at the onset of the EEG and acceleration signal recordings for each validation subject. During each iteration, the signals underwent normalization, followed by threshold evaluation. In cases where the signal fell below the defined threshold, the window advanced by 30 ms. Conversely, if the signal exceeded the threshold, state-space features were extracted and input into the classifier. When the classifier predicted an “Expected” outcome, the window advanced with a step size of 30 ms. However, if the result was classified as “Unexpected”, the window advanced by 1 s to mitigate the possibility of counting the same fall more than once. We note that the computational time for each iteration was estimated to be less than the step size (30 ms). A true positive or false positive classification was assigned for every event that exceeded the threshold based on the labeled data. In instances where a labeled event was not detected by the real-time algorithm, a false negative was recorded.

2.7. Performance Metrics

Sensitivity, specificity, and accuracy were the variables used as the indicators of performance. These measures are typically calculated from the number of true positive (TP), true negative (TN), false negative (FN), and false positive (FP) events predicted by the model, as determined by a “Standard of Truth” [33]. The sensitivity gauges the model’s ability to correctly identify instances of the positive class, while specificity measures its proficiency in recognizing the negative class. Accuracy provides an overall assessment of the model’s correctness by considering both true positives and true negatives in relation to all predicted outcomes. In this case, the “Unpredicted” class was considered the positive class and the “Predicted” class was considered the negative class.
S e n s i t i v i t y = T P T P + F N
S p e c i f i c i t y = T N T N + F P
A c c u r a c y = T P + T N T P + T N + F P + F N

3. Results

Table 1 presents the classification performance of the AR model combined with the Shannon entropy features for various epoch lengths starting at the onset of the event (0 ms cut-off). The results indicate that as the epoch length increased, there was a notable improvement in the sensitivity, specificity, and accuracy. Specifically, the sensitivity increased from 60.3% for the 0–350 ms epoch to 74.3% for the 0–1000 ms epoch. Similarly, the specificity remained high, where it ranged from 90.1% to 94.3%, while the accuracy rose from 81.7% to 88.6%. Table 2 highlights the classification performance of the same AR and Shannon entropy features, but with epochs that started 80 ms after the onset of the event. In this scenario, the sensitivity showed an upward trend, beginning at 66.7% for the 80–350 ms epoch and reaching 76.8% for the 80–1000 ms epoch. The specificity remained high, ranging from 93.6% to 94.3%, while the accuracy improved from 86.0% to 89.3%.
Table 3 summarizes the performance of the system using the state-space method for different EEG epoch lengths starting at the onset of the event. The results indicate that as the epoch length increased, the overall accuracy of the state-space method also improved. Specifically, the accuracy improved from 93.1% for the 350 ms epoch to 94.5% for the 1000 ms epoch. The specificity of the system, which measured the proportion of true negatives correctly identified, remained consistently high across all epoch lengths, ranging from 96.1% to 96.9%. On the other hand, the sensitivity of the system, which measured the proportion of correctly identified true positives, increased as the epoch length increased. The sensitivity ranged from 87.1% for the 350 ms epoch to 90.9% for the 1000 ms epoch. Table 4 demonstrates that when the EEG epochs began at 80 ms after the onset of the event, the overall performance of the classification system improved across all epoch lengths. For example, the accuracy for the 80–1000 ms epoch length increased from 94.5% to 95.2%, while the sensitivity for the same epoch length rose from 90.9% to 91.9%. The specificity also showed improvement, with the highest value of 97.3% achieved for the 80–1000 ms epoch length.
The analysis thus far was conducted offline without leveraging the potential of the motion information. Recognizing that both signals contained valuable information about falls, we considered a comprehensive analysis that integrated both datasets. This section demonstrates that unexpected falls could be identified with an accuracy exceeding 99% by integrating the EEG and motion sensor data. The algorithm successfully detected 100% of the unexpected falls in over 85% of the subjects, with low rates of false positives and false negatives. Notably, the majority of errors were concentrated within a small subset of subjects. Table 5 summarizes the results across six training sets, demonstrating an average sensitivity of 93 ± 3%, specificity of 99.6 ± 0.1%, and accuracy of 99.5 ± 0.3%.

4. Discussion

In this study, we utilized a state-space based system identification approach to extract features from EEG signals for classifying postural perturbation events. We evaluated the impact of different epoch lengths on the classification performance, comparing the results from the event onset and 80 ms after the onset. The results indicate that the EEG signals following the postural perturbations were accurately classified, where longer epoch lengths generally improved the classification performance, though this improvement plateaued. Notably, using EEG epochs that started at 80 ms after the event onset yielded better performances than those that started at the onset.
The imbalance between “Predicted” and “Unpredicted” events in our dataset was an intentional aspect of the study design. In real-world scenarios, “Predicted” events dominate daily activities, but accuracy alone does not fully capture the detector performance. For example, a 97% accuracy might reflect high specificity but low sensitivity if the “Predicted” class makes up the majority of the data. To address this, we would increase the proportion of “Unpredicted” cases during training, aiming for a ratio of 1:2 or 1:3 between “Predicted” and “Unpredicted” classes. This will ensure that the classifier learns patterns from both classes, rather than focusing only on the dominant class, while still respecting the real-world prevalence. This approach emphasizes the importance of reliably detecting the rare yet critical “Unpredicted” events.
The superior classification performance of the state-space method stems from its ability to capture the dynamic nature of EEG signals by modeling temporal relationships within the data. In contrast, the AR and Shannon entropy methods primarily rely on static features, potentially overlooking complex temporal dynamics. The state-space approach generates a rich feature vector from state matrices that encapsulate both the dynamic behavior and statistical properties of EEG signals, enhancing its ability to discern patterns related to fall events. For example, for the 1000 ms epoch length, the state-space method achieved a sensitivity of 90.9% and an accuracy of 94.5%, while the AR and Shannon entropy methods reached only 74.3% sensitivity and 88.6% accuracy. This significant difference underscored the effectiveness of the state-space method in improving the classification performance for EEG-based fall detection.
Longer epoch lengths allow for capturing more temporal dynamics related to movement intention, providing additional data for training and testing the classifier. However, they also require longer wait times to collect data, which can delay fall detection and increase the computational time. In fall detection, speed is crucial, as delays can result in insufficient time for the protection system to deploy and prevent injury. Our results indicate that a 256 ms epoch can be sufficient when leveraging EEG and motion sensor fusion. Initiating EEG epochs at 80 ms following the onset of the event appears to further enhance the classifier’s performance. It is well documented that automatic postural reactions in the lower limb are elicited at approximately an 80–90 ms latency with a second peak between 90 and 120 ms [34,35]. Cortical EEG activity arising after 80 ms likely reflects the cortical component of the longer latency response [36], suggesting that EEG activity between 0 and 80 ms may not contain critical information for fall detection. Additionally, EEG signals in the 0–80 ms segment may be more vulnerable to noise or artifacts, potentially impacting classifier performance. Therefore, considering computational efficiency and detection speed, initiating epochs at 80 ms is preferable.
Despite these promising results, limitations existed in this study. The relatively small dataset may restrict the generalizability of our findings. A more significant and diverse dataset could help validate the results and improve the system performance across different populations, which could be achieved via collaborating with hospitals or research centers. A larger dataset would aid in refining the classifier parameters, adjusting the signal thresholds, and optimizing the training parameter set. This could improve the accuracy and reduce false events, which are the current barriers to the fall protection system. Furthermore, this study focused on one type of postural perturbation, which may not encompass the full range of real-life falls or daily activities. Future studies should evaluate the system’s effectiveness across various fall types and activities. Future reports will also include a more extensive analysis with results from additional EEG channels due to the fact our study only focused on the analysis of a single EEG channel.
Our current study focused on the fusion of EEG and accelerometer data to demonstrate the feasibility and potential of this approach for fall detection. In future work, we aim to expand upon this by integrating additional sensors, such as gyroscopes, pressure sensors, and visual sensors, to further enhance the accuracy and reliability of the system. The inclusion of multimodal data will provide a more comprehensive understanding of fall dynamics, contributing to the development of robust fall detection technologies that are effective, accessible, and capable of improving the safety and quality of life for the elderly and other vulnerable populations.
Refining the accuracy of our detection system requires exploring suitable features and algorithms for each data type and their synergistic integration. Advanced neural network architectures, such as Graph Neural Networks (GNNs), offer promising avenues for modeling complex relationships in neural data, as highlighted in [37]. Similarly, EEG-based analyses were applied to detect physiological states, including driver fatigue, using spatiotemporal fusion networks with brain-region-partitioning strategies [38]. Adopting such approaches could further enhance the classification performance in detecting perturbations.
Furthermore, while our proposed method achieved high accuracy in identifying unexpected falls, optimizing the algorithm to enhance real-time performance and reduce computational complexity is a crucial next step. These advancements are essential to translate our research findings into a practical fall detection system, which can be effectively deployed to enhance safety in elderly adults and vulnerable populations across various everyday environments.

5. Conclusions

Our study demonstrated the potential of utilizing EEG and motion data to detect postural perturbations and predict falls. The findings indicate that postural perturbations significantly modify EEG activity related to balance adjustments, which can be leveraged to differentiate between falls and non-falls. Notably, our state-space method outperformed the traditional autoregressive (AR) and Shannon entropy (SE) methods, where it achieved a sensitivity of 90.9%, specificity of 97.3%, and accuracy of 95.2%. By integrating the EEG and motion sensor data, we demonstrated that unexpected falls could be detected in real time with over 99% accuracy. Future research should focus on utilizing larger and more diverse datasets, investigating the system’s performance across various types of falls and daily living activities, and expanding the analysis to include EEG data from multiple channels while integrating other sensors, such as accelerometers. Developing real-time processing and classification methods will further enhance the effectiveness of fall detection systems. Overall, this study underscored the promising role of EEG data, particularly when combined with other sensor modalities, in creating more accurate and efficient fall detection systems that can improve the safety and quality of life for elderly adults and individuals with mobility impairments.

Author Contributions

Conceptualization, E.A.K. and A.G.; methodology, V.G. and E.A.K.; software, Z.W. and A.N.; validation, Z.W., A.N. and V.G.; formal analysis, Z.W. and A.N.; investigation, Z.W., J.B.III, M.G. and V.G.; resources, V.G., T.S. and O.G.; data curation, Z.W.; writing—original draft preparation, Z.W.; writing—review and editing, Z.W., V.G., E.A.K., and A.N.; visualization, Z.W.; supervision, E.A.K., V.G. and A.G.; project administration, V.G.; funding acquisition, E.A.K., O.G., A.G. and V.G. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported in part by the NIH—National Institute on Aging (NIA) under Grant 1R43AG067843 and in part by the Center for Injury Research and Prevention at the Children’s Hospital of Philadelphia.

Institutional Review Board Statement

This work involved human subjects in its research. The study was conducted in accordance with the Declaration of Helsinki, and approved by the Institutional Review Board of the Children’s Hospital of Philadelphia (IRB 17-014203) on 27 January 2022.

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

The data supporting this study’s findings are available from the corresponding authors upon reasonable request. The data is not publicly available due to the data sharing restriction from the IRB.

Acknowledgments

We would like to express our gratitude to the staff and students who volunteered their time to assist with the testbed assembly and experiment conducting. And we would like to thank the Institutional Review Board (IRB) at the Children’s Hospital of Philadelphia for their thorough review and approval of this study, which allowed us to conduct the research in an ethical and responsible manner.

Conflicts of Interest

The authors declare no financial or personal conflicts of interest that could have influenced the work reported in this study. Some authors are affiliated with GraceFall Inc. and have additional affiliations with academic institutions, including Drexel University and Ruppin Academic Center. All affiliations have been disclosed to ensure transparency.

References

  1. Centers for Disease Control and Prevention. Facts About Falls. Available online: https://www.cdc.gov/falls/data-research/facts-stats/index.html (accessed on 28 October 2024).
  2. World Health Organization. Falls. Available online: https://www.who.int/news-room/fact-sheets/detail/falls (accessed on 28 October 2024).
  3. National Center for Injury Prevention and Control (U.S.) Cost of Older Adult Falls. Available online: https://stacks.cdc.gov/view/cdc/122747 (accessed on 28 November 2024).
  4. The National Council on Aging. Get the Facts on Falls Prevention. Available online: https://www.ncoa.org/article/get-the-facts-on-falls-prevention/ (accessed on 28 October 2024).
  5. Gillard, P.J.; Ganapathy, V.; Graham, G.D.; DiBonaventura, M.D.; Goren, A.; Zorowitz, R. Caregiver burden, productivity loss, and indirect costs associated with caring for patients with poststroke spasticity. Clin. Interv. Aging 2015, 10, 1793. [Google Scholar] [CrossRef] [PubMed]
  6. National Institute on Aging. Cognitive Health and Older Adults. Available online: https://www.nia.nih.gov/health/brain-health/cognitive-health-and-older-adults (accessed on 28 October 2024).
  7. Moreland, B.; Kakara, R.; Henry, A. Trends in nonfatal falls and Fall-Related injuries among adults aged ≥65 years—United States, 2012–2018. MMWR Morb. Mortal. Wkly. Rep. 2020, 69, 875–881. [Google Scholar] [CrossRef] [PubMed]
  8. Bloem, B.R.; Grimbergen, Y.A.; Cramer, M.; Willemsen, M.; Zwinderman, A.H. Prospective assessment of falls in Parkinson’s disease. J. Neurol. 2001, 248, 950–958. [Google Scholar] [CrossRef] [PubMed]
  9. Mayo Clinic. Seizures. Available online: https://www.mayoclinic.org/diseases-conditions/seizure/symptoms-causes/syc-20365711 (accessed on 28 October 2024).
  10. Lord, S.R.; Menz, H.B.; Tiedemann, A. A Physiological Profile Approach to falls Risk assessment and Prevention. Phys. Ther. 2003, 83, 237–252. [Google Scholar] [CrossRef] [PubMed]
  11. SBNation. The Injury Bug: Why Some Athletes Are More Prone to Getting Hurt. Available online: https://www.baltimorebeatdown.com/2015/8/12/9128337/the-injury-bug-why-some-athletes-are-more-prone-to-getting-hurt (accessed on 28 October 2024).
  12. Wu, F.; Zhao, H.; Zhao, Y.; Zhong, H. Development of a Wearable-Sensor-Based fall detection system. Int. J. Telemed. Appl. 2015, 2015, 576364. [Google Scholar] [CrossRef]
  13. Lee, Y.; Yeh, H.; Kim, K.-H.; Choi, O. A real-time fall detection system based on the acceleration sensor of smartphone. Int. J. Eng. Bus. Manag. 2018, 10, 184797901775066. [Google Scholar] [CrossRef]
  14. De Miguel, K.; Brunete, A.; Hernando, M.; Gambao, E. Home Camera-Based Fall Detection System for the Elderly. Sensors 2017, 17, 2864. [Google Scholar] [CrossRef]
  15. Ding, C.; Zou, Y.; Sun, L.; Hong, H.; Zhu, X.; Li, C. Fall detection with multi-domain features by a portable FMCW radar. In Proceedings of the 2019 IEEE MTT-S International Wireless Symposium (IWS), Guangzhou, China, 19–22 May 2019. [Google Scholar] [CrossRef]
  16. Li, N.Y.; Ho, K.C.; Popescu, M. A microphone array system for automatic fall detection. IEEE Trans. Biomed. Eng. 2012, 59, 1291–1301. [Google Scholar] [CrossRef]
  17. Wang, Z.; Ramamoorthy, V.; Gal, U.; Guez, A. Possible Life Saver: A Review on Human Fall Detection Technology. Robotics 2020, 9, 55. [Google Scholar] [CrossRef]
  18. Martínez-Villaseñor, L.; Ponce, H.; Brieva, J.; Moya-Albor, E.; Núñez-Martínez, J.; Peñafort-Asturiano, C. UP-Fall Detection Dataset: A multimodal approach. Sensors 2019, 19, 1988. [Google Scholar] [CrossRef]
  19. Dhole, S.R.; Kashyap, A.; Dangwal, A.N.; Mohan, R. A novel helmet design and implementation for drowsiness and fall detection of workers on-site using EEG and Random-Forest Classifier. Procedia Comput. Sci. 2019, 151, 947–952. [Google Scholar] [CrossRef]
  20. Mayo Clinic. EEG (Electroencephalogram). Available online: https://www.mayoclinic.org/tests-procedures/eeg/about/pac-20393875 (accessed on 28 October 2024).
  21. Latt, L.D.; Sparto, P.J.; Furman, J.M.; Redfern, M.S. The steady-state postural response to continuous sinusoidal galvanic vestibular stimulation. Gait Posture 2003, 18, 64–72. [Google Scholar] [CrossRef] [PubMed]
  22. Keshner, E.A.; Kenyon, R.V. The influence of an immersive virtual environment on the segmental organization of postural stabilizing responses. J. Vestib. Res. 2000, 10, 207–219. [Google Scholar] [CrossRef]
  23. Looney, D.; Kidmose, P.; Park, C.; Ungstrup, M.; Rank, M.; Rosenkranz, K.; Mandic, D. The in-the-ear recording concept: User-centered and wearable brain monitoring. IEEE Pulse 2012, 3, 32–42. [Google Scholar] [CrossRef] [PubMed]
  24. Liang, H.; Wang, Y.; Li, H.; Wang, Y.; Liu, P.X.; Liu, R. Development and characterization of a dry Ear-EEG sensor with a generic flexible earpiece. IEEE Trans. Instrum. Meas. 2023, 72, 1–12. [Google Scholar] [CrossRef]
  25. Wang, Z.; Graci, V.; Seacrist, T.; Guez, A.; Keshner, E.A. Localizing EEG recordings associated with a balance threat during unexpected postural translations in young and elderly adults. IEEE Trans. Neural Syst. Rehabil. Eng. 2023, 31, 4514–4520. [Google Scholar] [CrossRef]
  26. Shen, H.; Xu, M.; Guez, A.; Li, A.; Ran, F. An accurate sleep stages classification method based on state space model. IEEE Access 2019, 7, 125268–125279. [Google Scholar] [CrossRef]
  27. Wang, Z.; Sperling, M.R.; Wyeth, D.; Guez, A. Automated seizure detection based on State-Space Model Identification. Sensors 2024, 24, 1902. [Google Scholar] [CrossRef]
  28. Vijayan, A.E.; Sen, D.; Sudheer, A.P. EEG-Based Emotion Recognition Using Statistical Measures and Auto-Regressive Modeling. In Proceedings of the 2015 IEEE International Conference on Computational Intelligence & Communication Technology, Riga, Latvia, 3–5 June 2015; pp. 587–591. [Google Scholar] [CrossRef]
  29. Shahabi, H.; Moghimi, S. Toward automatic detection of brain responses to emotional music through analysis of EEG effective connectivity. Comput. Hum. Behav. 2016, 58, 231–239. [Google Scholar] [CrossRef]
  30. Kayikcioglu, T.; Maleki, M.; Eroglu, K. Fast and accurate PLS-based classification of EEG sleep using single channel data. Expert Syst. Appl. 2015, 42, 7825–7830. [Google Scholar] [CrossRef]
  31. Hsu, W.-Y. Assembling a Multi-Feature EEG classifier for Left–Right motor imagery data using Wavelet-Based fuzzy approximate entropy for improved accuracy. Int. J. Neural Syst. 2015, 25, 1550037. [Google Scholar] [CrossRef] [PubMed]
  32. Van Overschee, P.; De Moor, B. N4SID: Subspace algorithms for the identification of combined deterministic-stochastic systems. Automatica 1994, 30, 75–93. [Google Scholar] [CrossRef]
  33. Zhu, W.; Zeng, N.; Wang, N. Sensitivity, specificity, accuracy, associated confidence interval and ROC analysis with practical SAS implementations. In Proceedings of the NESUG Proceedings: Health Care and Life Sciences, Baltimore, MD, USA, 14–17 November 2010; Volume 19, p. 67. [Google Scholar]
  34. Nashner, L.M. Fixed patterns of rapid postural responses among leg muscles during stance. Exp. Brain Res. 1977, 30, 13–24. [Google Scholar] [CrossRef] [PubMed]
  35. Allum, J.H.J. Organization of stabilizing reflex responses in tibialis anterior muscles following ankle flexion perturbations of standing man. Brain Res. 1983, 264, 297–301. [Google Scholar] [CrossRef]
  36. Tsao, H.; Galea, M.P.; Hodges, P.W. Reorganization of the motor cortex is associated with postural control deficits in recurrent low back pain. Brain 2008, 131, 2161–2171. [Google Scholar] [CrossRef]
  37. Wu, Z.; Pan, S.; Chen, F.; Long, G.; Zhang, C.; Yu, P.S. A Comprehensive Survey on Graph Neural Networks. IEEE Trans. Neural Netw. Learn. Syst. 2021, 32, 4–24. [Google Scholar] [CrossRef]
  38. Hu, F.; Zhang, L.; Yang, X.; Zhang, W.-A. EEG-Based driver Fatigue Detection using Spatio-Temporal Fusion network with brain region partitioning strategy. IEEE Trans. Intell. Transp. Syst. 2024, 25, 9618–9630. [Google Scholar] [CrossRef]
Figure 1. (A) Experimental setup illustration, showing “upright stance” (left) and “balance perturbation” (right) stages. (B) Scalp EEG channel locations. (C) Full view of the cEEGrid piece. (D) Illustration of subject wearing the cEEGrids. (E) Mastoid channel locations.
Figure 1. (A) Experimental setup illustration, showing “upright stance” (left) and “balance perturbation” (right) stages. (B) Scalp EEG channel locations. (C) Full view of the cEEGrid piece. (D) Illustration of subject wearing the cEEGrids. (E) Mastoid channel locations.
Sensors 24 07779 g001
Figure 2. Exemplary EEG (Ch. R6) activity during two unpredictable events and ten predictable events. Minimal differences in the EEG activity were observed between these two types of events up to 80 ms following the event onset. The vertical line marks the onset of each event.
Figure 2. Exemplary EEG (Ch. R6) activity during two unpredictable events and ten predictable events. Minimal differences in the EEG activity were observed between these two types of events up to 80 ms following the event onset. The vertical line marks the onset of each event.
Sensors 24 07779 g002
Figure 3. Exemplar event data illustrating EEG (Ch. R6) and platform-based acceleration. The EEG data shown were detrended, baseline corrected, and filtered using a 2nd-order Butterworth bandpass filter (2.5 Hz–30 Hz) and a 60 Hz notch filter. The acceleration data represents one-dimensional raw measurements in the antero-posterior direction from the platform. The vertical line marks the onset of the event.
Figure 3. Exemplar event data illustrating EEG (Ch. R6) and platform-based acceleration. The EEG data shown were detrended, baseline corrected, and filtered using a 2nd-order Butterworth bandpass filter (2.5 Hz–30 Hz) and a 60 Hz notch filter. The acceleration data represents one-dimensional raw measurements in the antero-posterior direction from the platform. The vertical line marks the onset of the event.
Sensors 24 07779 g003
Figure 4. Block diagram for the proposed dynamic state-space model.
Figure 4. Block diagram for the proposed dynamic state-space model.
Sensors 24 07779 g004
Figure 5. Algorithm workflow for real-time fall detection using EEG and acceleration signals.
Figure 5. Algorithm workflow for real-time fall detection using EEG and acceleration signals.
Sensors 24 07779 g005
Table 1. Results of classification for different windows (0 ms–cut-off) using AR and SE features.
Table 1. Results of classification for different windows (0 ms–cut-off) using AR and SE features.
Epoch LengthSensitivitySpecificityAccuracy
0–350 ms60.3%90.1%81.7%
0–500 ms62.9%91.6%83.5%
0–750 ms72.2%94.8%88.4%
0–1000 ms74.3%94.3%88.6%
Table 2. Results of classification for different windows (80 ms–cut-off) using AR and SE features.
Table 2. Results of classification for different windows (80 ms–cut-off) using AR and SE features.
Epoch LengthSensitivitySpecificityAccuracy
80–350 ms66.7%93.6%86.0%
80–500 ms69.2%93.5%86.6%
80–750 ms73.4%94.0%88.1%
80–1000 ms76.8%94.3%89.3%
Table 3. Results of classification for different windows (0 ms–cut-off) using state-space features.
Table 3. Results of classification for different windows (0 ms–cut-off) using state-space features.
Epoch LengthSensitivitySpecificityAccuracy
0–350 ms87.1%96.1%93.1%
0–500 ms88.7%96.6%94.0%
0–750 ms89.1%96.8%94.1%
0–1000 ms90.9%96.9%94.5%
Table 4. Results of classification for different windows (80 ms–cut-off) using state-space features.
Table 4. Results of classification for different windows (80 ms–cut-off) using state-space features.
Epoch LengthSensitivitySpecificityAccuracy
80–350 ms87.9%96.7%93.9%
80–500 ms89.1%97.0%94.4%
80–750 ms89.4%97.2%94.6%
80–1000 ms90.9%97.3%95.2%
Table 5. Results of real-time sensor fusion algorithm.
Table 5. Results of real-time sensor fusion algorithm.
Training SetSensitivitySpecificityAccuracy
198.1%99.7%99.7%
294.7%99.6%99.6%
390.1%99.6%99.5%
487.5%99.60%98.9%
595.3%99.8%99.8%
693.1%99.5%99.5%
Average93 ± 3%99.6 ± 0.1%99.5 ± 0.3%
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Wang, Z.; Noah, A.; Graci, V.; Keshner, E.A.; Griffith, M.; Seacrist, T.; Burns, J., III; Gal, O.; Guez, A. Real-Time Postural Disturbance Detection Through Sensor Fusion of EEG and Motion Data Using Machine Learning. Sensors 2024, 24, 7779. https://doi.org/10.3390/s24237779

AMA Style

Wang Z, Noah A, Graci V, Keshner EA, Griffith M, Seacrist T, Burns J III, Gal O, Guez A. Real-Time Postural Disturbance Detection Through Sensor Fusion of EEG and Motion Data Using Machine Learning. Sensors. 2024; 24(23):7779. https://doi.org/10.3390/s24237779

Chicago/Turabian Style

Wang, Zhuo, Avia Noah, Valentina Graci, Emily A. Keshner, Madeline Griffith, Thomas Seacrist, John Burns, III, Ohad Gal, and Allon Guez. 2024. "Real-Time Postural Disturbance Detection Through Sensor Fusion of EEG and Motion Data Using Machine Learning" Sensors 24, no. 23: 7779. https://doi.org/10.3390/s24237779

APA Style

Wang, Z., Noah, A., Graci, V., Keshner, E. A., Griffith, M., Seacrist, T., Burns, J., III, Gal, O., & Guez, A. (2024). Real-Time Postural Disturbance Detection Through Sensor Fusion of EEG and Motion Data Using Machine Learning. Sensors, 24(23), 7779. https://doi.org/10.3390/s24237779

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop