1. Introduction
More than
older adults (OAs) are admitted annually to hospitals with hip replacements and lower limb fractures affecting the hip, femur, or pelvis in Canada [
1]. This number is predicted to increase as the population ages [
2]. It is projected that the worldwide annual incidence of hip fractures will increase to approximately
million by the year 2050 [
3]. The Canadian government currently spends CAD 650 million annually on hip fractures alone, and this amount is expected to rise to CAD
billion by 2041 [
4], indicating an urgent economic need to support physical recovery for this group. OAs who experience lower limb fractures coupled with other living conditions, such as living alone and limited mobility, are at greater risk of poor health outcomes, including social isolation (SI) and functional decline (FD) [
5].
Hip fractures are the most common type of fracture incurred among OAs. It is estimated that 20% of OAs with post-hip fracture experience SI [
6]. SI impacts one in five OAs in the United States of America [
7] and up to 24% of OAs in Canada [
8]. Social Isolation is defined as the state of having a small network of relationships with family members and friends, resulting in few or infrequent interactions [
9]. According to a nationally representative study of adults aged 50 and older [
10], SI was expressed by seven indicators, including the presence or absence of (1) adult children, (2) friends, (3) other family members, (4) being unmarried, (5) living alone, and (6) refusing to participate in social groups or (7) religious activities. Life transitions such as retirement, loss of a spouse, migration of children, disability or loss of mobility [
9], frailty, and cognitive vulnerabilities [
10] can also greatly affect older people and put them at risk of SI [
9]. SI is also a public health concern with economic consequences [
11]. In Canada, socially isolated OAs tend to visit their doctor and emergency rooms more often, use more medications, fall more often, and enter residential long-term care homes sooner, compared to those who are not socially isolated [
11]. This leads to an increase in healthcare and social service costs [
11].
FD is a risk factor of SI and is common in OAs who have been hospitalized for an acute health problem, such as a hip fracture [
12]. In this context, FD is defined as the “drop in physical and/or cognitive functioning” [
13]. In some cases, FD can occur as early as on the second day of hospitalization, and only 50% of the hospitalized patients recover three months after hospital discharge [
13]. The risk factors of FD include depression, increased length of hospital stay, poor living accommodation, and poor social support [
12]. According to a study by Helvik et al. [
12], poor quality of life was significantly associated with FD 2 months after hospital discharge. This finding underscores the importance of this time frame to the long-term health of OAs after a hip fracture.
Wearable devices and sensor technologies in conjunction with cloud services are increasingly being used in healthcare, more specifically in the hip fracture population, for improving care and enhancing quality of life [
4,
14]. The ubiquity and wide availability of mobile technologies and other smart home devices offer enhanced clinical care and facilitate new research in the field [
4,
14]. An in-home sensor system has the potential to objectively collect physical, physiological, contextual, and social interaction data among OAs living in the community. Using smart devices allows for remote data acquisition on individuals and their daily routines and behaviors in real time. These innovative technologies and sensor systems are promising to support aging at home, which is aligned with
of OA homeowners in Canada who wish to live at home for as long as possible [
15]. These technological advances have made monitoring systems unobtrusive and highly effective in monitoring OAs’ activities of daily living and supporting aging at home [
16].
3. Gaps in the Literature
From the literature review in the previous section, few studies have focused on objectively measuring FD in OAs using sensors. Most of the studies available in the literature focus on collecting sensor data to assess or monitor cognitive decline rather than physical/functional decline. Additionally, previous research using sensors to assess SI have generally used only one sensor or modality to collect data from participants to track and monitor their activities of daily living. The single modality approach leads to one-dimensional (view of) data, which may not be sufficiently representative to assess SI and FD. To comprehensively assess SI and FD, a more holistic approach is required that can capture different types of data based on mobility, physiological indicators, indoor and outdoor motion, sleep, and social behaviors [
17]. A suggested approach [
17] is using multiple or multimodal sensors for collecting objective data from participants to comprehensively monitor a multitude of daily routines and behaviors. This multimodal sensor approach ensures more reliable data collection, as data are gathered from multiple sensors, reducing the risk of data loss due to sensor malfunction. Further, it can provide complementary information from different sensors, enhancing our understanding of SI and the functional recovery of OAs, and facilitating the development of generalizable predictive models in the future.
Most studies have collected sensor data over a short duration, typically around one week, to monitor patients for SI [
17]. This duration may be insufficient, as SI does not change drastically over time; it is not an instant event (i.e., a fall) [
17]. Long-term studies spanning several months are more beneficial, as SI develops gradually. To enhance the accuracy of assessments and predictive algorithms, various types of objective data can be collected, including social behaviors, sleep metrics, mobility, indoor motion, GPS data, step count, and physiological data (e.g., heart rate). The aforementioned data types are aligned and correlated with the definitions of SI and FD and capture patients’ indoor and outdoor behaviors, sleep patterns, and physiological and mobility indicators.
The aim of this current study was to investigate the correlations between SI and FD using various clinical and sensor data, including sleep patterns, physiological indicators, mobility, and social interactions. Collecting data from various sensing modalities is crucial for understanding the complex behavioral, physiological, and social dynamics of OAs recovering from lower limb fractures in the community. To address this need, our team employed MAISON (Multimodal AI-based Sensor Platform for Older iNdividuals), a multimodal sensor system designed to collect data from multiple modalities [
29]. The multimodal data are encrypted and stored in a private cloud [
30]. This work is the foundational step to inform a clinically validated system to evaluate and assess OAs’ social and physical mobility activities.
4. Methods
In this longitudinal observational study, eight OAs were recruited who were admitted to the inpatient rehabilitation at Toronto Rehabilitation Institute (TRI), University Health Network (UHN), with lower limb fractures and subsequently discharged to go home. The data collection period for each participant was 8 weeks once they returned home. The study’s duration was based on prior research indicating that the first 1 to 3 months post-discharge are crucial for the overall healing trajectory of OAs after a lower limb fracture [
31]. A literature review on complications in patients with postoperative hip fractures found that the first month after discharge is particularly vulnerable, posing high risks for 18 health complications including infections, depression, cardiac failure, and hospital readmission [
31]. The review also noted that 5 complications remain likely for up to 3 months post-discharge [
31]. In another study, symptoms of depression and loneliness were exhibited immediately and 6 weeks after surgery [
32]. Therefore, an 8-week period immediately following discharge was chosen to effectively capture these significant changes in health and function.
The inclusion criteria for the study were as follows: (a) 50 years old or older; (b) had a surgically-repaired hip, femur, or pelvis fracture, or a hip replacement surgery; (c) speaks English and can provide consent; (d) lives within a 60-minute commute from TRI-UHN; (e) lives alone (both with or without pets); (f) has Wi-Fi; (g) does not have cognitive impairment (participant has Mini Mental State Examination (MMSE) score of 24 or more); (h) should not have any physician orders in the medical chart restricting hip flexion greater than 90 degrees. Patient recruitment was conducted through UHN Central Recruitment, where the patients were screened for eligibility and consent was obtained. Patient recruitment and enrollment started prior to the discharge date of the patients; therefore, the data collection started immediately or shortly after the discharge date. This study received ethical approval from the UHN Research Ethics Board #20-5113.
4.1. Clinical Data Collection
Table 1 lists the clinical data (two physical tests and three questionnaires) that were collected through biweekly video calls using the Microsoft Teams application. The video calls were recorded and conducted biweekly, and the interviews were audio recorded during the initial sensor installation and final sensor takedown.
The biweekly assessments were scheduled based on participant’s availability, resulting in a mix of morning and afternoon sessions. Research indicates that circadian rhythms can indeed influence physical performance and cognitive function, for example, [
33]. Similarly, cognitive performance and alertness can vary throughout the day [
34]. To mitigate these circadian effects, we ensured that assessments were consistently scheduled at similar times for each participant throughout the study. This approach aimed to reduce intra-individual variability related to time-of-day effects.
Further, demographic data (e.g., age, gender, type of fracture, employment status, etc.) were collected at baseline during sensor installation.
The clinical assessments conducted in this study are part of standard care in lower limb fracture rehabilitation. The questionnaires include SIS [
19], Oxford Hip Score [
35], and Oxford Knee Score [
36]. The physical tests are Timed Up and Go Test (TUG) [
37] and the 30 s chair stand [
38]. The Oxford Hip Score, the Oxford Knee Score, and the physical tests were chosen to evaluate functional mobility in OAs. The results from these assessments were compared to the sensor data collected to determine correlations between the functional recovery of the patients and their SI as they recovered from their lower limb fractures.
4.1.1. Social Isolation Scale
The SIS questionnaire consists of six multiple-choice questions, three of which are focused on the number of friends the individual has and overall social connectedness, and the other three are focused on their sense of belonging and contentment [
19]. Each question is labeled with a score, and the sum of all of them out of 30 is the SI score. The total SIS score ranges from 6–30 with higher values indicating lower levels of SI.
4.1.2. Oxford Hip Score and Oxford Knee Score
These two questionnaires are designed to assess functional mobility in the hip and the knee. The two questionnaires consist of 12 multiple-choice questions each, focusing on the patient’s recovery for and any pain experienced during activities they are able to perform independently, such as doing the household shopping, walking down a flight of stairs, and taking a shower. The score ranges for each are from 0 to 48, with the higher value indicating higher physical health or functional ability in the hip and/or the knee. The score for each item on these scales ranges from 1 to 5, with the higher value indicating lower physical health in the hip/knee. The overall score is calculated by the sum of the score from each item in the questionnaire, subtracted from 60.
4.1.3. Timed up and Go Test
This is a physical test measuring basic mobility movements [
37]. The TUG is appropriate for following clinical changes in the patient over time [
37]. In this test, the OA is asked to get up from a chair, walk three meters using an aid, if required, and then walk back three meters and sit back down in the chair. The time taken to complete this exercise is recorded as the score for the TUG test.
4.1.4. 30 s Chair Stand
This physical test is a functional evaluation to measure the lower body strength [
38]. This test is of high clinical importance since it is related to some of the demanding daily life activities, such as climbing stairs or getting out of a chair [
38]. In this test, the patient is asked to fully get up from a chair while having their arms crossed and is asked to sit back down continuously. The number of times the patient is able to perform this exercise in 30 s is the score for the 30 s chair stand.
4.2. Sensor Data Collected Using MAISON
The MAISON system [
29] consists of four main components: (1) a smartphone application, (2) a smartwatch application, (3) wearable and non-wearable sensors along with a provision to collect data directly from the device or their associated clouds, and (4) a central cloud for data storage. These components can be seen in
Figure 1. In this study, the MAISON was utilized to collect multimodal sensor data from eight participants (
). More specifically, MAISON [
29] consists of four smart devices including an Android smartphone with a MAISON app, a smartwatch with a MAISON app, a motion sensor, and a sleep-tracking mattress sensor. The Android smartwatch is worn by the participant throughout the day, and they take the smartphone with them when they leave their homes to collect GPS data. The sleep-tracking mattress sensor is positioned under the mattress where the participant sleeps. During our intake survey, we identified where the participant generally sleeps on their bed and placed the mattress sensor underneath. The motion sensor was put in the living room to capture patterns of motion throughout the day. For participants who had pets, we strategically placed sensors to minimize interference from pets. For instance, motion sensors were positioned at heights and locations less likely to capture pet movements and the participants’ pets did not sleep with them on the bed.
MAISON’s sensors collect the following data: raw acceleration, heart rate, number of steps, GPS, walking speed, sleep related metrics, and indoor motion.
Table 2 summarizes the types of data collected from each smart device.
The smartphone application collects GPS data at a frequency of
Hz (one observation in 30 s). The smartwatch application collects heart rate, step count, and raw acceleration data while it is worn by the patient. The accelerometer data on the watch are collected at a frequency of 1 Hz, and the heart rate data are collected every 30 min for 30 s. The motion sensors collect the frequency and pattern of indoor movement in the living room, and lastly, the Withings sleep-tracking mattress sensor [
39] collects sleep-related metrics. Several papers have reported scientific evaluations of the Withings sleep sensor. Kainec et al. [
40] reported that total sleep time was accurately estimated by the Withings sleep sensor within clinically acceptable ranges. Another study by Mantua et al. [
41] found a strong correlation between the total sleep time and Polysonomography (PSG) for the Withings sleep sensor. The sampling frequency for each of the devices was chosen based on the literature, feedback from clinicians, and convenience for the participant as considerations about data collection and battery life. For example, clinicians were consulted regarding the frequency and duration of the collection of heart rate data, and the frequency of acceleration data were based on the available literature. Among the different data modalities collected by MAISON, one modality is available on a per-day basis (e.g., sleep metrics), three are event based (e.g., indoor motion, GPS, number of steps), and two are collected on a continuous basis (e.g., raw acceleration, heart rate).
4.3. Feature Extraction
From the collected sensor data, different types of features can be extracted and analyzed. One category of features is statistical features, which illustrate patterns and trends about the data collected. Some of the statistical features include average step count, total step count, average heart rate during the day and during sleep, total sleep duration, number of motion events, and average acceleration. For acceleration data, a python library called FLIRT [
42] was used to extract additional general time domain features from acceleration data. FLIRT [
42] uses a function where it takes the
x,
y, and
z components of raw acceleration data as input and the output is 85 features, demonstrating various metrics such as mean, minimum, maximum, skewness, kurtosis, etc., for
and the
norm. Out of the 85 features, only the features for the
norm are kept for further analysis since the individual
x,
y, and
z components do not reveal any useful data for the correlation analysis. The GPS data collected from the smartphone can provide insight into the patients’ outdoor movements, the amount of time spent outside, and the distance traveled. For sleep-related features, the start date of the sleep was considered for the sleep episode (the night after). A total of 53 features extracted from eight participants can be seen in
Table 3, categorized into data modalities.
The features were extracted from the sensor data on a daily or event-based basis. However, the clinical data were collected every two weeks. Therefore, two different time resolutions were used for the extracted features: daily and biweekly. To obtain the biweekly features, the daily features extracted from the sensor data were processed in various ways. For example, the summation of position-count daily feature (described in
Table 3) was used as the biweekly position-count feature. Various functions were used to compute the biweekly features from the daily features, such as
,
,
, and
. The last column in
Table 3 presents the function used for biweekly feature extraction (FE) on each daily feature. Several algorithms were created in Python for preprocessing and feature extraction purposes for daily and biweekly time scales.
6. Discussion and Conclusions
The findings in this paper suggest that different dimensions of health, such as number of social interactions, sense of belonging, and physical mobility, may be moderately interconnected with physiological measurements and behaviors, although the findings are nuanced and not straightforward. The strong correlations between the questions from the Oxford Hip and Knee scores and the sensor data highlight the direct impact of physical health status on measurable daily physical activities.
The findings from the daily correlation analysis suggest that as OAs experience improvements or declines in joint function and pain, the sensors are able to capture these changes in their movement and level of activity. Comparing the results between daily and biweekly correlation analyses, it can be seen that similar correlations between the clinical data and features extracted from the sensor data were found. Looking at
Table 7 and
Table 11, item 6 on the SIS has a daily moderate correlation and a biweekly strong correlation with duration of sleep (
and
, respectively). A similar occurrence can be seen between item 6 on SIS and the step-sum (total step count) with
and
correlation values of daily and biweekly, respectively. The repeated correlation, although with different values, gives strength to the findings in this paper. The consistency in correlation results suggests that sensor data can provide reliable insights into changes in participants’ SI levels over time. From both
Table 7 and
Table 11, sleep metrics have repeatedly shown correlations with the SIS metrics, marking the importance of sleep metrics in assessing SI, which is consistent with the previous literature [
18,
45,
46].
Table 6,
Table 8,
Table 9,
Table 12 and
Table 13 show that the Oxford Knee Score was strongly correlated with the daily and biweekly features extracted from the sensor data in comparison to the Oxford Hip Score. Therefore, the Oxford Knee Score can be considered as a superior instrument for evaluating the correlation between functional recovery and sensor data when addressing various types of lower limb fractures.
The outcomes of daily and biweekly correlations between SI and sensor data are similar, both demonstrating a relationship between SIS scores and sleep data with various degrees of correlation. This similarity also exists with the findings from daily and biweekly correlations between FD and sensor data, in that they are both correlated with heart rate and motion features. However, in the biweekly correlation of FD and SI with sensor data, strong correlations exist between the clinical data and acceleration (i.e., -max, -mean) and position features (distance traveled from home), which is not found in daily correlations. This finding shows that the changes in acceleration and position features are not visible on a day-to-day basis but rather on a longer, 2-week period.
There are limitations in this research. A limited sample size from only one recruitment site may limit the findings of our study. Psychological state or pre-existing conditions of depression or anxiety disorders of the participants were not tracked and controlled in this study. Another limitation is the gender and ethnicity of the patients who participated in the study. Although the inclusion criteria were not limited to any specific ethnicity or gender, the consented population turned out to be mostly females and Caucasians. Efforts to diversify the recruitment sites for this study can mitigate this potential bias. Further, when collecting GPS data, the locations the participants traveled to were not considered a feature as this would require more inputs from the participants and risk overburdening them. One of the features that was explored and supported by the work of Goonawardene et al. [
22] was the frequency of daytime napping. Day time napping was originally considered as a feature, but as participants took naps sometimes on the couch or other places other than the bed, it was difficult to accurately account for the midday naps. Another limitation of this study is that socialization data were not collected, for example, monitoring phone contacts or phone usage. For privacy reasons, we were unable to collect these data in a de-identified manner without undue burden on our OA participants who had just returned home from rehabilitation.
The missing clinical data were primarily due to technical issues with using video recordings using MS Teams, participants being physically unfit to perform assessments, and the refusal of participants to answer certain questions because some participants thought it was not applicable to them (e.g., answering questions about knee function after a pelvis fracture).
Future work will include scaling this study to include more recruitment sites, advancing diversity in sex/gender, age, and ethnicity, and investigating major changes from the current findings. The plan includes assessing the correlations with each fracture type from both daily and biweekly perspectives. The work presented in this paper is the foundation of creating a feature dictionary used to develop an algorithm for detecting SI and FD. The development of predictive models to assess SI and FD using the data presented in this paper is currently underway. Moreover, we plan to continue data collection from 20 more participants to increase the sample size, which will not only enhance the statistical power of our findings but also allow for a more robust and comprehensive analysis of the correlations between SI, FD, and sensor data. We are also exploring relationships between various types of patients and features by combining unsupervised learning and formal concept analysis [
47]. Future research investigating the patient experience using the system and engaging in co-designing it to ensure it meets patients’ and clinicians’ needs is warranted.