Correlations between Social Isolation and Functional Decline in Older Adults after Lower Limb Fractures Using Multimodal Sensors: A Pilot Study

Abstract

Older adults (OAs) recovering from lower limb fractures experience social isolation (SI) and functional decline (FD) after they are discharged from inpatient rehabilitation due to reduced physical mobility. Our research used MAISON (Multimodal AI-based Sensor platform for Older iNdividuals), a multimodal sensor system comprising various smart devices collecting acceleration, heart rate, step count, frequency of indoor motion, GPS, and sleep metrics. This study aimed to investigate the correlations between SI and FD with multimodal sensor data from OAs following lower limb fractures. Multimodal sensor data from eight OAs (8 weeks per person) living at home were collected. Five clinical metrics were obtained via biweekly video calls, including three clinical questionnaires (Social Isolation Scale (SIS), Oxford Hip Score, Oxford Knee score) and two physical mobility assessments (Timed Up and Go, 30 s chair stand). From the sensor data collected, 53 statistical and domain features were extracted. Spearman correlation coefficients were calculated between the extracted features and clinical data. The results indicated strong correlations between various items of SIS and sleep metrics in OAs and various items of Oxford Knee Score with GPS and acceleration data. Strong correlations between the questions of the Oxford scores and sensor data highlight the direct impact of physical health status on measurable daily physical activities.

1. Introduction

More than $\mathrm{30,000}$ older adults (OAs) are admitted annually to hospitals with hip replacements and lower limb fractures affecting the hip, femur, or pelvis in Canada [1]. This number is predicted to increase as the population ages [2]. It is projected that the worldwide annual incidence of hip fractures will increase to approximately $4.5$ million by the year 2050 [3]. The Canadian government currently spends CAD 650 million annually on hip fractures alone, and this amount is expected to rise to CAD $2.4$ billion by 2041 [4], indicating an urgent economic need to support physical recovery for this group. OAs who experience lower limb fractures coupled with other living conditions, such as living alone and limited mobility, are at greater risk of poor health outcomes, including social isolation (SI) and functional decline (FD) [5].

Hip fractures are the most common type of fracture incurred among OAs. It is estimated that 20% of OAs with post-hip fracture experience SI [6]. SI impacts one in five OAs in the United States of America [7] and up to 24% of OAs in Canada [8]. Social Isolation is defined as the state of having a small network of relationships with family members and friends, resulting in few or infrequent interactions [9]. According to a nationally representative study of adults aged 50 and older [10], SI was expressed by seven indicators, including the presence or absence of (1) adult children, (2) friends, (3) other family members, (4) being unmarried, (5) living alone, and (6) refusing to participate in social groups or (7) religious activities. Life transitions such as retirement, loss of a spouse, migration of children, disability or loss of mobility [9], frailty, and cognitive vulnerabilities [10] can also greatly affect older people and put them at risk of SI [9]. SI is also a public health concern with economic consequences [11]. In Canada, socially isolated OAs tend to visit their doctor and emergency rooms more often, use more medications, fall more often, and enter residential long-term care homes sooner, compared to those who are not socially isolated [11]. This leads to an increase in healthcare and social service costs [11].

FD is a risk factor of SI and is common in OAs who have been hospitalized for an acute health problem, such as a hip fracture [12]. In this context, FD is defined as the “drop in physical and/or cognitive functioning” [13]. In some cases, FD can occur as early as on the second day of hospitalization, and only 50% of the hospitalized patients recover three months after hospital discharge [13]. The risk factors of FD include depression, increased length of hospital stay, poor living accommodation, and poor social support [12]. According to a study by Helvik et al. [12], poor quality of life was significantly associated with FD 2 months after hospital discharge. This finding underscores the importance of this time frame to the long-term health of OAs after a hip fracture.

Wearable devices and sensor technologies in conjunction with cloud services are increasingly being used in healthcare, more specifically in the hip fracture population, for improving care and enhancing quality of life [4,14]. The ubiquity and wide availability of mobile technologies and other smart home devices offer enhanced clinical care and facilitate new research in the field [4,14]. An in-home sensor system has the potential to objectively collect physical, physiological, contextual, and social interaction data among OAs living in the community. Using smart devices allows for remote data acquisition on individuals and their daily routines and behaviors in real time. These innovative technologies and sensor systems are promising to support aging at home, which is aligned with $86\%$ of OA homeowners in Canada who wish to live at home for as long as possible [15]. These technological advances have made monitoring systems unobtrusive and highly effective in monitoring OAs’ activities of daily living and supporting aging at home [16].

2. Literature Review

In this section, we present a brief overview of previous studies that used sensors to assess SI and/or FD in OAs.

2.1. Studies Assessing SI Using Sensors

Recent studies have increasingly utilized sensor-based technologies to objectively measure and predict SI. A scoping review by Khan et al. [17], comprising eight studies, focused on SI as an objective measure with OAs residing in the community. Studies involving OAs in congregate living settings, such as nursing homes, were excluded. The findings suggested that future research should broaden the range of behavioral and social features derived from sensors for the development of predictive models. Furthermore, this scoping review indicates that longitudinal studies to explore meaningful feature extraction from sensing modalities are needed to determine the significant associates of extracted features with SI. Both of these directions are required to enhance the accuracy of future predictive models.

Jiang et al. [18] explored the relationship between SI and OAs using both cross-sectional and longitudinal approaches. In their research, Social Isolation Scale (SIS) [19] was used to assess subjective and objective SI, and sleep data collected from community-dwelling OAs were used for cross-sectional ($n=108$) and longitudinal ($n=1554$) analyses. A strong association was found between SI and sleep metrics in OAs. Self-reported SI was directly linked to self-reported sleep quality, while the objective SI was correlated with objective sleep metrics. The results from the longitudinal analysis showed that loneliness significantly affected the relationship between SI and sleep over time, independent of demographic factors among the participants.

In the work of McCrory et al. [20], a physiological sensor for measuring electrocardiography was used to collect data from 4888 participants. The age range of participants was 50 years old and older (2242 males, 2646 females). In this study, the Berkman Social Disengagement Index [21] was used to assess SI by measuring the presence of a spouse, the number of in-person contacts with three or more relatives and close friends, and the number of non-visual (online or over the phone) yearly contact with 10 or more relatives and close friends. This scale also measures the frequency of participation in social activities. The study concluded that there is a negative correlation between social network size and the resting heart rate.

The research presented by Goonawardene et al. [22] focused on detecting SI using motion sensors. In this study, SI was described as the absence of interpersonal contacts with society. The data were collected from 46 participants above 60 years old (19 males, 27 females). The scale used in this work for assessing SI was the Lubben Social Network Scale (LSNS) [23]. The results from the study indicated a correlation between the average time spent outside of the home with the overall SI level. Ji et al. [24] used seven sensors to collect data from 22 participants over 140 weeks across the United States of America and Japan. A total of 22 features were extracted from the sensor data in order to predict loneliness caused by the SI intensified by COVID-19 pandemic. The UCLA loneliness score [25] was collected on a weekly basis and then correlated using Pearson’s (P) and Spearman’s ($\rho$) correlations with the sensor data. The LSNS [23] was also used at the baseline. The results of the study indicated that the sleep mattress and the temperature humidity sensors were the most predictive ones in detecting loneliness.

These studies collectively underscore the importance of comprehensive, longitudinal approaches to feature extraction and analysis to enhance the accuracy of SI predictive models and improve interventions for OAs in the community.

2.2. Studies Assessing FD Using Sensors

Aramendi et al. [26] utilized longitudinal smart home data collected from 29 OAs for an average of more than two years to predict standardized functional health scores. The sensors in this study were passive infra-red (PIR) presence sensors that tracked participants’ activities by capturing the movement whenever an event was detected by a sensor. These sensors were placed in different locations of the houses, and the number of sensors installed in each apartment differed based on the size and shape of the house. Their functional health was assessed every six months using Instrumental Activities of Daily Living-Compensation score [27]. The Spearman correlation was used in order to evaluate the relationship between functional health and the sensor data. The results were inconclusive in detecting functional health but emphasized the need for more in-depth feature selection analyses for future works. Alexander et al. [28] focused on evaluating a sensor system being facilitated to monitor functional ability in OAs based on data displays. The data collection took place in TigerPlace, an assisted living facility at the University of Missouri, where data from 14 residents were collected. The sensor network comprised a bed sensor or a wall-mounted room sensor detecting motion, a stove temperature sensor, a sensor detecting restlessness in bed, and physiological sensors measuring pulse and breathing rates. Due to a lack of a patient-focused design, the study was unable to verify the relationship between the sensor data and functional evaluation of the residents.

Both studies underscore the potential of using sensor-based systems in assessing functional health, and they also reveal the critical need for refined methodologies and patient-centric designs to improve accuracy and applicability in real-world settings.

3. Gaps in the Literature

From the literature review in the previous section, few studies have focused on objectively measuring FD in OAs using sensors. Most of the studies available in the literature focus on collecting sensor data to assess or monitor cognitive decline rather than physical/functional decline. Additionally, previous research using sensors to assess SI have generally used only one sensor or modality to collect data from participants to track and monitor their activities of daily living. The single modality approach leads to one-dimensional (view of) data, which may not be sufficiently representative to assess SI and FD. To comprehensively assess SI and FD, a more holistic approach is required that can capture different types of data based on mobility, physiological indicators, indoor and outdoor motion, sleep, and social behaviors [17]. A suggested approach [17] is using multiple or multimodal sensors for collecting objective data from participants to comprehensively monitor a multitude of daily routines and behaviors. This multimodal sensor approach ensures more reliable data collection, as data are gathered from multiple sensors, reducing the risk of data loss due to sensor malfunction. Further, it can provide complementary information from different sensors, enhancing our understanding of SI and the functional recovery of OAs, and facilitating the development of generalizable predictive models in the future.

Most studies have collected sensor data over a short duration, typically around one week, to monitor patients for SI [17]. This duration may be insufficient, as SI does not change drastically over time; it is not an instant event (i.e., a fall) [17]. Long-term studies spanning several months are more beneficial, as SI develops gradually. To enhance the accuracy of assessments and predictive algorithms, various types of objective data can be collected, including social behaviors, sleep metrics, mobility, indoor motion, GPS data, step count, and physiological data (e.g., heart rate). The aforementioned data types are aligned and correlated with the definitions of SI and FD and capture patients’ indoor and outdoor behaviors, sleep patterns, and physiological and mobility indicators.

The aim of this current study was to investigate the correlations between SI and FD using various clinical and sensor data, including sleep patterns, physiological indicators, mobility, and social interactions. Collecting data from various sensing modalities is crucial for understanding the complex behavioral, physiological, and social dynamics of OAs recovering from lower limb fractures in the community. To address this need, our team employed MAISON (Multimodal AI-based Sensor Platform for Older iNdividuals), a multimodal sensor system designed to collect data from multiple modalities [29]. The multimodal data are encrypted and stored in a private cloud [30]. This work is the foundational step to inform a clinically validated system to evaluate and assess OAs’ social and physical mobility activities.

4. Methods

In this longitudinal observational study, eight OAs were recruited who were admitted to the inpatient rehabilitation at Toronto Rehabilitation Institute (TRI), University Health Network (UHN), with lower limb fractures and subsequently discharged to go home. The data collection period for each participant was 8 weeks once they returned home. The study’s duration was based on prior research indicating that the first 1 to 3 months post-discharge are crucial for the overall healing trajectory of OAs after a lower limb fracture [31]. A literature review on complications in patients with postoperative hip fractures found that the first month after discharge is particularly vulnerable, posing high risks for 18 health complications including infections, depression, cardiac failure, and hospital readmission [31]. The review also noted that 5 complications remain likely for up to 3 months post-discharge [31]. In another study, symptoms of depression and loneliness were exhibited immediately and 6 weeks after surgery [32]. Therefore, an 8-week period immediately following discharge was chosen to effectively capture these significant changes in health and function.

The inclusion criteria for the study were as follows: (a) 50 years old or older; (b) had a surgically-repaired hip, femur, or pelvis fracture, or a hip replacement surgery; (c) speaks English and can provide consent; (d) lives within a 60-minute commute from TRI-UHN; (e) lives alone (both with or without pets); (f) has Wi-Fi; (g) does not have cognitive impairment (participant has Mini Mental State Examination (MMSE) score of 24 or more); (h) should not have any physician orders in the medical chart restricting hip flexion greater than 90 degrees. Patient recruitment was conducted through UHN Central Recruitment, where the patients were screened for eligibility and consent was obtained. Patient recruitment and enrollment started prior to the discharge date of the patients; therefore, the data collection started immediately or shortly after the discharge date. This study received ethical approval from the UHN Research Ethics Board #20-5113.

4.1. Clinical Data Collection

Table 1. Data collection timeline.

Task	Baseline	Week 2	Week 4	Week 6	Week 8
Inpatient physical training (20–30 min)	✓
Sensor Installation (1 h)	✓
Demographic Data	✓
Interviews on MS Teams (1 h)	✓				✓
Questionnaire completion on MS Teams (15 min)		✓	✓	✓	✓
Physical Tests on MS Teams (15 min)		✓	✓	✓	✓
Ongoing data collection with sensors (location, motion, heart rate, sleep)		✓	✓	✓	✓
Sensor take down (1 h)					✓

lists the clinical data (two physical tests and three questionnaires) that were collected through biweekly video calls using the Microsoft Teams application. The video calls were recorded and conducted biweekly, and the interviews were audio recorded during the initial sensor installation and final sensor takedown.

The biweekly assessments were scheduled based on participant’s availability, resulting in a mix of morning and afternoon sessions. Research indicates that circadian rhythms can indeed influence physical performance and cognitive function, for example, [33]. Similarly, cognitive performance and alertness can vary throughout the day [34]. To mitigate these circadian effects, we ensured that assessments were consistently scheduled at similar times for each participant throughout the study. This approach aimed to reduce intra-individual variability related to time-of-day effects.

Further, demographic data (e.g., age, gender, type of fracture, employment status, etc.) were collected at baseline during sensor installation.

The clinical assessments conducted in this study are part of standard care in lower limb fracture rehabilitation. The questionnaires include SIS [19], Oxford Hip Score [35], and Oxford Knee Score [36]. The physical tests are Timed Up and Go Test (TUG) [37] and the 30 s chair stand [38]. The Oxford Hip Score, the Oxford Knee Score, and the physical tests were chosen to evaluate functional mobility in OAs. The results from these assessments were compared to the sensor data collected to determine correlations between the functional recovery of the patients and their SI as they recovered from their lower limb fractures.

4.1.1. Social Isolation Scale

The SIS questionnaire consists of six multiple-choice questions, three of which are focused on the number of friends the individual has and overall social connectedness, and the other three are focused on their sense of belonging and contentment [19]. Each question is labeled with a score, and the sum of all of them out of 30 is the SI score. The total SIS score ranges from 6–30 with higher values indicating lower levels of SI.

4.1.2. Oxford Hip Score and Oxford Knee Score

These two questionnaires are designed to assess functional mobility in the hip and the knee. The two questionnaires consist of 12 multiple-choice questions each, focusing on the patient’s recovery for and any pain experienced during activities they are able to perform independently, such as doing the household shopping, walking down a flight of stairs, and taking a shower. The score ranges for each are from 0 to 48, with the higher value indicating higher physical health or functional ability in the hip and/or the knee. The score for each item on these scales ranges from 1 to 5, with the higher value indicating lower physical health in the hip/knee. The overall score is calculated by the sum of the score from each item in the questionnaire, subtracted from 60.

4.1.3. Timed up and Go Test

This is a physical test measuring basic mobility movements [37]. The TUG is appropriate for following clinical changes in the patient over time [37]. In this test, the OA is asked to get up from a chair, walk three meters using an aid, if required, and then walk back three meters and sit back down in the chair. The time taken to complete this exercise is recorded as the score for the TUG test.

4.1.4. 30 s Chair Stand

This physical test is a functional evaluation to measure the lower body strength [38]. This test is of high clinical importance since it is related to some of the demanding daily life activities, such as climbing stairs or getting out of a chair [38]. In this test, the patient is asked to fully get up from a chair while having their arms crossed and is asked to sit back down continuously. The number of times the patient is able to perform this exercise in 30 s is the score for the 30 s chair stand.

4.2. Sensor Data Collected Using MAISON

The MAISON system [29] consists of four main components: (1) a smartphone application, (2) a smartwatch application, (3) wearable and non-wearable sensors along with a provision to collect data directly from the device or their associated clouds, and (4) a central cloud for data storage. These components can be seen in Figure 1. In this study, the MAISON was utilized to collect multimodal sensor data from eight participants ($N=8$). More specifically, MAISON [29] consists of four smart devices including an Android smartphone with a MAISON app, a smartwatch with a MAISON app, a motion sensor, and a sleep-tracking mattress sensor. The Android smartwatch is worn by the participant throughout the day, and they take the smartphone with them when they leave their homes to collect GPS data. The sleep-tracking mattress sensor is positioned under the mattress where the participant sleeps. During our intake survey, we identified where the participant generally sleeps on their bed and placed the mattress sensor underneath. The motion sensor was put in the living room to capture patterns of motion throughout the day. For participants who had pets, we strategically placed sensors to minimize interference from pets. For instance, motion sensors were positioned at heights and locations less likely to capture pet movements and the participants’ pets did not sleep with them on the bed.

MAISON’s sensors collect the following data: raw acceleration, heart rate, number of steps, GPS, walking speed, sleep related metrics, and indoor motion.

Table 2. Summary of devices and data modalities collected from each.

Device	Data
Smartphone	GPS
Smart Watch	Raw Acceleration
	Heart rate
	Step count
Motion Sensor	Frequency of motion
Sleep Sensor	Sleep metrics (i.e., sleep duration, wake up count, heart rate during sleep, etc.)

summarizes the types of data collected from each smart device.

The smartphone application collects GPS data at a frequency of $0.033$ Hz (one observation in 30 s). The smartwatch application collects heart rate, step count, and raw acceleration data while it is worn by the patient. The accelerometer data on the watch are collected at a frequency of 1 Hz, and the heart rate data are collected every 30 min for 30 s. The motion sensors collect the frequency and pattern of indoor movement in the living room, and lastly, the Withings sleep-tracking mattress sensor [39] collects sleep-related metrics. Several papers have reported scientific evaluations of the Withings sleep sensor. Kainec et al. [40] reported that total sleep time was accurately estimated by the Withings sleep sensor within clinically acceptable ranges. Another study by Mantua et al. [41] found a strong correlation between the total sleep time and Polysonomography (PSG) for the Withings sleep sensor. The sampling frequency for each of the devices was chosen based on the literature, feedback from clinicians, and convenience for the participant as considerations about data collection and battery life. For example, clinicians were consulted regarding the frequency and duration of the collection of heart rate data, and the frequency of acceleration data were based on the available literature. Among the different data modalities collected by MAISON, one modality is available on a per-day basis (e.g., sleep metrics), three are event based (e.g., indoor motion, GPS, number of steps), and two are collected on a continuous basis (e.g., raw acceleration, heart rate).

4.3. Feature Extraction

From the collected sensor data, different types of features can be extracted and analyzed. One category of features is statistical features, which illustrate patterns and trends about the data collected. Some of the statistical features include average step count, total step count, average heart rate during the day and during sleep, total sleep duration, number of motion events, and average acceleration. For acceleration data, a python library called FLIRT [42] was used to extract additional general time domain features from acceleration data. FLIRT [42] uses a function where it takes the x, y, and z components of raw acceleration data as input and the output is 85 features, demonstrating various metrics such as mean, minimum, maximum, skewness, kurtosis, etc., for $x,y,z$ and the $l2$ norm. Out of the 85 features, only the features for the $l2$ norm are kept for further analysis since the individual x, y, and z components do not reveal any useful data for the correlation analysis. The GPS data collected from the smartphone can provide insight into the patients’ outdoor movements, the amount of time spent outside, and the distance traveled. For sleep-related features, the start date of the sleep was considered for the sleep episode (the night after). A total of 53 features extracted from eight participants can be seen in

Table 3. Feature dictionary (ACC = Acceleration).

Data Modality	Feature	Description	Function Used for Biweekly FE
Step	step-sum	Total number of steps per day	sum
	step-ratio	The ratio of the number of hours with step to the number of hours without step per day	mean
	step-mean	Average number of steps in hours of a day with at least one step	mean
	step-max	Maximum number of steps in hours of a day with at least one step	max
	step-max-timestamp	The timestamp (hour) in which there has been a maximum number of steps a day	mean
Heart rate	heart rate-min	Minimum heart rate in a day	min
	heart rate-max	Maximum heart rate in a day	max
	heart rate-mean	Mean of heart rate in a day	mean
	heart rate-std	Standard deviation of heart rate in a day	mean
Acceleration	acceleration-sum	Total number of acceleration data per day	sum
	$l2$-mean	Mean of the ACC signal (g)	mean
	$l2$-std	Standard deviation of the ACC signal (g)	mean
	$l2$-min/$l2$-max	Minimum and maximum of the ACC signal (g)	mean
	$l2$-ptp	Range (peak to peak) of the ACC signal (g)	mean
	$l2$-sum	Sum of the ACC signal (g)	mean
	$l2$-energy	Energy of the ACC signal ($g^2$)	mean
	$l2$-skewness	Skewness of the ACC signal (g)	mean
	$l2$-kurtosis	Kurtosis of the ACC signal	mean
	$l2$-peaks	Number of the ACC signal	mean
	$l2$-rms	Root mean square of the ACC signal (g)	mean
	$l2$-lineintegral	Integral under the ACC signal	mean
	$l2$-n-above-mean	Number of the ACC signal above the mean	mean
	$l2$-n-below-mean	Number of the ACC signal below the mean	mean
	$l2$-n-sign-changes	Number of changes in the ACC signal slope	mean
	$l2$-iqr	Interquartile range between the 25th and 75th percentiles of the ACC signal (g)	mean
	$l2$-iqr-5-95	Interquartile range between the 5th and 95th percentiles of the ACC signal (g)	mean
	$l2$-pct-5	5th percentile of the ACC signal	mean
	$l2$-pct-95	95th percentile of the ACC signal	mean
	$l2$-entropy	Entropy of the ACC signal	mean
	$l2$-perm-entropy	Permutation entropy of the ACC signal	mean
	$l2$-svd-entropy	Singular value decomposition of the ACC signal	mean
Position	position-count	Total number of position data per day	sum
	position-duration	The duration (in minutes) of being outside of the home in a day	sum
	position-maximum-distance	The maximum distance traveled from home in a day	max
	Time spent outside (mins)	Amount of time spent outside of the home in a day	sum
	Distance travelled from home (km)	Distance traveled from the home in a day	sum
Motion	motion-sum	Total number of motions per day	sum
	motion-ratio	The ratio of the number of hours with motion to the number of hours without motion per day	mean
	motion-mean	Average number of motions in hours of a day with at least one motion	mean
	motion-max	Maximum number of motions in hours of a day with at least one motion	max
	motion-max-timestamp	The timestamp (hour) in which there has been a maximum number of motions a day	mean
Sleep	totalsleepduration	Total sleep duration (hours)	sum
	deepsleepduration	Deep sleep duration (hours)	sum
	lightsleepduration	Light sleep duration (hours)	sum
	remsleepduration	REM sleep duration (hours)	sum
	snoring	Snoring duration (hours)	sum
	durationtosleep	Duration to sleep (hours)	mean
	durationtowakeup	Duration to wake up (hours)	mean
	wakeupcount	Wake up count (count)	sum
	hr-average	Mean of heart rate during sleep	mean
	hr-min	Minimum heart rate during sleep	min
	hr-max	Maximum heart rate during sleep	max

, categorized into data modalities.

The features were extracted from the sensor data on a daily or event-based basis. However, the clinical data were collected every two weeks. Therefore, two different time resolutions were used for the extracted features: daily and biweekly. To obtain the biweekly features, the daily features extracted from the sensor data were processed in various ways. For example, the summation of position-count daily feature (described in Table 3) was used as the biweekly position-count feature. Various functions were used to compute the biweekly features from the daily features, such as $sum$, $mean$, $maximum$, and $minimum$. The last column in Table 3 presents the function used for biweekly feature extraction (FE) on each daily feature. Several algorithms were created in Python for preprocessing and feature extraction purposes for daily and biweekly time scales.

5. Results and Analysis

5.1. Demographic Data

A total of eight ($N=8$) participants were recruited from January 2022 to April 2024. The demographic data for the participants are shown in

Table 4. Demographic data ($N=8$).

Category	Variable	Mean (SD) or (%)
Age	Mean (SD)	$72.75$ ($6.91$)
	60–70	5 ($62.5$%)
	70–80	1 ($12.5$%)
	80–90	2 (25%)
Gender	Male	1 ($12.5$%)
	Female	7 ($87.5$%)
Type of fracture	Hip	4 (50%)
	Pelvis	2 (25%)
	Femur	1 ($12.5$%)
	Hip Replacement	1 ($12.5$%)
Ethnicity	Caucasian	7 ($87.5$%)
	African-Canadian	1 ($12.5$%)
Relationship Status	Single	3 ($37.5$%)
	Married	1 ($12.5$%)
	Widowed	1 ($12.5$%)
	Separated	1 ($12.5$%)
	Divorced	2 (25%)
Employment Status	Unemployed	1 ($12.5$%)
	Employed	3 ($37.5$%)
	Retired	2 (25%)
	Other	2 (25%)

. The mean age of the participants was $72.75$ years, ranging from 60 to 84 with a median of $74.5$ years. Among the participants, $12.5$% were male, $87.5$% were Caucasian, and $12.5$% were African Canadian. Out of 8 of them, 4 (50%) had a hip fracture, 1 ($12.5$%) had a hip replacement, 2 (25%) had a broken pelvis, and 1 ($12.5$%) had a femur fracture. Two out of the three participants who were employed were not working during the data collection period, and one was working part-time.

5.2. Clinical Data

As described in Section 4.1, a total of five tests and questionnaires were conducted during the biweekly video calls. The summary of the scores are presented in

Table 5. Clinical data collected from 8 participants during biweekly video calls.

Timeline	Variable	Mean (SD)
Week 2	TUG (seconds)	$18.99$ ($9.78$)
	30 s chair stand	$8.63$ ($3.60$)
	SIS	25 ($2.87$)
	Oxford Hip Score	$32.88$ ($7.88$)
	Oxford Knee Score	$36.63$ ($7.47$)
Week 4	TUG (seconds)	$19.63$ ($9.96$)
	30 s chair stand	$10.86$ ($2.17$)
	SIS	$24.13$ ($2.80$)
	Oxford Hip Score	$31.86$ ($8.44$)
	Oxford Knee Score	$31.71$ ($11.10$)
Week 6	TUG (seconds)	$18.64$ ($10.13$)
	30 s chair stand	$11.13$ ($2.80$)
	SIS	$25.38$ ($2.60$)
	Oxford Hip Score	$34.71$ ($7.69$)
	Oxford Knee Score	$36.67$ ($8.33$)
Week 8	TUG (seconds)	$17.2$ ($7.90$)
	30 s chair stand	$12.13$ ($2.93$)
	SIS	$23.88$ ($3.55$)
	Oxford Hip Score	$33.14$ ($8.18$)
	Oxford Knee Score	$34.71$ ($8.48$)

.

5.3. Sensor Data

The features listed in Table 3 extracted from the multimodal sensor data are presented in this section.

A selection of the features (seven out of 53) extracted from the sensor data are presented in Figure 2 and Figure 3. They were selected as the fundamental features in order to represent the different groups of participants: those who were affected only by hip, only by pelvis, only by femur, and all participants. From Figure 2 and Figure 3, it can be seen that the highest variation in the sensor data was among the group of participants with an injured hip. Figure 2c and Figure 3b show that while the average magnitude of acceleration ($l2$-mean) and total motion events (motion-sum) were decreasing for participants with an injured hip, they remained more or less consistent for the rest of the participants. Figure 2a and Figure 3a also show variations for total step count (increasing) and distance traveled from home (decreasing) for the pelvis fracture group, while they had minimal changes for the rest of the participants. We cover the results from all the participants (lower plot) in this paper.

5.4. Correlation Analyses

Correlation analyses were completed for the eight participants comprising two categories of daily and biweekly time resolutions, which were computed from daily and biweekly features. We recognize that every participant’s willingness to socialize and personal biases as well as physiological parameters could be different. Therefore, when the data were imputed for the days of missing data, the imputation was performed for every participant separately in order to account for personal differences and biases.

Based on the data types from the features extracted (continuous and ordinal), Spearman’s correlation coefficient ($\rho$) was used [43] to determine the correlation between the features extracted from the sensor data and SI and FD in OAs following lower limb fractures. The Spearman’s correlation coefficient investigates a linear relationship between two variables [44]. The correlation values range from $-1$ (linear negative relationship) to $+1$ (linear positive relationship). The closer the correlation coefficient is to zero, the smaller the degree of its linear correlation [44]. A linear correlation means that when there is a relationship between the two variables and one changes, it will lead to a change in the other variable, demonstrated by their correlation value. A positive correlation means that both variables increase or decrease together. A negative correlation means that when one variable increases, the other decreases, and vice versa. The following guidelines are considered appropriate [43] for evaluating the degree of correlation for studies involving biomedical, biological, sociological, and psychological data. In this paper, the strong and moderate correlations are evaluated.

$\left|\rho \right|<0.3\to$ Weak relationship

$0.3\le \left|\rho \right|\le 0.5\to$ Moderate relationship

$\left|\rho \right|>0.5\to$ Strong relationship

5.4.1. Daily Correlation Analysis

In order to evaluate the correlation between the daily features extracted from sensor data and clinical data, both need to be available in a daily format. The clinical data were collected on a biweekly basis; however, for the purposes of this analysis, they were repeated for every day of the two-week period. For example, if the SIS score collected on week 2 is 25, this score is applied to every day included in the first two weeks of data collection. These metrics were all saved in a csv file. The resulting file containing the daily sensor and clinical features consisted of 90 columns (53 features (5 position features, 5 motion features, 4 heart rate features, 11 sleep features, 5 step features, 23 acceleration features), 9 SIS measurements, 13 Oxford Hip Scores, 13 Oxford Knee Scores, and 2 physical tests features). Next, the correlation analysis was performed following the steps below:

• Four different daily-feature csv files were created for each type of clinical data, since the types of data missing for each clinical data were different. For example, in the daily feature file with respect to TUG, the days where TUG was missing were removed. The four resulting daily feature files consisted of the following: (1) daily features with respect to SIS, (2) daily features with respect to TUG, (3) daily features with respect to 30 s chair stand, (4) daily features with respect to Oxford Hip Score and Oxford Knee Score.
• Using Python programming language and the Visual Studio Code IDE, the Spearman’s correlation was calculated for every column of the daily features. In this step, any correlations with a p value less than $0.05$ (statistically significant) were kept, and the rest were marked as “0” in the correlation matrix.
• For each resulting correlation matrix generated in step 2, a second round of analysis was conducted in order to remove redundant features. In this step, every two features with an absolute correlation value higher than $0.90$ were deemed as “redundant”, and one of the two features was removed from the correlation matrix.

The resulting daily correlation values after removing the redundancies are presented in this section. The Oxford Hip Score, the Oxford Knee Score, and the two physical tests (TUG and 30 s chair stand) represent the level of functional recovery in this population.

The results of the strong correlations in

Table 6. Daily strong correlations between SI, FD, and sensor data for 8 participants.

Variable 1	Variable 2	Correlation Value
Oxford Hip Score—Q11	hr-average	$0.73$
Oxford Knee Score—Q11	hr-average	$0.66$
Oxford Knee Score—Q10	motion-sum	$0.63$
Oxford Knee Score—Q9	hr-average	$0.59$
Oxford Knee Score—Q5	motion-sum	$0.58$
Oxford Knee Score—Q4	hr-average	$0.56$
Oxford Knee Score	hr-average	$0.54$
SIS—Q3	hr-average	$0.51$
Oxford Hip Score—Q4	motion-sum	$-0.53$
Oxford Hip Score—Q1	motion-sum	$-0.53$
TUG	totalsleepduration	$-0.54$
Oxford Hip Score—Q11	motion-ratio	$-0.60$

The different items/questions in the Oxford Hip Score and Oxford Knee Score are illustrated in Appendix A in this paper.

indicate that the various items of the Oxford Knee Score and Oxford Hip score, which assess functional physical recovery, had a strong positive correlations with average heart rate during sleep and total motion events. In contrast, only one item on the SIS exhibited a strong correlation with sensor data; however, several moderate correlations between SIS and sensor data were found and are presented in

Table 7. Daily moderate correlations between SI and sensor data for 8 participants.

Variable 1	Variable 2	Correlation Value
SIS (Connectedness)	hr-average	$0.46$
SIS (Belonging)	step-sum	$0.45$
SIS	hr-max	$0.43$
SIS—Q6	step-sum	$0.42$
SIS	heart rate-mean	$0.39$
SIS (Belonging)	lightsleepduration	$0.37$
SIS—Q6	durationtosleep	$0.36$
SIS	hr-average	$0.36$
SIS	heart rate-min	$0.36$
SIS—Q2	heart rate-mean	$0.33$
SIS—Q6	totalsleepduration	$0.33$
SIS—Q4	lightsleepduration	$0.32$
SIS—Q5	motion-ratio	$0.31$
SIS (Connectedness)	snoring	$0.30$
SIS (Connectedness)	totalsleepduration	$-0.31$
SIS (Connectedness)	step-ratio	$-0.33$
SIS (Belonging)	snoring	$-0.37$
SIS—Q6	snoring	$-0.46$

The different items/questions in the SIS are illustrated in Appendix A in this paper.

. The strong positive correlations between the various items of the Oxford scores mean that the higher the participant’s score was on the questions on the Oxford Hip Score (Question 11) and Oxford Knee Score (Question $4,5,9,10$, and 11), the higher their average heart rate during sleep and the higher their motion count during the day. Conversely, the strong negative correlations between the various items of the Oxford scores mean that the higher the participant’s score was on the questions on the Oxford Hip Score (Question $1,4$, and 11), the lower their motion ratio and total motion events were. There also existed a strong negative correlation between the TUG and the total sleep duration, indicating that the longer it takes the participants to complete the TUG test, the shorter their sleep duration is that day/night.

The results of moderate correlations between SI and the sensor data are listed in Table 7. Various individual questions of the SIS are moderately positively correlated with average heart rate during the day and during sleep. Notably, the SIS questions related to the number of interactions are positively correlated with average heart rate during sleep and negatively correlated with total sleep duration. However, the SIS questions related to the sense of belonging are positively correlated with total step count and light sleep duration, while negatively correlated with snoring time.

The results of moderate correlations between FD and the sensor data are listed in

Table 8. Daily moderate correlations between Oxford Hip Score and sensor data for 8 participants.

Variable 1	Variable 2	Correlation Value
Oxford Hip Score—Q11	heart rate-mean	$0.50$
Oxford Hip Score—Q11	heart rate-min	$0.49$
Oxford Hip Score—Q12	hr-average	$0.48$
Oxford Hip Score—Q11	hr-max	$0.48$
Oxford Hip Score—Q12	heart rate-mean	$0.45$
Oxford Hip Score—Q12	step-mean	$0.42$
Oxford Hip Score—Q10	step-sum	$0.42$
Oxford Hip Score—Q6	hr-max	$0.42$
Oxford Hip Score—Q4	deepsleepduration	$0.38$
Oxford Hip Score—Q10	heart rate-min	$0.37$
Oxford Hip Score—Q3	lightsleepduration	$0.37$
Oxford Hip Score—Q6	heart rate-max	$0.36$
Oxford Hip Score	heart rate-mean	$0.35$
Oxford Hip Score	step-mean	$0.34$
Oxford Hip Score	heart rate-min	$0.34$
Oxford Hip Score—Q5	step-mean	$0.34$
Oxford Hip Score—Q1	deepsleepduration	$0.34$
Oxford Hip Score—Q3	motion-ratio	$0.33$
Oxford Hip Score—Q7	motion-sum	$0.30$
Oxford Hip Score—Q4	hr-max	$-0.31$
Oxford Hip Score—Q12	motion-ratio	$-0.33$
Oxford Hip Score—Q4	hr-average	$-0.35$

The different items/questions in the Oxford Hip Score are illustrated in Appendix A in this paper.

,

Table 9. Daily moderate correlations between Oxford Knee Score and sensor data for 8 participants.

Variable 1	Variable 2	Correlation Value
Oxford Knee Score—Q10	acceleration-sum	$0.50$
Oxford Knee Score—Q12	hr-max	$0.48$
Oxford Knee Score—Q10	$l2$-mean	$0.48$
Oxford Knee Score—Q12	hr-average	$0.46$
Oxford Knee Score—Q11	heart rate-min	$0.46$
Oxford Knee Score—Q7	heart rate-mean	$0.45$
Oxford Knee Score—Q4	motion-sum	$0.44$
Oxford Knee Score—Q10	lightsleepduration	$0.43$
Oxford Knee Score	hr-max	$0.39$
Oxford Knee Score—Q12	step-sum	$0.38$
Oxford Knee Score—Q10	$l2$-peaks	$0.38$
Oxford Knee Score—Q12	heart rate-max	$0.37$
Oxford Knee Score—Q7	$l2$-svd-entropy	$0.36$
Oxford Knee Score—Q2	hr-max	$0.36$
Oxford Knee Score	heart rate-mean	$0.33$
Oxford Knee Score—Q10	$l2$-std	$0.32$
Oxford Knee Score—Q10	step-ratio	$0.32$
Oxford Knee Score—Q10	$l2$-n-sign-changes	$0.32$
Oxford Knee Score—Q7	step-mean	$0.32$
Oxford Knee Score—Q4	wakeupcount	$0.30$
Oxford Knee Score—Q3	acceleration-sum	$0.30$
Oxford Knee Score—Q8	lightsleepduration	$0.30$
Oxford Knee Score—Q7	motion-sum	$-0.30$
Oxford Knee Score—Q3	$l2$-svd-entropy	$-0.32$
Oxford Knee Score—Q10	snoring	$-0.32$
Oxford Knee Score—Q9	motion-ratio	$-0.38$
Oxford Knee Score	motion-ratio	$-0.44$
Oxford Knee Score—Q7	motion-ratio	$-0.49$
Oxford Knee Score—Q11	motion-ratio	$-0.50$

The different items/questions in the Oxford Knee Score are illustrated in Appendix A in this paper.

and

Table 10. Daily moderate correlations between physical tests and sensor data for 8 participants.

Variable 1	Variable 2	Correlation Value
30 s chair stand	hr-average	$0.36$
TUG	snoring	$0.35$
TUG	motion-sum	$0.35$
TUG	wakeupcount	$-0.31$
TUG	durationtosleep	$-0.31$
TUG	lightsleepduration	$-0.33$
TUG	step-sum	$-0.36$
30 s chair stand	snoring	$-0.38$
30 s chair stand	motion-ratio	$-0.41$

. Multiple moderate correlations exist between the Oxford Hip Score questions, Oxford Knee Score questions, and heart rate, motion, acceleration, and sleep features. These findings indicate that physical health assessments, such as the Oxford scores, are moderately associated with various measurable aspects of physical activity, mobility, and sleep. There is also a moderate negative correlation between TUG and light sleep duration ($-0.33$) and the wake up count in a day, indicating that the longer the patient takes to complete the TUG test (lower physical health/higher FD), the shorter their light sleep duration is at night and the less often they wake up during their sleep episode. A negative correlation between TUG and wake up count here indicates that individuals with better mobility (lower TUG times) tend to wake up fewer times during the night. This could be because better mobility is often associated with better overall physical health, which might contribute to more uninterrupted sleep. The negative correlation between TUG and duration of sleep ($-0.33$) could suggest that individuals who are more mobile (indicated by lower TUG times) tend to fall asleep faster. This might be due to higher levels of physical activity contributing to better sleep efficiency and faster sleep onset.

5.4.2. Biweekly Correlation Analysis

To evaluate the correlation between the biweekly features from sensor data and clinical data, both need to be available in a biweekly format. Combining the biweekly features from the sensor data and the clinical data collected on a biweekly basis, we had a resulting csv file with 90 columns (53 features (5 position features, 5 motion features, 4 heart rate features, 11 sleep features, 5 step features, 23 acceleration features), 9 SIS measurement, 13 Oxford Hip Score, 13 Oxford Knee Score, and 2 physical tests features). Next, the correlation analysis is done following the same steps as listed in Section 5.4.1, for biweekly clinical and sensor features. The resulting biweekly correlation values after removing the redundancies are presented in this section.

Table 11. Biweekly strong correlations between SI and sensor data for 8 participants.

Variable 1	Variable 2	Correlation Value
SIS—Q6	durationtosleep	$0.68$
SIS (Belonging)	step-sum	$0.64$
SIS—Q6	step-sum	$0.62$
SIS (Belonging)	lightsleepduration	$0.60$
SIS—Q3	hr-average	$0.57$
SIS—Q4	distance travelled from home	$0.54$
SIS—Q6	totalsleepduration	$0.53$
SIS (Belonging)	wakeupcount	$0.53$
SIS (Belonging)	durationtosleep	$0.51$
SIS (Belonging)	distance travelled from home	$0.51$
SIS—Q1	$l2$-pct-95	$-0.52$
SIS—Q1	durationtosleep	$-0.53$
SIS—Q6	snoring	$-0.55$
SIS (Connectedness)	$l2$-pct-95	$-0.59$

The different items/questions in the SIS are illustrated in Appendix A in this paper.

lists the strong biweekly correlations between SI and sensor data. The results from this table show that SIS questions about sense of belonging (questions 4–6) have the most number of correlations with the biweekly sensor data. The strongest positive correlation is between the duration of sleep and the participant’s perception of how much time they spend engaged in social activities (SIS, question 6). Notably, SIS questions about belonging have strong positive correlations with total step count, distance traveled from home, and several sleep metrics. This means that as the participant’s SI level decreases (SIS belonging increases), their movement, activity outside their home, and sleep duration increase. This relationship can be observed in Figure 4. Moreover, there is a strong negative correlation between duration of sleep and the number of face-to-face interactions the participant has (SIS, question 1). This indicates that as the participant’s in-person interactions increase, the amount of time it takes them to fall asleep decreases.

The results of strong correlations between Oxford Hip Score, Oxford Knee Score, and the two physical tests are displayed in

Table 12. Biweekly strong correlations between Oxford Hip Score and sensor data for 8 participants.

Variable 1	Variable 2	Correlation Value
Oxford Hip Score—Q11	hr-average	$0.85$
Oxford Hip Score—Q11	heart rate-mean	$0.63$
Oxford Hip Score—Q11	step-max	$0.58$
Oxford Hip Score—Q10	step-sum	$0.57$
Oxford Hip Score—Q4	deepsleepduration	$0.53$
Oxford Hip Score—Q4	hr-min	$0.52$
Oxford Hip Score—Q11	hr-max	$0.52$
Oxford Hip Score—Q3	motion-ratio	$0.52$
Oxford Hip Score—Q3	lightsleepduration	$0.51$
Oxford Hip Score—Q4	durationtowakeup	$-0.51$
Oxford Hip Score	$l2$-max	$-0.53$
Oxford Hip Score—Q8	$l2$-max	$-0.58$
Oxford Hip Score—Q4	motion-sum	$-0.59$
Oxford Hip Score—Q2	heart rate-std	$-0.62$
Oxford Hip Score—Q1	motion-max	$-0.65$
Oxford Hip Score—Q11	motion-ratio	$-0.73$

The different items/questions in the Oxford Hip Score are illustrated in Appendix A in this paper.

,

Table 13. Biweekly strong correlations between Oxford Knee Score and sensor data for 8 participants.

Variable 1	Variable 2	Correlation Value
Oxford Knee Score—Q5	motion-max	$0.81$
Oxford Knee Score—Q11	hr-average	$0.78$
Oxford Knee Score—Q10	distance travelled from home	$0.67$
Oxford Knee Score—Q10	$l2$-mean	$0.67$
Oxford Knee Score	hr-average	$0.67$
Oxford Knee Score—Q10	motion-sum	$0.63$
Oxford Knee Score—Q10	$l2$-peaks	$0.63$
Oxford Knee Score—Q9	hr-max	$0.62$
Oxford Knee Score—Q7	heart rate-mean	$0.62$
Oxford Knee Score	motion-max	$0.60$
Oxford Knee Score—Q10	acceleration-sum	$0.59$
Oxford Knee Score—Q10	step-ratio	$0.59$
Oxford Knee Score—Q12	step-sum	$0.53$
Oxford Knee Score—Q11	step-max	$0.52$
Oxford Knee Score—Q10	durationtosleep	$0.51$
Oxford Knee Score—Q11	durationtowakeup	$0.51$
Oxford Knee Score	motion-ratio	$-0.54$
Oxford Knee Score—Q11	motion-ratio	$-0.61$
Oxford Knee Score—Q7	motion-ratio	$-0.71$

The different items/questions in the Oxford Knee Score are illustrated in Appendix A in this paper.

and

Table 14. Biweekly strong correlations between physical tests and sensor data for 8 participants.

Variable 1	Variable 2	Correlation Value
30 s chair stand	hr-max	$0.55$
TUG	step-sum	$-0.50$
30 s chair stand	motion-ratio	$-0.52$
TUG	totalsleepduration	$-0.69$

.

Table 12 shows that question 11 (“Can you do the household shopping on your own?”) on the Oxford Hip Score has several strong correlations with the sleep, motion, and heart rate features. Notably, the strong correlation of question 11 on the Oxford Hip Score with three heart rate metrics (average heart rate during sleep, average heart rate during the day, and maximum heart rate during sleep) can indicate an overall relationship between the activities of daily living and the heart rate of the participants. The positive strong correlation between item 11 on Oxford Hip Score and heart rate metrics indicates that the higher the score for item 11 (lower physical health/functional recovery), the higher the participant’s heart rate is during the day and during sleep. This relationship can be more clearly seen in Figure 5. Moreover, there is a negative correlation between this item and the motion ratio, indicating that the lower the participant’s physical health, the less time they spend in movement (captured by a motion sensor). Further, the Oxford Hip Score has a strong negative correlation with $l2$-max, indicating that the lower the participant’s physical health (higher value on the Oxford Hip Score), the lower their maximum magnitude of acceleration is, representing less movement.

The analysis reveals a strong biweekly correlation between Oxford Knee Score and the sensor data. Specifically, Table 13 indicates that items 10 and 11 on the Oxford Knee Score have the strongest correlations with the sensor data (7 and 4, respectively). Question 10 (“Have you felt your knee might suddenly give away or let you down?”) from this questionnaire has a strong positive correlation of $0.67$ with distance traveled from home and $l2$-mean. This relationship demonstrates that the higher the score on question 10 (lower physical health), the farther the participants traveled from home and the higher their magnitude of acceleration during the day. However, the destination/sites the participants visited were not recorded; therefore, we were unable to determine the cause of the relationship between lower physical health and the distance traveled. Question 11 on this questionnaire refers to the participant’s ability to do household shopping on their own (similar to question 11 on the Oxford Hip Score) with respect to their knee, which also has a strong positive correlation with average heart rate during sleep, maximum step count, and the time it takes the participant to wake up (durationtowakeup).

Lastly, Table 14 displays the biweekly strong correlations between the physical tests and the sensor data. Strong negative correlations exist between TUG and total sleep duration and total step count. This finding shows that participants who take longer to complete the TUG, indicating poorer functional mobility, tend to have shorter sleep durations and lower step counts. The 30 s chair stand has a strong positive correlation with the maximum heart rate during sleep, emphasizing that the higher the participant’s score is on this test (better physical health), the higher their heart rate is during sleep. Moreover, the 30 s chair stand has a strong negative correlation with the motion ratio, implying that participants with better physical health/functional recovery tend to have fewer motion events at home (spending less time at home).

6. Discussion and Conclusions

The findings in this paper suggest that different dimensions of health, such as number of social interactions, sense of belonging, and physical mobility, may be moderately interconnected with physiological measurements and behaviors, although the findings are nuanced and not straightforward. The strong correlations between the questions from the Oxford Hip and Knee scores and the sensor data highlight the direct impact of physical health status on measurable daily physical activities.

The findings from the daily correlation analysis suggest that as OAs experience improvements or declines in joint function and pain, the sensors are able to capture these changes in their movement and level of activity. Comparing the results between daily and biweekly correlation analyses, it can be seen that similar correlations between the clinical data and features extracted from the sensor data were found. Looking at Table 7 and Table 11, item 6 on the SIS has a daily moderate correlation and a biweekly strong correlation with duration of sleep ($0.36$ and $0.68$, respectively). A similar occurrence can be seen between item 6 on SIS and the step-sum (total step count) with $0.42$ and $0.62$ correlation values of daily and biweekly, respectively. The repeated correlation, although with different values, gives strength to the findings in this paper. The consistency in correlation results suggests that sensor data can provide reliable insights into changes in participants’ SI levels over time. From both Table 7 and Table 11, sleep metrics have repeatedly shown correlations with the SIS metrics, marking the importance of sleep metrics in assessing SI, which is consistent with the previous literature [18,45,46].

Table 6, Table 8, Table 9, Table 12 and Table 13 show that the Oxford Knee Score was strongly correlated with the daily and biweekly features extracted from the sensor data in comparison to the Oxford Hip Score. Therefore, the Oxford Knee Score can be considered as a superior instrument for evaluating the correlation between functional recovery and sensor data when addressing various types of lower limb fractures.

The outcomes of daily and biweekly correlations between SI and sensor data are similar, both demonstrating a relationship between SIS scores and sleep data with various degrees of correlation. This similarity also exists with the findings from daily and biweekly correlations between FD and sensor data, in that they are both correlated with heart rate and motion features. However, in the biweekly correlation of FD and SI with sensor data, strong correlations exist between the clinical data and acceleration (i.e., $l2$-max, $l2$-mean) and position features (distance traveled from home), which is not found in daily correlations. This finding shows that the changes in acceleration and position features are not visible on a day-to-day basis but rather on a longer, 2-week period.

There are limitations in this research. A limited sample size from only one recruitment site may limit the findings of our study. Psychological state or pre-existing conditions of depression or anxiety disorders of the participants were not tracked and controlled in this study. Another limitation is the gender and ethnicity of the patients who participated in the study. Although the inclusion criteria were not limited to any specific ethnicity or gender, the consented population turned out to be mostly females and Caucasians. Efforts to diversify the recruitment sites for this study can mitigate this potential bias. Further, when collecting GPS data, the locations the participants traveled to were not considered a feature as this would require more inputs from the participants and risk overburdening them. One of the features that was explored and supported by the work of Goonawardene et al. [22] was the frequency of daytime napping. Day time napping was originally considered as a feature, but as participants took naps sometimes on the couch or other places other than the bed, it was difficult to accurately account for the midday naps. Another limitation of this study is that socialization data were not collected, for example, monitoring phone contacts or phone usage. For privacy reasons, we were unable to collect these data in a de-identified manner without undue burden on our OA participants who had just returned home from rehabilitation.

The missing clinical data were primarily due to technical issues with using video recordings using MS Teams, participants being physically unfit to perform assessments, and the refusal of participants to answer certain questions because some participants thought it was not applicable to them (e.g., answering questions about knee function after a pelvis fracture).

Future work will include scaling this study to include more recruitment sites, advancing diversity in sex/gender, age, and ethnicity, and investigating major changes from the current findings. The plan includes assessing the correlations with each fracture type from both daily and biweekly perspectives. The work presented in this paper is the foundation of creating a feature dictionary used to develop an algorithm for detecting SI and FD. The development of predictive models to assess SI and FD using the data presented in this paper is currently underway. Moreover, we plan to continue data collection from 20 more participants to increase the sample size, which will not only enhance the statistical power of our findings but also allow for a more robust and comprehensive analysis of the correlations between SI, FD, and sensor data. We are also exploring relationships between various types of patients and features by combining unsupervised learning and formal concept analysis [47]. Future research investigating the patient experience using the system and engaging in co-designing it to ensure it meets patients’ and clinicians’ needs is warranted.