Next Article in Journal
Research on Out-of-Plane Bending Test of PVB Laminated Glass Plate with Different Number of Layers
Previous Article in Journal
Long-Range Imaging LiDAR with Multiple Denoising Technologies
Previous Article in Special Issue
Inertial and Flexible Resistive Sensor Data Fusion for Wearable Breath Recognition
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Study Protocol

Protocol for the Development of Automatic Multisensory Systems to Analyze Human Activity for Functional Evaluation: Application to the EYEFUL System

by
Paula Obeso-Benítez
1,
Marta Pérez-de-Heredia-Torres
1,*,
Elisabet Huertas-Hoyas
1,
Patricia Sánchez-Herrera-Baeza
1,
Nuria Máximo-Bocanegra
1,
Sergio Serrada-Tejeda
1,
Marta Marron-Romera
2,
Javier Macias-Guarasa
2,
Cristina Losada-Gutierrez
2,
Sira E. Palazuelos-Cagigas
2,
Jose L. Martin-Sanchez
2 and
Rosa M. Martínez-Piédrola
1
1
Department of Physical Therapy, Occupational Therapy, Rehabilitation and Physical Medicine, Rey Juan Carlos University, Avenida de Atenas s/n., Alcorcón, 28922 Madrid, Spain
2
Universidad de Alcalá, Department of Electronics, Edificio Politécnico, Ctra. Madrid-Barcelona, Km 33,600, Alcalá de Henares, 28805 Madrid, Spain
*
Author to whom correspondence should be addressed.
Appl. Sci. 2024, 14(8), 3415; https://doi.org/10.3390/app14083415
Submission received: 4 March 2024 / Revised: 8 April 2024 / Accepted: 16 April 2024 / Published: 18 April 2024
(This article belongs to the Special Issue Human Activity Recognition (HAR) in Healthcare, 2nd Edition)

Abstract

:

Featured Application

This study protocol is relevant as it serves as a guideline for describing the steps to follow for the creation of a multisensory system that is capable of automatically analyzing human activity and determining their functional ability and state of dependence. It also contains valuable information on the clinical validation process of the system.

Abstract

The EYEFUL system represents a pioneering initiative designed to leverage multisensory systems for the automatic evaluation of functional ability and determination of dependency status in people performing activities of daily living. This interdisciplinary effort, bridging the gap between engineering and health sciences, aims to overcome the limitations of current evaluation tools, which often lack objectivity and fail to capture the full range of functional capacity. Until now, it has been derived from subjective reports and observational methods. By integrating wearable sensors and environmental technologies, EYEFUL offers an innovative approach to quantitatively assess an individual’s ability to perform activities of daily living, providing a more accurate and unbiased evaluation of functionality and personal independence. This paper describes the protocol planned for the development of the EYEFUL system, from the initial design of the methodology to the deployment of multisensory systems and the subsequent clinical validation process. The implications of this research are far-reaching, offering the potential to improve clinical evaluations of functional ability and ultimately improve the quality of life of people with varying levels of dependency. With its emphasis on technological innovation and interdisciplinary collaboration, the EYEFUL system sets a new standard for objective evaluation, highlighting the critical role of advanced screening technologies in addressing the challenges of modern healthcare. We expect that the publication of the protocol will help similar initiatives by providing a structured approach and rigorous validation process.

1. Introduction

Functionality is a person’s ability to independently perform activities of daily living (ADLs) [1]. ADLs can be classified into basic activities of daily living (BADLs), instrumental activities of daily living (IADLs), and advanced activities of daily living (AADLs) [2,3]. BADLs are characterized by being related to basic human needs that are common to all populations and are linked to survival. IADLs are culturally biased, are linked to the environment, require greater cognitive and motor complexity, and involve interaction with the environment [4]. Furthermore, AADLs involve tasks related to the performance of social occupations such as leisure, community, religious, and work activities [3]. According to the conceptual framework of rehabilitation, dependency is defined as the permanent state of persons who, for reasons derived from age, illness, or disability, and linked to the lack or loss of physical, mental, intellectual, or sensory autonomy, require the attention of another person or persons or significant help to perform their basic activities of daily living or, in the case of persons with intellectual disabilities or mental illness, require other support for their personal autonomy [5,6,7]. Depending on each country, people with dependency receive a different level of social and/or economic support [7,8]. In Europe, the receipt for support for dependency in the population over 65 years of age is, e.g., 11.3% in Spain, 23.4% in Switzerland, and 18.4% in Germany [9].
For the assessment of ability and dependence, several methods and tools exist [10]; however, none have been considered as the gold standard [5,11]. These tools can be observational [12], self-administered, or hetero-administered instruments [13], or even involve interviews with family members and/or caregivers [14]. The scientific literature considers that the existing instruments for assessing functional capacities present practical limitations; since the information they detail is not related to the quality of performance [14], they require a long period of time for their administration and implicitly produce different biases [5,13]. Certain biases exist due to subjectivity [11,15], gender and cultural biases [16], evaluator bias [17], and respondent bias [5,13]. In the case of personal interviews, a reliable source of information is required [18,19]; self-administered instruments are subject to recall bias [17,18,20] and observational instruments may be adversely affected by observer interventions or biases [15].
Technology can lead to improvements in the assessment of functional capacity through the monitoring of activities and the behaviors required [5]. In addition, technology facilitates the development of more effective assessments [15,21] by enabling objective quantification [15], and can even contribute to the early detection of changes in functionality [5]. The emergence of wearable devices in recent years has meant the inclusion of a new powerful and flexible tool for monitoring, storing, and transferring different clinical measurements of the human body: movement, temperature, blood pressure, sleep pattern, location, etc. [22]. Data from wearables can be used in isolation or together with other sensors [23] and are a valid method to characterize activity parameters [24] such as frequency, intensity, or duration [21].
Most of the current technological methods of functional assessment only focus on one functional component [5,25] and not on the functionality, as this requires the integration of several tools that include more than one domain. Among the works related to the recognition of human actions available in the literature [26,27], the protocol by Morgan C et al. [28] seeks to assess the symptoms of Parkinson’s disease in an everyday environment, using the Sensor Platform for Healthcare in a Residential Environment (SPHERE) technology. By analyzing the metrics obtained by the sensors, its objective is to infer clinical and functional data on Parkinson’s disease. Our current objectives seek to create and clinically validate a multisensory technological system capable of automatically assessing a subject’s level of dependence when performing activities of daily living in a controlled environment that reflects an everyday home. The value of the assessments performed in real environments [5,17,24,29] is that these can eliminate biases [18] and provide a more representative measure of the person’s real capacity [24,29]. In addition, intelligent systems have been shown to reduce the socioeconomic impact of dependent people [21], as they can be more accurate, objective, avoid biases, and can measure several parameters at the same time, which translates into a cost-efficient assessment of the person’s health status [24].
The protocol presented in this paper corresponds to a guide for the development of an automatic multisensory system to analyze human activity for functional evaluation, specifically the EYEFUL system. This protocol aims to set out the sequences and actions necessary to develop a final system of these characteristics so that it can be applied to other initiatives in which human functional evaluations by automatic technological systems are required. The operational objectives for the development and clinical validation of systems similar to EYEFUL are as follows:
  • To design and develop a multisensory system suitable for the objective and automatic assessment of functional capacity and determination of dependency status.
  • To validate the system in clinical settings, ensuring its reliability, sensitivity, and specificity in assessment of functionality during the performance of ADLs.
  • To provide a practical guide for the development of a multisensory automatic assessment system and the process to clinically validate it.
By integrating technological innovation with clinical expertise, the EYEFUL project aims to advance the field of functional evaluation and contribute to the improvement of care for people with different levels of dependency. The description of the protocol to be used aims to allow similar initiatives for automatic functional evaluations to benefit from its structured approach.

2. Protocol Description

This section describes the protocol design, outlining a structured, interdisciplinary approach to develop an automatic and objective multisensory system for the evaluation of functional ability, from the considerations of the development team to the clinical validation and dissemination of results. In the case of the EYEFUL system, it is finally aimed at evaluate subjects’ dependence statuses.
To ease understanding of the protocol structure, we provide graphical flowcharts of the main phases in Figure 1, Figure 2 and Figure 3, in which we show the profile of the research members (clinical or technical), the input information (in blue background color), the output information (in light orange background color), and the data and process paths (in red and blue colors, respectively).

2.1. Development Team and Collaborative Framework

The project team in charge of adopting the protocol for the development of a multisensory system for objective and automatic assessments of functionality through the performance of ADLs must include specialists from diverse fields, including telecommunication engineering, industrial electronics, automation engineering, advanced electronic systems engineering, and health-related fields such as medical specialist and occupational therapists. This interdisciplinary team will ensure a holistic approach to the design, development, and validation of the system. The team development effort is shown in Figure 1, as a preliminary stage in the protocol application.
A tight coordination of the interdisciplinary team must be considered to properly address the strict requirements of the clinical design and validation, with the realistic expectations on the capabilities of the technical systems to address the clinical requirements. This organization and coordination will facilitate a seamless integration of technological advancements and clinical insights.
A careful design of the communication and sharing information tools is to be performed, paying special attention to adequate training of the interdisciplinary team members to properly handle them.
Figure 1. Protocol flowchart: development team (Section 2.1), selection of ADLs (Section 2.2), and definitions of sensorization and equipment (Section 2.3).
Figure 1. Protocol flowchart: development team (Section 2.1), selection of ADLs (Section 2.2), and definitions of sensorization and equipment (Section 2.3).
Applsci 14 03415 g001

2.2. Selection of Activities of Daily Living (ADLs)

The initial phase of the protocol involves a systematic selection of ADLs to be assessed by the system, as shown in Figure 1. This selection is based on a comprehensive analysis of existing classifications of ADLs, with a focus on basic activities of daily living (BADLs) and instrumental activities of daily living (IADLs). The selection criteria emphasize objectivity, relevance to a broad population, and the ability to capture a wide range of functional capacities. The selected ADLs must undergo a rigorous evaluation by medical specialists and occupational therapists to ensure they are indicative of dependency levels.
This phase of selection of ADLs will be carried out in the following several stages, also considering iterations across them, if required:
  • In the first stage, the complete list of all available ADLs will be carefully analyzed, using as a reference the classification of activities from the Assessment Motor and Process Skills (AMPS) [12], as it is an observational assessment tool that evaluates the quality of performance in ADLs and has a wide range of activities classified according to their difficulty of execution. The research group of clinicians, particularly the occupational therapists, will make a selection based on the most complete activities. Consideration will be given to the number of performance components and functional requirements they contain, their feasibility based on environmental characteristics, their level of difficulty and physical effort, their efficiency and safety, and the need for assistance to ensure a correct and complete evaluation of functional capacities. The selection strategy will consider the following criteria:
    • Allow for the most objective evaluation of the subject regardless of gender.
    • Be capable of evaluating performance with the greatest possible independence from the subject’s pathology or physical or psychological impairment.
    • Be a known activity for the largest possible population.
    • Be complementary activities to each other in order to observe the most limiting components of performance.
    The list of selected activities will be fully detailed, considering their movement characteristics, cognitive processing of the action, and performance requirements.
  • In the second stage, the selected ADLs will be examined by the team of medical specialists to determine if limitations and characteristic signs of dependence could be detected after the performance and observation of these activities.
  • In the third stage, a group of clinical experts in occupational therapy, external to the research team will independently score the activities according to the following: (i) the suitability of the activity to measure the desired parameters; (ii) the appearance of relevant functionality parameters. This scoring will be based on observations made in a population with dependence when performing the selected activities.
    The three previous phases will conclude with the selection of the final ADLs, which must complement each other and serve as the basis for the development of the EYEFUL system.
  • In the final fourth stage, the clinical research team, along with the engineering team, will evaluate the technical implications of the final selection of ADLs, in what respect to both suitable sensor configurations, and the requirements of parameters to be measured. The engineering team will provide information on the current and foreseeable capabilities of automatic systems to successfully analyze the available activities. From this coordinated effort, the engineering and clinical experts will also propose the required modifications needed to shape the final activities. This will allow for an early integration of the technical and clinical perspectives in the methodological design.

2.3. Definition of the Sensorization, Computing, and Communication Configuration for ADL Monitoring

Following the selection of ADLs, the protocol requires the definition of the optimal configuration of sensors, data acquisition methods, and required computational and communication equipment (the flowchart for this phase is shown in Figure 1). This includes the type of sensors (e.g., depth sensors, audio sensors, and wearable devices), their placement (considering the details of the space where the functional assessment will take place), and the data they will collect (including required data throughput and storage requirements).
The configuration should capture the detailed information on the functional parameters, environmental interactions, and physiological parameters during the performance of ADLs. A key goal is to ensure that the data collected are robust, reliable, and capable of supporting the objective assessment of functional capacity. Coordination between the clinical and engineering teams is essential in this task, as clinics will need to adapt and modify the proposed activities based on the characteristics and functions of the selected sensors together with the engineers, as well as their placement.
Special attention should be devoted to communication and storage specifications, as they will be crucial to avoid problems during the data acquisition efforts.

2.4. Definition of Quantitative Metrics and Their Relationship with Measurable Functional Parameters

A critical component of the protocol is the detailed description of parameters, metrics, and characteristics that, when measured or estimated, encompass a complete and comprehensive evaluation of functionality (the flowchart for this phase is shown in Figure 2). This set of data will be referred to as metrics, in general. For this process, a catalogue of metrics will be developed from the necessary aspects to be considered in a functional assessment:
  • Metrics related to physical aspects such as body position, balance, speed of movement, bimanual coordination, etc.
  • Metrics related to interaction with objects such as fine coordination, object manipulation, hand–eye coordination, etc.
  • Metrics related to cognitive aspects such as problem solving, sequencing, following instructions, etc.
In turn, a series of metrics obtained from wearable devices, such as heart rate, number of steps, energy expenditure, etc., will be also defined, as they can provide relevant information to be used in the functional assessment process.
The development of these metrics involves, again, close collaboration between engineers and healthcare professionals to ensure technical feasibility, and that they are clinically relevant and accurately reflect people’s functional capacity.
Figure 2. Protocol flowchart: definition of quantitative metrics (Section 2.4), definition of inferences (Section 2.5), and definition of the technical and clinical validation processes (Section 2.6).
Figure 2. Protocol flowchart: definition of quantitative metrics (Section 2.4), definition of inferences (Section 2.5), and definition of the technical and clinical validation processes (Section 2.6).
Applsci 14 03415 g002

2.5. Definition of Relevant Inferences

In order to continue in the process of designing a system similar to EYFUL, it is necessary to specify the quantitative and qualitative inferences that we want the system to generate. These inferences will be composed by both quantitative data (high level measurements) and also qualitative information. They will be obtained from the previously defined metrics, also based on machine learning techniques. The flowchart associated for this phase is shown in Figure 2.
In the case of EYEFUL, being a system that evaluates functionality and dependency status, the most relevant inference will be the dependency status. To show another example, we could think of “Walking capacity” as an inference, while the distance travelled in meters, or the time spent in walking would be associated metrics. Comparing the quantitative high level measurements with values calculated on users without dependence we could even relate them to specific alterations. Thus, a new catalogue of inferences should be defined from the previously established catalogue of metrics.
This task will be crucial in the algorithmic design since any modification in the catalogue of inferences will have an impact on the algorithms and vice versa. A joint work between engineers for the technical part and clinicians for the inference decisions will be necessary.

2.6. Design of the Technical and Clinical Validation Processes

The final system will undergo extensive technical and clinical validation to ensure its accuracy, reliability, and clinical utility. This phase is critical for establishing the system’s effectiveness as a tool for clinical assessment and research (its flowchart is shown in Figure 2):
  • Technical validation: A fully detailed technical validation will be designed, in which the performance and correct functioning of the systems related to both the characterization of subjects’ activity and the estimation of evaluation values will be assessed. A population over 18 years old will be selected. For the initial evaluations, the subjects will not have any pathology or functional limitation, fulfilling the strict requirements of data privacy and informed consent. These subjects will perform the selected activities for the automatic systems training, verification, and analytical validation of EYEFUL [30].
  • Clinical validation: A comprehensive clinical validation will be designed with a focus on determining if the final system meets sensitivity and specificity criteria to detect the necessary functional limitations to discriminate a situation of dependence [30,31,32]. As a guide to the clinical validation of a system such as EYEFUL, the same processes and clinical validation of traditional tests, tools, and measurement instruments will be followed [30]. The steps involved in the clinical validation process are reproducibility analysis, content validity analysis, concurrent validity analysis, and discriminant validity analysis [30,31,32].
This phase will include deciding which of the available standardized evaluation tools will be used to compare their results with those of the developed system for criterion validity. These tools should be the gold standard for the concepts being measured.

2.7. Algorithmic Development and Technical Validation

This phase is devoted to the design, implementation, and technical evaluation of the algorithms and systems in charge of the automatic analysis of the recorded sequences to estimate the metrics and inferences devised in previous phases. It also includes the generation and evaluation of the relevant inferences from the automatic analysis. The associated flowchart is shown in Figure 3.
The development and evaluation will require an initial compilation of existing sets of labeled data already published in the literature and available for research purposes. There are many datasets available for body analysis tasks that are may not be directly related to the final system objectives but are general enough to be used for pre-training the models that will be used and developed in different tasks (for example, body pose estimation and tracking, recognition of human activities, etc.).
A crucial task will be the characterization of human activity at body and face level, with the aim of exploiting and improving existing human activity characterization systems to meet the new challenges posed by the proposed system (EYEFUL in our case). It will address the development of robust algorithms and the use of the different types of sensors (as previously defined), for the extraction of different data. The process will include an extensive work in the fine-grained analysis and characterization of human activities. Algorithmic solutions based on deep learning will be designed, implemented, and tested using existing datasets, and refined with data extracted from recorded sessions in the space where the people will carry out the selected ADLs.
Once the set of metrics has been defined and the algorithms for their automatic extraction have been developed, their integration will be addressed in two directions.
  • The first one by integrating them into the process of assessing functionality. So, from the catalog of defined elements, we will select the most discriminative subset in order to score the functional level. Machine learning techniques are to be used in the selection of those metrics in a data driven approach. The close coordination with the clinical team will again be necessary in this effort.
  • The second one, once the extracted metrics form different sensors are validated, their fusion is to be considered to improve the overall estimations (related to the users’ location and pose, activity characterization and assessment, emotional state characterization, etc.).
Figure 3. Protocol flowchart: algorithmic development and technical validation (Section 2.7) and deployment and clinical validation (Section 2.8).
Figure 3. Protocol flowchart: algorithmic development and technical validation (Section 2.7) and deployment and clinical validation (Section 2.8).
Applsci 14 03415 g003
Finally, in the technical validation stage, the results of the algorithms will be compared with manually annotated and labeled data. As detailed above, these data will initially come from publicly available datasets and, later on, from actual recordings of subjects’ test sessions within the protocol scope. Specifically, careful image annotation work will be needed for the evaluation of the defined metrics (more on this in Section 2.8). Again, a close coordination between engineers clinicians will be necessary to properly assess the evaluation results of the metrics and inferences with a clinical perspective.
As an example of the application of this phase to the EYEFUL project, Figure 4 shows a prospective simplified schematic example of the algorithmic system modules needed for automatic human functional evaluation, showing the sensor sources, some automatic analysis modules (object detection, human action recognition, etc.), and the output information to be generated (quantitative metrics and inferences).
Figure 4. Algorithmic system modules for automatic human functional assessment in the EYEFUL system.
Figure 4. Algorithmic system modules for automatic human functional assessment in the EYEFUL system.
Applsci 14 03415 g004

2.8. Deployment and Clinical Validation

This phase covers the actual hardware and software deployment, and all the tasks leading to the clinical validation process. The associated flowchart is shown in Figure 3.
The system deployment in the assessment space will be the responsibility of the engineering group. They will install and configure the sensor, communication, and processing equipment in the space where the people will carry out the selected ADLs, and to verify its correct functioning. Once the hardware is deployed and verified, a full installation of all the integrated algorithmic modules will be performed, also checking their correct operation.
A user control interface will be designed and implemented. It is necessary to integrate all the information from the different sensors in a single interface, which allows the handling by the clinical staff during the recording and data capture phases, and also during the assessment system operation. The clinicians will specify how they want the user control interface, and the engineers will make the necessary modifications.
To carry out the clinical validation process of the system following this protocol, it is necessary to have cohorts of people with different characteristics. In the case of the EYEFUL system, we will require people with and without dependence. The inclusion criteria for the group of people without dependence, who will form the control group, are as follows:
  • Be over 18 years of age.
  • Have no recognized degree of dependence.
  • Accept and sign the informed consent form.
The inclusion criteria for the group of people with dependence, who will form the experimental group, are the following:
  • To be over 18 years of age.
  • To have a recognized degree of dependence (from degree I, level 1 to degree II, level 2).
  • To be able to walk independently, with or without support products (canes, crutches, walkers, etc.)
  • To accept and sign the informed consent form.
Exclusion criteria for the study are specified as:
  • Having a cognitive, physical, or sensory deficit that impedes the understanding of questionnaires or the performance of the tests.
  • Wheelchair users.
A sample of the control group will be required for the training of the system. The control group will be obtained by proximity to the research team and the university environment, for convenience and for matching purposes. All subjects will be informed and must sign the informed consent form approved by the Ethics Committee before participating in the study.
This phase also includes the recordings for the users of both the control and dependence groups, which implies intensive work in users’ recruitment, recording sessions scheduling, and labeling. The labeling process is a specialized task that requires the use of specific annotation applications and will be carried out by the occupational therapy group, as they specialize in administering standardized observational tests that assess functional limitation. It is a time-consuming task, but it is crucial in the training and evaluation phase of the systems from a technical standpoint and in the final clinical validation. We have identified the ELAN annotation application [33] as a suitable tool for this task, as it allows the temporal labeling of multimedia sequences (specifically audio and video streams), that can be applied to any system within this protocol scope.
Once the verification, analytical validation and technical validation of the data generated by the system have been completed, the clinical validation will be carried out, i.e., it will be demonstrated whether the system acceptably identifies, measures, or predicts the level of dependence.
Applying the documentation of clinical validation of traditional measurement tools [30,31,32], the clinical validation in the case of the EYEFUL system will address the following analyses:
  • Reliability analysis will examine the degree to which EYEFUL measures without error, and internal consistency will evaluate whether the metrics measuring dependence are homogeneous with each other.
  • Content validity will evaluate whether EYEFUL covers all the metrics it is intended to measure.
  • Criterion validity will seek to establish the degree of correlation between the scores obtained with EYEFUL and those obtained from the assessment tools established as the gold standard. This will require a sample of 45 subjects without dependence.
  • Finally, for the analysis of construct validity, the degree of EYEFUL’s ability to distinguish between the population without dependence and the population with dependence will be determined. This will require a sample of 90 subjects, 45 non-dependent and 45 dependent, and the resulting EYEFUL scores of both groups will be used. These subjects will follow the inclusion and exclusion criteria described above. To determine the appropriate sample size for the clinical validity phases, the G*Power software (version 3.1.7) was used [31,32].

2.9. Dissemination of Results

Upon completion of the development and validation phases, the final phase will be in charge of disseminating the results through scientific publications, presentations at conferences, and scientific events in the fields of health sciences and engineering. The aim of this dissemination will be to share the knowledge acquired in the project, to promote the adoption of the system, and to encourage research in the field of automatic and objective evaluation of functional limitations and dependency status.

3. Materials

For the development of systems using the described protocol, it will be necessary to have a specific space for the deployment of equipment and where to capture data, recording people while carrying out ADLs. In the specific case of the EYEFUL system, the “Activity Sensory Analysis Laboratory” is available. It is a 100 m2 simulated house divided into 4 rooms (bedroom, living room, kitchen, and bathroom) of 25 m2 each. All of them are fully equipped with adapted furniture, electrical appliances, and kitchenware. This laboratory will be equipped with different sensor equipment, developed as the project progresses (following the proposed protocol), capable of collecting data during the performance of the proposed ADLs (see Figure 5).

4. Organizational Details and Reproducibility

Regarding the reproducibility of the study, this section elaborates on the organizational details and criteria for choices made in the EYEFUL project.

4.1. Criteria for Choosing ADLs

The selection of ADLs for evaluation using the EYEFUL system will be guided by comprehensive criteria. This decision-making process, detailed in the protocol, ensures that the chosen ADLs provide a balanced view of an individual’s functional status.

4.2. Flowcharts and Documentation

Flowcharts outlining the step-by-step process of process of creating EYEFUL have been provided in Figure 1, Figure 2 and Figure 3. These visual guides, along with detailed documentation of the study protocol, provide a roadmap for replicating the study in other settings.

5. Discussion

The EYEFUL system will represent a significant advancement in the objective evaluation of functional capacities and dependency status, employing a multisensory system to capture the nuances of performance during ADLs. The project will be carried out strictly following the proposed protocol and is expected to be in development for three years. The following discussion underscores the potential implications of our findings for both clinical practice and research, emphasizing the innovative integration of technology and healthcare.

5.1. Implications for Clinical Practice

The introduction of the EYEFUL system and its development protocol into clinical settings could improve the way functional capacities are assessed, moving beyond subjective observations and self-reports to a more accurate and unbiased evaluation. The system’s ability to provide detailed metrics when performing ADLs can help healthcare professionals diagnose functional limitations, plan personalized care, and track the progression of conditions over time. In addition, the use of this technology can increase the accuracy of evaluations of dependency status, thereby improving resource allocation and support for these individuals.

5.2. Contributions to Research

From a research point of view, the proposed protocol offers a valuable guidance tool for designing and creating other systems to study the dynamics of human activity and dependency. By providing a standardized method for assessing functional capacity, it facilitates comparative studies across populations and conditions, contributing to a deeper understanding of the factors influencing independence and quality of life in various cohorts. Additionally, the data generated by the EYEFUL system can inform the development of new interventions aimed at preserving or enhancing functional abilities.

5.3. Addressing Limitations and Future Directions

While the EYEFUL system introduces a promising approach to functional capacity assessment, it is not without limitations. Given that the system is not yet built, it is not known to what extent the EYEFUL system will be able to detect complex aspects of human activity, and this will have to be evaluated after the system implementation and evaluation. Future research will aim to validate the system in a variety of environments and populations to ensure its broad applicability.

6. Conclusions

The described protocol, based on the proposal to develop the EYEFUL system, introduces a novel pathway for the development of new systems for the evaluation of functional abilities and dependency status, highlighting the potential of multisensory systems such as EYEFUL to transform clinical and research practices.
By generating objective and detailed evaluations during ADL performance, the EYEFUL system will provide information on individual functional abilities, contributing to more personalized and effective care strategies.
The protocol underlines the importance of interdisciplinary collaboration in advancing health technologies and lays the foundation for future innovations in functional capacity evaluation. The proposed structured approach will help initiatives similar to the EYEFUL project to grow on a common basis with a rigorous design and development process, and comprehensive technical and clinical validation efforts.
As future tasks, the full development and validation of the EYEFUL system will be carried out, specifically focusing on improving its accuracy, ease of use, and applicability in diverse populations and settings, with the goal of setting a new standard in the evaluation of functionality and dependency statuses.

Author Contributions

Conceptualization, M.P.-d.-H.-T., E.H.-H., M.M.-R., J.M.-G. and R.M.M.-P.; methodology, M.P.-d.-H.-T., E.H.-H., P.S.-H.-B., N.M.-B., S.S.-T., M.M.-R., J.M.-G., C.L.-G., S.E.P.-C., J.L.M.-S. and R.M.M.-P.; software, M.M.-R., J.M.-G., C.L.-G., S.E.P.-C., and J.L.M.-S.; validation, P.O.-B., M.P.-d.-H.-T., E.H.-H., P.S.-H.-B., N.M.-B., S.S.-T., M.M.-R., J.M.-G., C.L.-G., S.E.P.-C., J.L.M.-S. and R.M.M.-P.; formal analysis, P.O.-B. and S.S.-T.; investigation, P.O.-B., M.P.-d.-H.-T., E.H.-H., P.S.-H.-B., N.M.-B., S.S.-T., M.M.-R., J.M.-G. and R.M.M.-P.; resources, M.P.-d.-H.-T., P.S.-H.-B., M.M.-R., J.M.-G., C.L.-G., S.E.P.-C., J.L.M.-S. and R.M.M.-P.; data curation, P.O.-B., M.P.-d.-H.-T., E.H.-H., P.S.-H.-B., N.M.-B., S.S.-T., M.M.-R., J.M.-G. and R.M.M.-P.; writing—original draft preparation, P.O.-B., M.P.-d.-H.-T., M.M.-R., J.M.-G. and R.M.M.-P.; writing—review and editing, P.O.-B., M.P.-d.-H.-T., E.H.-H., P.S.-H.-B., N.M.-B., S.S.-T., M.M.-R., J.M.-G., C.L.-G., S.E.P.-C. and R.M.M.-P.; visualization, P.O.-B., M.P.-d.-H.-T., M.M.-R., J.M.-G. and R.M.M.-P.; supervision, P.O.-B., M.P.-d.-H.-T., E.H.-H., P.S.-H.-B., N.M.-B., S.S.-T., M.M.-R., J.M.-G., C.L.-G., S.E.P.-C., J.L.M.-S. and R.M.M.-P.; project administration, P.O.-B., M.P.-d.-H.-T., M.M.-R., J.M.-G. and R.M.M.-P.; funding acquisition, M.P.-d.-H.-T., M.M.-R., J.M.-G. and R.M.M.-P. All authors have read and agreed to the published version of the manuscript.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Spanish Ministry of Science and Innovation, [grant number MICINN/AEI/10. 13039/501100011033], in the framework of the EYEFUL-UAH (PID2020-113118RB-C31) and EYEFUL-URJC (PID2020-113118RB-C33) projects; by Comunidad Autónoma de Madrid under project CONCORDIA (CM/JIN/2021-015); and by UAH under projects ARGOS+ (PIUAH21/IA-016) and METIS (PIUAH22/IA-037).

Institutional Review Board Statement

The study was conducted in accordance with the Declaration of Helsinki, and approved by the Research Ethics Committee of the Universidad Rey Juan Carlos (reference code is 2202202204922).

Acknowledgments

We thank the collaboration of Alessandro Melino-Carrero, Alvaro Nieva-Suarez, Irene Guardiola-Luna, Marina Murillo-Teruel, Lucía Hernández-Hernández, and Beatriz Redondo-Cardeña, supported by the European Union, Next Generation EU, through the Investigo program. We also gratefully acknowledge the computer resources at Artemisa, funded by the European Union ERDF and Comunitat Valenciana as well as the technical support provided by the Instituto de Fisica Corpuscular, IFIC (CSIC-UV).

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. Kekäläinen, T.; Luchetti, M.; Sutin, A.; Terracciano, A. Functional Capacity and Difficulties in Activities of Daily Living from a Cross-National Perspective. J. Aging Health. 2023, 35, 356–369. [Google Scholar] [CrossRef] [PubMed]
  2. Dias, E.N.; da Silva, J.V.; Pais-Ribeiro, J.L.; Martins, T. Validation of the advanced activities of daily living scale. Geriatr. Nurs. 2019, 40, 7–12. [Google Scholar] [CrossRef] [PubMed]
  3. Dias, E.G.; Duarte, Y.A.D.O.; Almeida, M.H.M.D.; Lebrão, M.L. As Atividades avançadas de vida diária como componente da avaliação funcional do idoso. Rev. Ter. Ocup. Univ. São Paulo 2014, 25, 225–232. [Google Scholar] [CrossRef]
  4. Romero Ayuso, D.M. Actividades de la Vida Diaria. An. Psicol. 2007, 23, 264–271. [Google Scholar]
  5. Cook, D.J.; Schmitter-Edgecombe, M.; Jonsson, L.; Morant, A.V. Technology-Enabled Assessment of Functional Health. IEEE Rev. Biomed. Eng. 2019, 12, 319–332. [Google Scholar] [CrossRef] [PubMed]
  6. González-Rodríguez, R. El análisis de la actividad en el proceso de valoración de la situación de dependencia. Rev. Electrónica Ter. Ocup. Galicia TOG 2015, 7, 66–86. [Google Scholar]
  7. Ley 39/2006, de 14 de Diciembre, de Promoción de la Autonomía Personal y Atención a las Personas en Situación de Dependencia. Boletín Oficial del Estado, Número 299, 15 de Diciembre de 2006. Available online: https://www.boe.es/eli/es/l/2006/12/14/39/con (accessed on 15 April 2024).
  8. Real Decreto 174/2011 por el que se Aprueba el Baremo de Valoración de la Situación de Dependencia Establecido por la Ley 39/2006, de 14 de Diciembre, de Promoción de la Autonomía Personal y Atención a las Personas en Situación de Dependencia. (Boletín Oficial del Estado, Número 42, 18 de Febrero de 2011). Available online: https://www.boe.es/eli/es/rd/2011/02/11/174 (accessed on 15 April 2024).
  9. Jiménez, S.; Viola, A. Observatorio de la Dependencia: Cuarto Informe; Estudios Sobre la Economía Española no. 2022-16; Fedea: Madrid, Spain, 2022. [Google Scholar]
  10. Cohen, M.E.; Marino, R.J. The tools of disability outcomes research functional status measures. Arch. Phys. Med. Rehabil. 2000, 81 (Suppl. S2), S21–S29. [Google Scholar] [CrossRef] [PubMed]
  11. Giambelluca, E.; Panigazzi, M.; Saade, A.; Imbriani, M. Assessment of functional status and rehabilitative strategies in occupational therapy: Role of the Groningen Activity Restriction Questionnaire. G. Ital. Med. Lav. Ergon. 2019, 41, 52–57. [Google Scholar] [PubMed]
  12. Bray, K.; Fisher, A.G.; Duran, L. The Validity of Adding New Tasks to the Assessment of Motor and Process Skills. Am. J. Occup. Ther. 2001, 55, 409–415. [Google Scholar] [CrossRef]
  13. Mcalister, C.; Schmitter-Edgecombe, M.; Lamb, R. Examination of Variables That May Affect the Relationship Between Cognition and Functional Status in Individuals with Mild Cognitive Impairment: A Meta-Analysis. Arch. Clin. Neuropsychol. 2016, 31, 123–147. [Google Scholar] [CrossRef]
  14. Fisher, A.G.; Griswold, L.A.; Munkholm, M.; Kottorp, A. Evaluating domains of everyday functioning in people with developmental disabilities. Scand. J. Occup. Ther. 2017, 24, 1–9. [Google Scholar] [CrossRef] [PubMed]
  15. Lussier, M.; Adam, S.; Chikhaoui, B.; Consel, C.; Gagnon, M.; Gilbert, B.; Giroux, S.; Guay, M.; Hudon, C.; Imbeault, H.; et al. Smart Home Technology: A New Approach for Performance Measurements of Activities of Daily Living and Prediction of Mild Cognitive Impairment in Older Adults. J. Alzheimers. Dis. 2019, 68, 85–96. [Google Scholar] [CrossRef]
  16. Nef, T.; Urwyler, P.; Büchler, M.; Tarnanas, I.; Stucki, R.; Cazzoli, D.; Müri, R.; Mosimann, U. Evaluation of Three State-of-the-Art Classifiers for Recognition of Activities of Daily Living from Smart Home Ambient Data. Sensors 2015, 15, 11725–11740. [Google Scholar] [CrossRef] [PubMed]
  17. Alberdi Aramendi, A.; Weakley, A.; Aztiria Goenaga, A.; Schmitter-Edgecombe, M.; Cook, D.J. Automatic assessment of functional health decline in older adults based on smart home data. J. Biomed. Inform. 2018, 81, 119–130. [Google Scholar] [CrossRef] [PubMed]
  18. Jekel, K.; Damian, M.; Storf, H.; Hausner, L.; Frölich, L. Development of a Proxy-Free Objective Assessment Tool of Instrumental Activities of Daily Living in Mild Cognitive Impairment Using Smart Home Technologies. J. Alzheimers Dis. 2016, 52, 509–517. [Google Scholar] [CrossRef] [PubMed]
  19. Suzuki, Y.; Teruya, K.; Mochizuki, H.; Nagasawa, A.; Kondo, T.; Shimoda, N. Evaluation of Activities of Daily Living/Instrumental Activities of Daily Living to Accurately Determine Severity of Moderate and Severe Alzheimer’s Disease: Comparison of Assessments by Receiver Operating Characteristic Curve and Discriminant Analyses. Dement. Geriatr. Cogn. Dis. Extra. 2019, 9, 227–235. [Google Scholar] [CrossRef] [PubMed]
  20. Nielsen, L.M.; Kirkegaard, H.; Østergaard, L.G.; Bovbjerg, K.; Breinholt, K.; Maribo, T. Comparison of self-reported and performance-based measures of functional ability in elderly patients in an emergency department: Implications for selection of clinical outcome measures. BMC Geriatr. 2016, 16, 199. [Google Scholar] [CrossRef] [PubMed]
  21. Lago, P.; Roncancio, C.; Jiménez-Guarín, C. Learning and Managing Context Enriched Behavior Patterns in Smart Homes. Future Gener. Comput. Syst. 2019, 91, 191–205. [Google Scholar] [CrossRef]
  22. Niknejad, N.; Ismail, W.B.; Mardani, A.; Liao, H.; Ghani, I. A comprehensive overview of smart wearables: The state of the art literature, recent advances, and future challenges. Eng. Appl. Artif. Intell. 2020, 90, 103529. [Google Scholar] [CrossRef]
  23. Noor, M.H.M.; Salcic, Z.; Wang, K.I. Ontology-based sensor fusion activity recognition. J. Ambient. Intell. Humaniz. Comput. 2020, 11, 3073–3087. [Google Scholar] [CrossRef]
  24. Dawadi, P.N.; Cook, D.J.; Schmitter-Edgecombe, M. Automated Cognitive Health Assessment from Smart Home-Based Behavior Data. IEEE J. Biomed. Health Inform. 2016, 20, 1188–1194. [Google Scholar] [CrossRef] [PubMed]
  25. Kour, N.; Sunanda Arora, S. Computer-Vision Based Diagnosis of Parkinson’s Disease via Gait: A Survey. IEEE Access 2019, 7, 156620–156645. [Google Scholar] [CrossRef]
  26. Munro, J.; Damen, D. Multi-modal domain adaptation for fine-grained action recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 14–19 June 2020; pp. 122–132. [Google Scholar]
  27. Nguyen, T.H.; Nebel, J.C.; Florez-Revuelta, F. Recognition of Activities of Daily Living with Egocentric Vision: A Review. Sensors 2016, 16, 72. [Google Scholar] [CrossRef] [PubMed]
  28. Morgan, C.; Craddock, I.; Tonkin, E.L.; Kinnunen, K.M.; McNaney, R.; Whitehouse, S.; Mirmehdi, M.; Heidarivincheh, F.; McConville, R.; Carey, J.; et al. Protocol for PD SENSORS: Parkinson’s Disease Symptom Evaluation in a Naturalistic Setting producing Outcome measuRes using SPHERE technology. An observational feasibility study of multi-modal multi-sensor technology to measure symptoms and activities of daily living in Parkinson’s disease. BMJ Open 2020, 10, e041303. [Google Scholar] [CrossRef] [PubMed]
  29. Park, S.; Fisher, A.G.; Velozo, C.A. Using the assessment of motor and process skills to compare occupational performance between clinic and home settings. Am. J. Occup. Ther. 1994, 48, 697–709. [Google Scholar] [CrossRef] [PubMed]
  30. Goldsack, J.C.; Coravos, A.; Bakker, J.P.; Bent, B.; Dowling, A.V.; Fitzer-Attas, C.; Dunn, J.; Godfrey, A.; Godino, J.G.; Gujar, N.; et al. Verification, analytical validation, and clinical validation (V3): The foundation of determining fit-for-purpose for Biometric Monitoring Technologies (BioMeTs). NPJ Digit. Med. 2020, 3, 55. [Google Scholar] [CrossRef] [PubMed]
  31. Carvajal, A.; Centeno, C.; Watson, R.; Martínez, M.; Sanz Rubiales, Á. How is an instrument for measuring health to be validated? Anales Sis. San. Navarra 2011, 34, 63–72. [Google Scholar]
  32. Luján-Tangarife, J.A.; Cardona-Arias, J.A. Construcción y validación de escalas de medición en salud: Revisión de propiedades psicométricas. Arch. Med. 2015, 11, 1–10. [Google Scholar]
  33. Max Planck Institute for Psycholinguistics, The Language Archive, “Elan (Version 6.6),” Max Planck Institute for Psycholinguistics, The Language Archive, Nijmegen. 2023. Available online: https://archive.mpi.nl/tla/elan (accessed on 15 April 2024).
Figure 5. Floor plan of the sensory activity analysis laboratory.
Figure 5. Floor plan of the sensory activity analysis laboratory.
Applsci 14 03415 g005
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Obeso-Benítez, P.; Pérez-de-Heredia-Torres, M.; Huertas-Hoyas, E.; Sánchez-Herrera-Baeza, P.; Máximo-Bocanegra, N.; Serrada-Tejeda, S.; Marron-Romera, M.; Macias-Guarasa, J.; Losada-Gutierrez, C.; Palazuelos-Cagigas, S.E.; et al. Protocol for the Development of Automatic Multisensory Systems to Analyze Human Activity for Functional Evaluation: Application to the EYEFUL System. Appl. Sci. 2024, 14, 3415. https://doi.org/10.3390/app14083415

AMA Style

Obeso-Benítez P, Pérez-de-Heredia-Torres M, Huertas-Hoyas E, Sánchez-Herrera-Baeza P, Máximo-Bocanegra N, Serrada-Tejeda S, Marron-Romera M, Macias-Guarasa J, Losada-Gutierrez C, Palazuelos-Cagigas SE, et al. Protocol for the Development of Automatic Multisensory Systems to Analyze Human Activity for Functional Evaluation: Application to the EYEFUL System. Applied Sciences. 2024; 14(8):3415. https://doi.org/10.3390/app14083415

Chicago/Turabian Style

Obeso-Benítez, Paula, Marta Pérez-de-Heredia-Torres, Elisabet Huertas-Hoyas, Patricia Sánchez-Herrera-Baeza, Nuria Máximo-Bocanegra, Sergio Serrada-Tejeda, Marta Marron-Romera, Javier Macias-Guarasa, Cristina Losada-Gutierrez, Sira E. Palazuelos-Cagigas, and et al. 2024. "Protocol for the Development of Automatic Multisensory Systems to Analyze Human Activity for Functional Evaluation: Application to the EYEFUL System" Applied Sciences 14, no. 8: 3415. https://doi.org/10.3390/app14083415

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop