Research on Designing Context-Aware Interactive Experiences for Sustainable Aging-Friendly Smart Homes
Abstract
:1. Introduction
2. Survey of Elderly Users Living Alone
2.1. Characteristics of Elderly Users Aged 60–74
2.2. Needs of Elderly Users Aged 60–74
2.3. Interactive Design Principles for Aging-Friendly Smart Homes
- Safety: Prevent risks by avoiding sharp or fragile items and ensuring electrical safety and reliability. Features such as automatic power-off and remote monitoring can be included.
- Ease of Use: Simplify the operation interface through intuitive interaction methods such as gesture control and light control. Enhance readability and interactivity with large fonts, high-contrast colors, and voice feedback.
- Adaptability: Smart facilities and functions should flexibly adapt to the needs of elderly users and changes in family structure. Address varying needs at different times and under different environmental conditions by using adjustable equipment and layouts. Context-aware technologies should enhance service autonomy and diversity.
- Emotional Engagement: Incorporate emotional care into the interaction experience, such as warm interfaces and voice services. Support remote social interactions like video calls and family photo sharing to alleviate feelings of loneliness. Additionally, provide automatic error correction features to improve command tolerance.
- Sensory Compensation: Use sensory compensation designs, such as tactile and auditory feedback, to assist information delivery. For example, vibration feedback, temperature change signals, and auditory guidance can help users perceive and operate the system effectively [10].
3. Context-Aware Processes for Sustainable Aging-Friendly Smart Homes
- User Context Elements: These include physiological, behavioral, and psychological characteristics of elderly users.
- Environmental Context Elements: These encompass information about sound, light intensity, location, temperature, and humidity.
- Device Context Elements: These pertain to device usage duration, functional status, and interaction modes.
- Information Filtering: Identifying and eliminating erroneous data points, duplicate information, and data irrelevant to the current analysis goals.
- Information Synthesis: Extracting features, identifying patterns and trends from the data, and combining and integrating the filtered data to form a comprehensive understanding of user behavior, environmental information, and device status.
- Prioritization: Evaluating and determining the priority and urgency of different context information based on predefined criteria or the importance of the data, and adjusting response strategies accordingly.
- Responsive Decision-Making: This involves executing services based on the identification of specific commands from elderly users, such as voice, gesture, or touch, combined with context information to provide on-demand services, thereby avoiding unnecessary energy consumption.
- Predictive Decision-Making: This approach employs complex data processing and analysis algorithms, such as machine learning and artificial intelligence technologies, to learn user habits and recommend services. By using visual, auditory, and tactile feedback, predictive services can be recommended to elderly users, including inquiries and choices for the next steps, to achieve optimal energy usage efficiency. For instance, when the smart home system detects that an elderly user is cooking, it analyzes the activity patterns, predicts cooking behavior, and suggests services such as whether to activate the range hood, open windows for ventilation, or adjust the heat level.
- Proactive Decision-Making: This method involves perceiving user intentions based on environmental factors, device status, and user behavior preferences, and interpreting multimodal information. The system not only responds to and predicts needs but also proactively offers services even in the absence of explicit requests. For example, if a solitary elderly user falls, the system will automatically alert and use a voice assistant to comfort the user, providing self-rescue measures.
4. User Behavior Recognition Methods in Aging-Friendly Smart Homes
4.1. Speech Recognition
4.2. Gesture Recognition
4.3. Non-Contact Activity Recognition
5. Context-Aware Interactive Experience Design for Aging-Friendly Smart Homes
5.1. Model Architecture for Context Aware Interactive Experience in Aging-Friendly Smart Homes
- (1)
- Multimodal Context Information Data Collection
- (2)
- Intelligent Analysis of Context Information Data
- (3)
- Service Feedback in Aging-friendly Smart Homes
5.2. Typical Application Scenarios of Context-Aware Aging-Friendly Smart Homes—A Case Study of Fall Detection
- (1)
- Smart Home Device Layout for Fall Detection
- (2)
- User interface
- (3)
- Hardware
- (4)
- Evaluation
6. Conclusions and Future Scopes
Author Contributions
Funding
Data Availability Statement
Conflicts of Interest
References
- Li, Y.C.; Ghazilla, R.A.R.; Abdul-Rashid, S.H. QFD-Based Research on Sustainable User Experience Optimization Design of Smart Home Products for the Elderly: A Case Study of Smart Refrigerators. Int. J. Environ. Res. Public Health 2022, 19, 13742. [Google Scholar] [CrossRef] [PubMed]
- Valipoor, S.; Pati, D.; Stock, M.S.; Bazuin, D. Safer chairs for elderly patients: Design evaluation using electromyography and force measurement. Ergonomics 2018, 61, 902–912. [Google Scholar] [CrossRef] [PubMed]
- Turjamaa, R.; Pehkonen, A.; Kangasniemi, M. How smart homes are used to support older people: An integrative review. Int. J. Older People Nurs. 2019, 14, e12260. [Google Scholar] [CrossRef] [PubMed]
- Karsh, B.; Holden, R.J.; Or, C.K.L. Human factors and ergonomics of health information technology implementation. In Handbook of Human Factors and Ergonomics in Health Care and Patient Safety, 2nd ed.; CRC Press: Boca Raton, FL, USA, 2011; pp. 249–264. [Google Scholar]
- Masai, K.; Kunze, K.; Sakamoto, D.; Sugiura, Y.; Sugimoto, M. Face commands-user-defined facial gestures for smart glasses. In Proceedings of the IEEE International Symposium on Mixed and Augmented Reality (ISMAR), Recife, Brazil, 9–13 November 2020; pp. 374–386. [Google Scholar]
- Ding, J.; Wang, Y.; Si, H.; Ma, J.; He, J.; Liang, K.; Fu, S. Multimodal Fusion-GMM based Gesture Recognition for Smart Home by WiFi Sensing. In Proceedings of the 2022 IEEE 95th Vehicular Technology Conference: (VTC2022-Spring), Helsinki, Finland, 19–22 June 2022; pp. 1–6. [Google Scholar]
- Miraoui, M.; El-Etriby, S.; Tadj, C. A Hybrid Modular Context Aware Services Adaptation for a Smart Living Room. Intell. Autom. Soft Comput. 2018, 24, 299–307. [Google Scholar] [CrossRef]
- Yang, M.; Huang, H.; Yuan, H.; Sun, Q. Interaction Design of Products for the Elderly in Smart Home Under the Mode of Medical Care and Pension. In Proceedings of the HCI International 2016—Posters’ Extended Abstracts: 18th International Conference, HCI International 2016, Toronto, ON, Canada, 17–22 July 2016; Springer International Publishing: Berlin/Heidelberg, Germany, 2016; pp. 145–156. [Google Scholar]
- Duan, Q.; Fang, K.; Liu, M.; Pan, Y. Design Research of Geriatric Rehabilitation Products Based on Natural Interaction. In Proceedings of the HCI International 2016—Posters’ Extended Abstracts: 18th International Conference, HCI International 2016, Toronto, ON, Canada, 17–22 July 2016; Springer International Publishing: Berlin/Heidelberg, Germany, 2016; pp. 426–431. [Google Scholar]
- Cho, M.E.; Kim, M.J. Characterizing the interaction design in healthy smart home devices for the elderly. Indoor Built Environ. 2014, 23, 141–149. [Google Scholar] [CrossRef]
- Schilit, B.; Adams, N.; Want, R. Context-awareness computing applications. In Proceedings of the Workshop on Mobile Computing Systems and Applications, Washington, DC, USA, 8–9 December 1994; pp. 89–101. [Google Scholar]
- Abowd, G.D.; Dey, A.K.; Brown, P.J.; Davies, N.; Smith, M.; Steggles, P. Towards a Better Understanding of Context and Context-awareness. In Proceedings of the 1st International Symposium on Handheld and Ubiquitous Computing, Karlsruhe, Germany, 27–29 September 1999; Springer: Berlin, Germany, 1999. [Google Scholar]
- Park, J.S.; Kim, S.H. Noise Cancellation Based on Voice Activity Detection Using Spectra Variation for Speech Recognition in Smart Home Devices. Intell. Autom. Soft Comput. 2020, 26, 149–159. [Google Scholar]
- Bissoli, A.; Lavino-Junior, D.; Sime, M.; Encarnação, L.; Bastos-Filho, T. A Human-Machine Interface Based on Eye Tracking for Controlling and Monitoring a Smart Home Using the Internet of Things. Sensors 2019, 19, 859. [Google Scholar] [CrossRef] [PubMed]
- Ranieri, C.M.; MacLeod, S.; Dragone, M.; Vargas, P.A.; Romero, R.A.F. Activity recognition for ambient assisted living with videos, inertial units and ambient sensors. Sensors 2021, 21, 768. [Google Scholar] [CrossRef] [PubMed]
- Kołakowska, A.; Szwoch, W.; Szwoch, M. A Review of Emotion Recognition Methods Based on Data Acquired via Smartphone Sensors. Sensors 2020, 20, 6367. [Google Scholar] [CrossRef] [PubMed]
- Muaaz, M.; Chelli, A.; Gerdes, M.W.; Pätzold, M. Wi-Sense: A passive human activity recognition system using Wi-Fi and convolutional neural network and its integration in health information systems. Ann. Telecommun. 2021, 77, 163–175. [Google Scholar] [CrossRef]
- Nan, M.; Trăscău, M.; Florea, A.M.; Iacob, C.C. Comparison between Recurrent Networks and Temporal Convolutional Networks Approaches for Skeleton-Based Action Recognition. Sensors 2021, 21, 2051. [Google Scholar] [CrossRef] [PubMed]
- Anbarasan; Lee, J.S.A. Speech and Gestures for Smart-Home Control and Interaction for Older Adults. In Proceedings of the 3rd International Workshop on Multimedia for Personal Health and Health Care, Seoul, Republic of Korea, 22 October 2018; pp. 49–57. [Google Scholar]
- Kim, S.; Metze, F. Dialog-context aware end-to-end speech recognition. In Proceedings of the 2018 IEEE Spoken Language Technology Workshop (SLT), Athens, Greece, 18–21 December 2018; pp. 434–440. [Google Scholar]
- Lu, Y.; Wang, X.; Gong, J.; Zhou, L.; Ge, S. Classification, Application, Challenge, and Future of Midair Gestures in Augmented Reality. J. Sens. 2022, 2022, 3208047. [Google Scholar] [CrossRef]
- Polo, A.; Salucci, M.; Verzura, S.; Zhou, Z.; Massa, A. Real-Time CSI-Based Wireless Gesture Recognition for Human-Machine Interaction. In Proceedings of the 2021 10th International Conference on Modern Circuits and Systems Technologies (MOCAST), Thessaloniki, Greece, 5–7 July 2021; pp. 1–4. [Google Scholar]
- Al-QANESS, M.A.A. Device-free Human Micro-Activity Recognition Method Using WiFi Signals. Geo-Spat. Inf. Sci. 2019, 22, 11. [Google Scholar] [CrossRef]
- Lu, Y.; Zhou, L.; Zha, S.; Zhuo, X.; Ge, S. Application of Deep Learning and Intelligent Sensing Analysis in Smart Home. Sensors 2024, 24, 953. [Google Scholar] [CrossRef] [PubMed]
- Or, C.K.; Chan, A.H.S. Inspection methods for usability evaluation. In User Experience Methods and Tools in Human-Computer Interaction; CRC Press: Boca Raton, FL, USA, 2024; pp. 170–192. [Google Scholar]
Research Method | Essential Factor | Result |
---|---|---|
User observation | Home life theme | Wash one’s face and rinse one’s mouth, dining, household chores, leisure, socializing |
Product usage frequency | Ordinary household appliances (high frequency), home products (medium frequency), intelligent single products (low frequency) | |
Daily behavior habits | Repetitive behavior (daily routine operations, safety- and health-related operations, daily communication and information acquisition), occasional behavior (emergency situations, special event handling) | |
Multimodal operation instruction | Button, touch screen, voice | |
User interviews | Intelligent product user experience | Difficulty in fine touch control, insensitive speech recognition, redundant functions, and concerns about personal privacy |
Product interaction design | Intuitive behavior modes (sound, touch screen, etc.), simplified operation interface, operation guidance and training | |
Contextual awareness service | Customized suggestions, personalized preferences, and convenient services |
Functions | Types of Air Gestures | Scenarios | |
---|---|---|---|
Screenshot or turn on/off | Fist clenching gesture | Taking screenshots on smart screens; turning on/off devices like smart range hoods or air conditioners; answering or hanging up phone calls, etc. | |
Pause or continue | Press gesture | Pausing or playing videos on smart screens; pausing or playing music on smart speakers; opening or closing smart wardrobes, etc. | |
Control up/down movements | Swipe up/down gesture | Scrolling pages up or down on smart screens; adjusting the height of smart range hoods, etc. | |
Control left/right movements | Swipe left/right gesture | Switching application pages, reading pages or image pages on smart screens; opening or closing curtains, etc. | |
Size adjustment | Pinch gesture | Adjusting the size of the display on smart screens; controlling the volume of smart speakers; adjusting the flame intensity on smart stoves; regulating the temperature of smart air conditioners, etc. | |
Adjustment; move; select | Finger swipe gesture | Adjusting the display size on smart screens; controlling the volume of smart speakers; adjusting the flame intensity on smart stoves; regulating the temperature of smart air conditioners, etc. |
Types of Non- Contact Sensors Used in Smart Homes | Working Principle | Transmission Structure |
---|---|---|
Optical image sensor (applied in scenarios such as video calls, security monitoring, gesture recognition, etc.) | ||
Passive infrared sensor (applied in scenarios such as those involving motion-sensing lights, intrusion alarms, fall detection, etc.) | ||
Active Electromagnetic Sensors (applied to fall detection, anti-theft doors and windows, human monitoring, smoke detection, and other scenarios) | ||
Evaluation Dimension | Experimental Group Mean | Control Group Mean | Experimental Group Variance | Control Group Variance | t Value | Freedom | p Value |
---|---|---|---|---|---|---|---|
Subjective anxiety level | 3.78 | 3.25 | 2.53 | 0.29 | 0.78 | 8.75 | 0.48 |
User interface usability | 4.55 | 3.00 | 0.13 | 0.00 | 6.85 | 10.67 | <0.01 |
Ease of operation | 2.89 | 3.67 | 2.83 | 0.50 | −0.96 | 10.00 | 0.36 |
Interactive experience of contextual awareness system | 4.67 | 3.00 | 0.02 | 0.00 | 11.24 | 10.00 | <0.01 |
Safety | 3.83 | 3.33 | 2.22 | 0.50 | 0.82 | 10.00 | 0.43 |
System response speed | 3.66 | 3.33 | 1.42 | 0.50 | 0.52 | 10.00 | 0.62 |
Overall satisfaction | 4.83 | 3.00 | 0.09 | 0.00 | 9.17 | 10.00 | <0.01 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Lu, Y.; Zhou, L.; Zhang, A.; Wang, M.; Zhang, S.; Wang, M. Research on Designing Context-Aware Interactive Experiences for Sustainable Aging-Friendly Smart Homes. Electronics 2024, 13, 3507. https://doi.org/10.3390/electronics13173507
Lu Y, Zhou L, Zhang A, Wang M, Zhang S, Wang M. Research on Designing Context-Aware Interactive Experiences for Sustainable Aging-Friendly Smart Homes. Electronics. 2024; 13(17):3507. https://doi.org/10.3390/electronics13173507
Chicago/Turabian StyleLu, Yi, Lejia Zhou, Aili Zhang, Mengyao Wang, Shan Zhang, and Minghua Wang. 2024. "Research on Designing Context-Aware Interactive Experiences for Sustainable Aging-Friendly Smart Homes" Electronics 13, no. 17: 3507. https://doi.org/10.3390/electronics13173507