Overcoming Challenges in Video-Based Health Monitoring: Real-World Implementation, Ethics, and Data Considerations
Abstract
:1. Introduction
1.1. Challenges in Data Collection
1.2. Challenges in Processing Video Data
1.3. Ethical Challenges
2. Pilot in Knowledge Workers
2.1. Participant Demographics
2.2. Methods of Data Acquisition and Analysis
3. Results
4. Discussion
5. Opportunities in Video-Based Health Monitoring
6. Conclusions and Future Considerations
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Awad, A.; Trenfield, S.J.; Pollard, T.D.; Ong, J.J.; Elbadawi, M.; McCoubrey, L.E.; Goyanes, A.; Gaisford, S.; Basit, A.W. Connected healthcare: Improving patient care using digital health technologies. Adv. Drug Deliv. Rev. 2021, 178, 113958. [Google Scholar] [CrossRef] [PubMed]
- Guo, Y.; Liu, X.; Peng, S.; Jiang, X.; Xu, K.; Chen, C.; Wang, Z.; Dai, C.; Chen, W. A review of wearable and unobtrusive sensing technologies for chronic disease management. Comput. Biol. Med. 2021, 129, 104163. [Google Scholar] [CrossRef] [PubMed]
- Javaid, M.; Haleem, A.; Pratap Singh, R.; Suman, R.; Rab, S. Significance of machine learning in healthcare: Features, pillars and applications. Int. J. Intell. Netw. 2022, 3, 58–73. [Google Scholar] [CrossRef]
- Jagatheesaperumal, S.K.; Rajkumar, S.; Suresh, J.V.; Gumaei, A.H.; Alhakbani, N.; Uddin, M.Z.; Hassan, M.M. An iot-based framework for personalized health assessment and recommendations using machine learning. Mathematics 2023, 11, 2758. [Google Scholar] [CrossRef]
- El-Sherif, D.M.; Abouzid, M.; Elzarif, M.T.; Ahmed, A.A.; Albakri, A.; Alshehri, M.M. Telehealth and Artificial Intelligence Insights into Healthcare during the COVID-19 Pandemic. Healthcare 2022, 10, 385. [Google Scholar] [CrossRef]
- Snoswell, C.L.; Chelberg, G.; De Guzman, K.R.; Haydon, H.H.; Thomas, E.E.; Caffery, L.J.; Smith, A.C. The clinical effectiveness of telehealth: A systematic review of meta-analyses from 2010 to 2019. J. Telemed. Telecare 2021, 29, 669–684. [Google Scholar] [CrossRef]
- Houlding, E.; Mate, K.K.V.; Engler, K.; Ortiz-Paredes, D.; Pomey, M.P.; Cox, J.; Hijal, T.; Lebouché, B. Barriers to Use of Remote Monitoring Technologies Used to Support Patients With COVID-19: Rapid Review. JMIR mHealth uHealth 2021, 9, e24743. [Google Scholar] [CrossRef]
- Shaik, T.; Tao, X.; Higgins, N.; Li, L.; Gururajan, R.; Zhou, X.; Acharya, U.R. Remote patient monitoring using artificial intelligence: Current state, applications, and challenges. WIREs Data Min. Knowl. Discov. 2023, 13, e1485. [Google Scholar] [CrossRef]
- Rohmetra, H.; Raghunath, N.; Narang, P.; Chamola, V.; Guizani, M.; Lakkaniga, N.R. AI-enabled remote monitoring of vital signs for COVID-19: Methods, prospects and challenges. Computing 2023, 105, 783–809. [Google Scholar] [CrossRef]
- Bousefsaf, F.; Pruski, A.; Maaoui, C. 3D Convolutional Neural Networks for Remote Pulse Rate Measurement and Mapping from Facial Video. Appl. Sci. 2019, 9, 4364. [Google Scholar] [CrossRef]
- Cho, Y.; Bianchi-Berthouze, N.; Julier, S.J. DeepBreath: Deep learning of breathing patterns for automatic stress recognition using low-cost thermal imaging in unconstrained settings. In Proceedings of the 2017 Seventh International Conference on Affective Computing and Intelligent Interaction (ACII), San Antonio, TX, USA, 23–26 October 2017; pp. 456–463. [Google Scholar]
- Khalid, W.B.; Anwar, A.; Waheed, O.T. Contactless Vitals Measurement Robot. In Proceedings of the 2022 8th International Conference on Automation, Robotics and Applications (ICARA), Prague, Czech Republic, 18–20 February 2022; pp. 91–96. [Google Scholar]
- Laurie, J.; Higgins, N.; Peynot, T.; Fawcett, L.; Roberts, J. An evaluation of a video magnification-based system for respiratory rate monitoring in an acute mental health setting. Int. J. Med. Inform. 2021, 148, 104378. [Google Scholar] [CrossRef] [PubMed]
- Mahesh, V.G.V.; Chen, C.; Rajangam, V.; Raj, A.N.J.; Krishnan, P.T. Shape and Texture Aware Facial Expression Recognition Using Spatial Pyramid Zernike Moments and Law’s Textures Feature Set. IEEE Access 2021, 9, 52509–52522. [Google Scholar] [CrossRef]
- Zainuddin, A.A.; Superamaniam, S.; Andrew, A.C.; Muraleedharan, R.; Rakshys, J.; Miriam, J.; Bostomi, M.A.S.M.; Rais, A.M.A.; Khalidin, Z.; Mansor, A.F.; et al. Patient Monitoring System using Computer Vision for Emotional Recognition and Vital Signs Detection. In Proceedings of the 2020 IEEE Student Conference on Research and Development (SCOReD), Batu Pahat, Malaysia, 27–29 September 2020; pp. 22–27. [Google Scholar]
- Chowdary, M.K.; Nguyen, T.N.; Hemanth, D.J. Deep learning-based facial emotion recognition for human–computer interaction applications. Neural Comput. Appl. 2023, 35, 23311–23328. [Google Scholar] [CrossRef]
- Xu, Q.; Liu, X.; Luo, J.; Tang, Z. Emotion monitoring with RFID: An experimental study. CCF Trans. Pervasive Comput. Interact. 2020, 2, 299–313. [Google Scholar] [CrossRef]
- Ferreira, S.; Rodrigues, F.; Kallio, J.; Coelho, F.; Kyllonen, V.; Rocha, N.; Rodrigues, M.A.; Vildjiounaite, E. From Controlled to Chaotic: Disparities in Laboratory vs Real-World Stress Detection. In Proceedings of the 2024 International Conference on Content-Based Multimedia Indexing (CBMI), Reykjavik, Iceland, 18–20 September 2024; pp. 1–7. [Google Scholar]
- Selvaraju, V.; Spicher, N.; Wang, J.; Ganapathy, N.; Warnecke, J.M.; Leonhardt, S.; Swaminathan, R.; Deserno, T.M. Continuous Monitoring of Vital Signs Using Cameras: A Systematic Review. Sensors 2022, 22, 4097. [Google Scholar] [CrossRef]
- Aldoseri, A.; Al-Khalifa, K.N.; Hamouda, A.M. Re-Thinking Data Strategy and Integration for Artificial Intelligence: Concepts, Opportunities, and Challenges. Appl. Sci. 2023, 13, 7082. [Google Scholar] [CrossRef]
- Qian, C.; Marques, J.A.L.; Fong, S.J. Analysis of deep learning algorithms for emotion classification based on facial expression recognition. In Proceedings of the 2024 8th International Conference on Big Data and Internet of Things, Macau, China, 14–16 September 2024; Association for Computing Machinery: New York, NY, USA, 2024; pp. 161–167. [Google Scholar]
- Fabbrizzi, S.; Papadopoulos, S.; Ntoutsi, E.; Kompatsiaris, I. A survey on bias in visual datasets. Comput. Vision. Image Underst. 2022, 223, 103552. [Google Scholar] [CrossRef]
- Karpathy, A.; Toderici, G.; Shetty, S.; Leung, T.; Sukthankar, R.; Fei-Fei, L. Large-Scale Video Classification with Convolutional Neural Networks. In Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA, 23–28 June 2014; pp. 1725–1732. [Google Scholar]
- Phan, T.-C.; Phan, A.-C.; Cao, H.-P.; Trieu, T.-N. Content-Based Video Big Data Retrieval with Extensive Features and Deep Learning. Appl. Sci. 2022, 12, 6753. [Google Scholar] [CrossRef]
- Zhu, H.; Wei, H.; Li, B.; Yuan, X.; Kehtarnavaz, N. A Review of Video Object Detection: Datasets, Metrics and Methods. Appl. Sci. 2020, 10, 7834. [Google Scholar] [CrossRef]
- Sopidis, G.; Haslgrübler, M.; Ferscha, A. Counting Activities Using Weakly Labeled Raw Acceleration Data: A Variable-Length Sequence Approach with Deep Learning to Maintain Event Duration Flexibility. Sensors 2023, 23, 5057. [Google Scholar] [CrossRef]
- Naseer, M.; Prabakaran, B.S.; Hasan, O.; Shafique, M. UnbiasedNets: A dataset diversification framework for robustness bias alleviation in neural networks. Mach. Learn. 2023, 113, 2499–2526. [Google Scholar] [CrossRef]
- Pagano, T.P.; Loureiro, R.B.; Lisboa, F.V.N.; Peixoto, R.M.; Guimarães, G.A.S.; Cruz, G.O.R.; Araujo, M.M.; Santos, L.L.; Cruz, M.A.S.; Oliveira, E.L.S.; et al. Bias and Unfairness in Machine Learning Models: A Systematic Review on Datasets, Tools, Fairness Metrics, and Identification and Mitigation Methods. Big Data Cogn. Comput. 2023, 7, 15. [Google Scholar] [CrossRef]
- Shahbazi, N.; Lin, Y.; Asudeh, A.; Jagadish, H.V. Representation Bias in Data: A Survey on Identification and Resolution Techniques. ACM Comput. Surv. 2023, 55, 293. [Google Scholar] [CrossRef]
- Wilson, B.; Hoffman, J.; Morgenstern, J. Predictive Inequity in Object Detection. arXiv 2019, arXiv:1902.11097. [Google Scholar]
- Molinaro, N.; Schena, E.; Silvestri, S.; Bonotti, F.; Aguzzi, D.; Viola, E.; Buccolini, F.; Massaroni, C. Contactless Vital Signs Monitoring From Videos Recorded With Digital Cameras: An Overview. Front. Physiol. 2022, 13, 801709. [Google Scholar] [CrossRef]
- Hassan, M.A.; Malik, A.S.; Fofi, D.; Saad, N.; Meriaudeau, F. Novel health monitoring method using an RGB camera. Biomed. Opt. Express 2017, 8, 4838–4854. [Google Scholar] [CrossRef]
- Hadgraft, N.T.; Healy, G.N.; Owen, N.; Winkler, E.A.H.; Lynch, B.M.; Sethi, P.; Eakin, E.G.; Moodie, M.; LaMontagne, A.D.; Wiesner, G.; et al. Office workers’ objectively assessed total and prolonged sitting time: Individual-level correlates and worksite variations. Prev. Med. Rep. 2016, 4, 184–191. [Google Scholar] [CrossRef]
- Niven, A.; Baker, G.; Almeida, E.C.; Fawkner, S.G.; Jepson, R.; Manner, J.; Morton, S.; Nightingale, G.; Sivaramakrishnan, D.; Fitzsimons, C. “Are We Working (Too) Comfortably?”: Understanding the Nature of and Factors Associated with Sedentary Behaviour When Working in the Home Environment. Occup. Health Sci. 2023, 7, 71–88. [Google Scholar] [CrossRef]
- Tobien, P.; Lischke, L.; Hirsch, M.; Krüger, R.; Lukowicz, P.; Schmidt, A. Engaging people to participate in data collection. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct, Association for Computing Machinery, Heidelberg, Germany, 12–16 September 2016; pp. 209–212. [Google Scholar]
- Verma, J.; Agrawal, S. Big Data Analytics: Challenges And Applications For Text, Audio, Video, And Social Media Data. Int. J. Soft Comput. Artif. Intell. Appl. 2016, 5, 41–51. [Google Scholar] [CrossRef]
- Marx, V. The big challenges of big data. Nature 2013, 498, 255–260. [Google Scholar] [CrossRef]
- Rajpoot, Q.; Jensen, C.D. Video Surveillance: Privacy Issues and Legal Compliance. In Promoting Social Change and Democracy through Information Technology; IGI Global: Hershey, PA, USA, 2015; pp. 69–92. [Google Scholar]
- Golda, T.; Guaia, D.; Wagner-Hartl, V. Perception of Risks and Usefulness of Smart Video Surveillance Systems. Appl. Sci. 2022, 12, 10435. [Google Scholar] [CrossRef]
- Vijayakumar, V.; Nedunchezhian, R. A study on video data mining. Int. J. Multimed. Inf. Retr. 2012, 1, 153–172. [Google Scholar] [CrossRef]
- Vildjiounaite, E.; Kallio, J.; Kantorovitch, J.; Kinnula, A.; Ferreira, S.; Rodrigues, M.A.; Rocha, N. Challenges of learning human digital twin: Case study of mental wellbeing: Using sensor data and machine learning to create HDT. In Proceedings of the 16th International Conference on PErvasive Technologies Related to Assistive Environments, Association for Computing Machinery, Corfu, Greece, 5–7 July 2023; pp. 574–583. [Google Scholar]
- Hong, T.; Stauffer, W.R. Computational complexity drives sustained deliberation. Nat. Neurosci. 2023, 26, 850–857. [Google Scholar] [CrossRef]
- Bidwe, R.V.; Mishra, S.; Patil, S.; Shaw, K.; Vora, D.R.; Kotecha, K.; Zope, B. Deep Learning Approaches for Video Compression: A Bibliometric Analysis. Big Data Cogn. Comput. 2022, 6, 44. [Google Scholar] [CrossRef]
- Szegedy, C.; Wei, L.; Yangqing, J.; Sermanet, P.; Reed, S.; Anguelov, D.; Erhan, D.; Vanhoucke, V.; Rabinovich, A. Going deeper with convolutions. In Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA, 7–12 June 2015; pp. 1–9. [Google Scholar]
- Ferreira, S.; Rodrigues, M.; Rocha, N. Validation of a Video-based System to Determine Heart Rate for Stress Monitoring. In Safety Management and Human Factors; AHFE: Orlando, FL, USA, 2022. [Google Scholar]
- Addison, P.; Jacquel, D.; Foo, D.; Borg, U. Video-based heart rate monitoring across a range of skin pigmentations during an acute hypoxic challenge. J. Clin. Monit. Comput. 2018, 32, 871–880. [Google Scholar] [CrossRef]
- Liu, S.; Andrienko, G.; Wu, Y.; Cao, N.; Jiang, L.; Shi, C.; Wang, Y.-S.; Hong, S. Steering data quality with visual analytics: The complexity challenge. Vis. Inform. 2018, 2, 191–197. [Google Scholar] [CrossRef]
- Haidet, K.K.; Tate, J.; Divirgilio-Thomas, D.; Kolanowski, A.; Happ, M.B. Methods to improve reliability of video-recorded behavioral data. Res. Nurs. Health 2009, 32, 465–474. [Google Scholar] [CrossRef]
- Alzubaidi, L.; Bai, J.; Al-Sabaawi, A.; Santamaría, J.; Albahri, A.S.; Al-dabbagh, B.S.N.; Fadhel, M.A.; Manoufali, M.; Zhang, J.; Al-Timemy, A.H.; et al. A survey on deep learning tools dealing with data scarcity: Definitions, challenges, solutions, tips, and applications. J. Big Data 2023, 10, 46. [Google Scholar] [CrossRef]
- Lapa, I.; Ferreira, S.; Mateus, C.; Rocha, N.; Rodrigues, M.A. Real-Time Blink Detection as an Indicator of Computer Vision Syndrome in Real-Life Settings: An Exploratory Study. Int. J. Environ. Res. Public Health 2023, 20, 4569. [Google Scholar] [CrossRef]
- Hassan, M.A.; Malik, A.S.; Fofi, D.; Karasfi, B.; Meriaudeau, F. Towards health monitoring using remote heart rate measurement using digital camera: A feasibility study. Measurement 2020, 149, 106804. [Google Scholar] [CrossRef]
- Jayalakshmi, M.; Gomathi, V. Pervasive health monitoring through video-based activity information integrated with sensor-cloud oriented context-aware decision support system. Multimed. Tools Appl. 2020, 79, 3699–3712. [Google Scholar] [CrossRef]
- Del-Valle-Soto, C.; Briseño, R.A.; Velázquez, R.; Guerra-Rosales, G.; Perez-Ochoa, S.; Preciado-Bazavilvazo, I.H.; Visconti, P.; Varela-Aldás, J. Enhancing Elderly Care through Low-Cost Wireless Sensor Networks and Artificial Intelligence: A Study on Vital Sign Monitoring and Sleep Improvement. Future Internet 2024, 16, 323. [Google Scholar] [CrossRef]
- Carneiro, D.; Novais, P.; Augusto, J.C.; Payne, N. New Methods for Stress Assessment and Monitoring at the Workplace. IEEE Trans. Affect. Comput. 2019, 10, 237–254. [Google Scholar] [CrossRef]
- Keenan, A.J.; Tsourtos, G.; Tieman, J. The Value of Applying Ethical Principles in Telehealth Practices: Systematic Review. J. Med. Internet Res. 2021, 23, e25698. [Google Scholar] [CrossRef] [PubMed]
- Cai, Y.; Yu, F.; Kumar, M.; Gladney, R.; Mostafa, J. Health Recommender Systems Development, Usage, and Evaluation from 2010 to 2022: A Scoping Review. Int. J. Environ. Res. Public Health 2022, 19, 15115. [Google Scholar] [CrossRef]
- Lopes, L.; Rodrigues, A.; Cabral, D.; Campos, P. From Monitoring to Assisting: A Systematic Review towards Healthier Workplaces. Int. J. Environ. Res. Public Health 2022, 19, 16197. [Google Scholar] [CrossRef]
- De, R.; Pandey, N.; Pal, A. Impact of digital surge during Covid-19 pandemic: A viewpoint on research and practice. Int. J. Inf. Manag. 2020, 55, 102171. [Google Scholar]
- Boikanyo, K.; Zungeru, A.M.; Sigweni, B.; Yahya, A.; Lebekwe, C. Remote patient monitoring systems: Applications, architecture, and challenges. Sci. Afr. 2023, 20, e01638. [Google Scholar] [CrossRef]
- Kallio, J.; Vildjiounaite, E.; Kantorovitch, J.; Kinnula, A.; Bordallo López, M. Unobtrusive Continuous Stress Detection in Knowledge Work—Statistical Analysis on User Acceptance. Sustainability 2021, 13, 2003. [Google Scholar] [CrossRef]
- European Parliament. Regulation (EU) 2016/679 of the European Parliament and of the Council; Regulation (eu): Brussels, Belgium, 2016; Volume 679. [Google Scholar]
- Cohen, I.G.; Gerke, S.; Kramer, D.B. Ethical and Legal Implications of Remote Monitoring of Medical Devices. Milbank Q. 2020, 98, 1257–1289. [Google Scholar] [CrossRef]
- Krajčík, M.; Schmidt, D.A.; Baráth, M. Hybrid Work Model: An Approach to Work–Life Flexibility in a Changing Environment. Adm. Sci. 2023, 13, 150. [Google Scholar] [CrossRef]
- Wang, C.; He, T.; Zhou, H.; Zhang, Z.; Lee, C. Artificial intelligence enhanced sensors—Enabling technologies to next-generation healthcare and biomedical platform. Bioelectron. Med. 2023, 9, 17. [Google Scholar] [CrossRef] [PubMed]
- Kyriakou, K.; Resch, B.; Sagl, G.; Petutschnig, A.; Werner, C.; Niederseer, D.; Liedlgruber, M.; Wilhelm, F.; Osborne, T.; Pykett, J. Detecting Moments of Stress from Measurements of Wearable Physiological Sensors. Sensors 2019, 19, 3805. [Google Scholar] [CrossRef] [PubMed]
Monitoring Method | Advantages | Disadvantages | Effective/Real Impact |
---|---|---|---|
Video-based (AI-enhanced) | Non-invasive, does not require wearables, real-time tracking of facial features | Sensitive to lighting conditions, requires computational power, potential privacy concerns | Provides continuous, unobtrusive physiological monitoring, enabling early stress detection, fatigue assessment, and workplace health insights without requiring physical devices. Effective in scenarios where wearables may be impractical. |
Wearable devices (e.g., smartwatches, biosensors) | High accuracy, continuous physiological monitoring | Requires user compliance, intrusive for some users, battery life limitations | Highly effective for real-time tracking of heart rate and activity levels, particularly for mobile users, athletes, and clinical monitoring, but is highly dependent on the user. |
Clinical methods (e.g., ECG, polysomnography) | Gold standard for accuracy | Expensive, requires physical presence, not scalable for daily monitoring | Most reliable for diagnosing cardiovascular conditions and sleep disorders, but impractical for continuous long-term monitoring outside a clinical setting. |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Ferreira, S.; Marinheiro, C.; Mateus, C.; Rodrigues, P.P.; Rodrigues, M.A.; Rocha, N. Overcoming Challenges in Video-Based Health Monitoring: Real-World Implementation, Ethics, and Data Considerations. Sensors 2025, 25, 1357. https://doi.org/10.3390/s25051357
Ferreira S, Marinheiro C, Mateus C, Rodrigues PP, Rodrigues MA, Rocha N. Overcoming Challenges in Video-Based Health Monitoring: Real-World Implementation, Ethics, and Data Considerations. Sensors. 2025; 25(5):1357. https://doi.org/10.3390/s25051357
Chicago/Turabian StyleFerreira, Simão, Catarina Marinheiro, Catarina Mateus, Pedro Pereira Rodrigues, Matilde A. Rodrigues, and Nuno Rocha. 2025. "Overcoming Challenges in Video-Based Health Monitoring: Real-World Implementation, Ethics, and Data Considerations" Sensors 25, no. 5: 1357. https://doi.org/10.3390/s25051357
APA StyleFerreira, S., Marinheiro, C., Mateus, C., Rodrigues, P. P., Rodrigues, M. A., & Rocha, N. (2025). Overcoming Challenges in Video-Based Health Monitoring: Real-World Implementation, Ethics, and Data Considerations. Sensors, 25(5), 1357. https://doi.org/10.3390/s25051357