Leveraging Off-the-Shelf WiFi for Contactless Activity Monitoring
Abstract
:1. Introduction
- We propose a method for achieving contactless human activity sensing by utilizing ubiquitous WiFi signals, which is non-intrusive, low-cost, and secure for the user.
- We have designed a novel metric based on kurtosis and standard deviation to select an optimal subcarrier set that is sensitive to all target activities from the candidate 30 subcarriers.
- We have implemented our system prototype with a COTS router and a desktop computer and conducted extensive experiments with 10 people over a period of one month. The results indicate the accuracy and robustness of our system.
2. Preliminaries
2.1. Channel State Information
2.2. Support Vector Machine
3. System Design
3.1. System Overview
3.2. Data Collection
3.3. Data Processing
3.3.1. Hampel Filter
3.3.2. Wavelet Filter
3.3.3. Moving Average Filter
3.3.4. Carrier Selection
3.4. Motion Recognition
3.4.1. Data Segments
3.4.2. Feature Extraction and Classification
4. Evaluation
4.1. Implementation
4.1.1. Hardware and Software Implementation
4.1.2. Environment and Participants
4.1.3. Performance Metrics
4.2. Overall Performance
4.3. Impacts of Different Factors
4.3.1. Impact of Transceiver Distance
4.3.2. Impact of Transceiver Height
4.3.3. Impact of Different Classifiers
5. Discussion
- Subtle movement. The proposed system only uses a low-cost COTS router as a transmitter, which limits its ability to sense subtle movements (e.g., heartbeat, respiration, and cough). We will try to deploy a high-gain directional antenna to enhance the power of the transmitted signal or some wearable and skin-attachable sensors to collect more fine-grained information in future experiments so that the proposed method can provide a higher sensing accuracy in recognizing human activity.
- Multiple users. The proposed system can only be applied to sense one person’s body movements at a time. The receiver will collect a mixed signal recording multiple users’ body movements if they perform activities simultaneously, and it is challenging to decouple the CSI signals of individual body movements from the mixed signal.
- Cross-domain. The performance of the proposed system relies on environment-related domains including position, orientation, and static objects. As such, a classifier model trained using data collected in a given environment may not perform well in a new environment. We will try to apply transfer learning or adversarial learning methodologies to improve the cross-domain sensing performance in future work.
6. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Kushida, C.A.; Littner, M.R.; Morgenthaler, T.; Alessi, C.A.; Bailey, D.; Coleman, J., Jr.; Friedman, L.; Hirshkowitz, M.; Kapen, S.; Kramer, M.; et al. Practice parameters for the indications for polysomnography and related procedures: An update for 2005. Sleep 2005, 28, 499–523. [Google Scholar] [CrossRef] [PubMed]
- Aurora, R.N.; Zak, R.S.; Karippot, A.; Lamm, C.I.; Morgenthaler, T.I.; Auerbach, S.H.; Bista, S.R.; Casey, K.R.; Chowdhuri, S.; Kristo, D.A.; et al. Practice parameters for the respiratory indications for polysomnography in children. Sleep 2011, 34, 379–388. [Google Scholar] [CrossRef]
- Zhang, S.; McCullagh, P.; Zhang, J.; Yu, T. A smartphone based real-time daily activity monitoring system. Clust. Comput. 2014, 17, 711–721. [Google Scholar] [CrossRef]
- Qi, W.; Su, H.; Aliverti, A. A smartphone-based adaptive recognition and real-time monitoring system for human activities. IEEE Trans. Hum.-Mach. Syst. 2020, 50, 414–423. [Google Scholar] [CrossRef]
- Ramanujam, E.; Perumal, T.; Padmavathi, S. Human activity recognition with smartphone and wearable sensors using deep learning techniques: A review. IEEE Sens. J. 2021, 21, 13029–13040. [Google Scholar] [CrossRef]
- Zhuang, Z.; Xue, Y. Sport-related human activity detection and recognition using a smartwatch. Sensors 2019, 19, 5001. [Google Scholar] [CrossRef]
- Ashry, S.; Ogawa, T.; Gomaa, W. CHARM-deep: Continuous human activity recognition model based on deep neural network using IMU sensors of smartwatch. IEEE Sens. J. 2020, 20, 8757–8770. [Google Scholar] [CrossRef]
- Boyer, P.; Burns, D.; Whyne, C. Out-of-distribution detection of human activity recognition with smartwatch inertial sensors. Sensors 2021, 21, 1669. [Google Scholar] [CrossRef]
- Jahan, I.; Al-Nabhan, N.A.; Noor, J.; Rahaman, M.; Al Islam, A.A. Leveraging A Smartwatch for Activity Recognition in Salat. IEEE Access 2023, 11, 97284–97317. [Google Scholar] [CrossRef]
- Fang, B.; Lane, N.D.; Zhang, M.; Boran, A.; Kawsar, F. BodyScan: Enabling radio-based sensing on wearable devices for contactless activity and vital sign monitoring. In Proceedings of the 14th Annual International Conference on Mobile Systems, Applications, and Services, Singapore, 26–30 June 2016; pp. 97–110. [Google Scholar]
- Fang, B.; Lane, N.D.; Zhang, M.; Kawsar, F. Headscan: A wearable system for radio-based sensing of head and mouth-related activities. In Proceedings of the 2016 15th ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN), Vienna, Austria, 11–14 April 2016; pp. 1–12. [Google Scholar]
- Wang, T.; Zhang, D.; Wang, L.; Zheng, Y.; Gu, T.; Dorizzi, B.; Zhou, X. Contactless respiration monitoring using ultrasound signal with off-the-shelf audio devices. IEEE Internet Things J. 2018, 6, 2959–2973. [Google Scholar] [CrossRef]
- Chen, Y.; Gong, W.; Liu, J.; Cui, Y. I can hear more: Pushing the limit of ultrasound sensing on off-the-shelf mobile devices. In Proceedings of the IEEE INFOCOM 2018-IEEE Conference on Computer Communications, Honolulu, HI, USA, 16–19 April 2018; pp. 2015–2023. [Google Scholar]
- Zhai, S.; Ye, G.; Tang, Z.; Ren, J.; Fang, D.; Liu, B.; Wang, Z. Towards practical 3D ultrasound sensing on commercial-off-the-shelf mobile devices. Comput. Netw. 2021, 191, 107990. [Google Scholar] [CrossRef]
- Zhang, S.; Wei, Z.; Nie, J.; Huang, L.; Wang, S.; Li, Z. A review on human activity recognition using vision-based method. J. Healthc. Eng. 2017, 2017, 3090343. [Google Scholar] [CrossRef]
- Beddiar, D.R.; Nini, B.; Sabokrou, M.; Hadid, A. Vision-based human activity recognition: A survey. Multimed. Tools Appl. 2020, 79, 30509–30555. [Google Scholar] [CrossRef]
- Qi, W.; Wang, N.; Su, H.; Aliverti, A. DCNN based human activity recognition framework with depth vision guiding. Neurocomputing 2022, 486, 261–271. [Google Scholar] [CrossRef]
- Wang, X.; Wu, Z.; Jiang, B.; Bao, Z.; Zhu, L.; Li, G.; Wang, Y.; Tian, Y. Hardvs: Revisiting human activity recognition with dynamic vision sensors. In Proceedings of the AAAI Conference on Artificial Intelligence, Vancouver, BC, Canada, 20–28 February 2024; Volume 38, pp. 5615–5623. [Google Scholar]
- Gu, F.; Chung, M.H.; Chignell, M.; Valaee, S.; Zhou, B.; Liu, X. A survey on deep learning for human activity recognition. ACM Comput. Surv. (CSUR) 2021, 54, 1–34. [Google Scholar] [CrossRef]
- Li, X.; He, Y.; Jing, X. A survey of deep learning-based human activity recognition in radar. Remote Sens. 2019, 11, 1068. [Google Scholar] [CrossRef]
- Li, X.; He, Y.; Fioranelli, F.; Jing, X. Semisupervised human activity recognition with radar micro-Doppler signatures. IEEE Trans. Geosci. Remote Sens. 2021, 60, 1–12. [Google Scholar] [CrossRef]
- Cao, Z.; Li, Z.; Guo, X.; Wang, G. Towards cross-environment human activity recognition based on radar without source data. IEEE Trans. Veh. Technol. 2021, 70, 11843–11854. [Google Scholar] [CrossRef]
- Yu, C.; Xu, Z.; Yan, K.; Chien, Y.R.; Fang, S.H.; Wu, H.C. Noninvasive human activity recognition using millimeter-wave radar. IEEE Syst. J. 2022, 16, 3036–3047. [Google Scholar] [CrossRef]
- Alhazmi, A.K.; Alanazi, M.A.; Alshehry, A.H.; Alshahry, S.M.; Jaszek, J.; Djukic, C.; Brown, A.; Jackson, K.; Chodavarapu, V.P. Intelligent Millimeter-Wave System for Human Activity Monitoring for Telemedicine. Sensors 2024, 24, 268. [Google Scholar] [CrossRef]
- Huang, T.; Liu, G.; Li, S.; Liu, J. RPCRS: Human Activity Recognition Using Millimeter Wave Radar. In Proceedings of the 2022 IEEE 28th International Conference on Parallel and Distributed Systems (ICPADS), Nanjing, China, 10–12 January 2023; pp. 122–129. [Google Scholar]
- Yin, W.; Shi, L.F.; Shi, Y. Indoor human action recognition based on millimeter-wave radar micro-doppler signature. Measurement 2024, 235, 114939. [Google Scholar] [CrossRef]
- Khan, M.B.; Yang, X.; Ren, A.; Al-Hababi, M.A.M.; Zhao, N.; Guan, L.; Fan, D.; Shah, S.A. Design of software defined radios based platform for activity recognition. IEEE Access 2019, 7, 31083–31088. [Google Scholar] [CrossRef]
- Ashleibta, A.M.; Zahid, A.; Shah, S.A.; Abbasi, Q.H.; Imran, M.A. Flexible and scalable software defined radio based testbed for large scale body movement. Electronics 2020, 9, 1354. [Google Scholar] [CrossRef]
- Saeed, U.; Shah, S.A.; Khan, M.Z.; Alotaibi, A.A.; Althobaiti, T.; Ramzan, N.; Imran, M.A.; Abbasi, Q.H. Software-defined radio based contactless localization for diverse human activity recognition. IEEE Sens. J. 2023, 23, 12041–12048. [Google Scholar] [CrossRef]
- Wang, Y.; Zheng, Y. Modeling RFID signal reflection for contact-free activity recognition. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 2018, 2, 1–22. [Google Scholar] [CrossRef]
- Liu, C.; Xiong, J.; Cai, L.; Feng, L.; Chen, X.; Fang, D. Beyond respiration: Contactless sleep sound-activity recognition using RF signals. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 2019, 3, 1–22. [Google Scholar] [CrossRef]
- He, X.; Zhu, J.; Su, W.; Tentzeris, M.M. RFID based non-contact human activity detection exploiting cross polarization. IEEE Access 2020, 8, 46585–46595. [Google Scholar] [CrossRef]
- Wang, F.; Liu, J.; Gong, W. Multi-adversarial in-car activity recognition using RFIDs. IEEE Trans. Mob. Comput. 2020, 20, 2224–2237. [Google Scholar] [CrossRef]
- Colella, R.; Tumolo, M.R.; Sabina, S.; Leo, C.G.; Mincarone, P.; Guarino, R.; Catarinucci, L. Design of UHF RFID sensor-tags for the biomechanical analysis of human body movements. IEEE Sens. J. 2021, 21, 14090–14098. [Google Scholar] [CrossRef]
- Guo, X.; Liu, J.; Shi, C.; Liu, H.; Chen, Y.; Chuah, M.C. Device-free Personalized Fitness Assistant Using WiFi. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 2018, 2, 1–23. [Google Scholar] [CrossRef]
- Halperin, D.; Hu, W.; Sheth, A.; Wetherall, D. Tool release: Gathering 802.11 n traces with channel state information. ACM SIGCOMM Comput. Commun. Rev. 2011, 41, 53. [Google Scholar] [CrossRef]
Time-Domain Features |
---|
max, min, mean, variance, standard deviation, max–min, kurtosis, skewness, mean crossing rate |
Frequency-Domain Features |
energy, frequency, sensitivity |
Classifier | ||||
---|---|---|---|---|
Metric (%) | SVM | k-NN | DT | RF |
Accuracy | 95.7 | 92.8 | 91.6 | 94.3 |
F1-score | 96 | 92.3 | 92.1 | 94.8 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Zhu, Z.; Liu, W.; Zhang, H.; Lu, J. Leveraging Off-the-Shelf WiFi for Contactless Activity Monitoring. Electronics 2024, 13, 3351. https://doi.org/10.3390/electronics13173351
Zhu Z, Liu W, Zhang H, Lu J. Leveraging Off-the-Shelf WiFi for Contactless Activity Monitoring. Electronics. 2024; 13(17):3351. https://doi.org/10.3390/electronics13173351
Chicago/Turabian StyleZhu, Zixuan, Wei Liu, Hao Zhang, and Jinhu Lu. 2024. "Leveraging Off-the-Shelf WiFi for Contactless Activity Monitoring" Electronics 13, no. 17: 3351. https://doi.org/10.3390/electronics13173351