Evaluating the Effectiveness of Complexity Features of Eye Movement on Computer Activities Detection
Abstract
:1. Introduction
1.1. Significance of Computer Activity Recognition on Healthcare Issues
1.2. Eye-Movement Complexity Features for Activity Detection Models
1.3. Research Objectives
2. Materials and Methods
2.1. Participant, Apparatus, and Materials
2.2. Experimental Setting and Task
2.3. Data Preprocessing and Feature Selections
2.4. AI Modelling for Computer Activities Detection
3. Results
3.1. Important Eye-Movement Features Screened Using ANOVA
3.2. Computer Activities Detection Models Performances
4. Discussion
4.1. Roles of Screened Important Features to Help AI for Distinguishing the Computer Activities
4.2. Complexity Eye-Movement Features Potency for AI Modelling
4.3. Contribution, Possible Applications in Healthcare, and Future Research
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Acknowledgments
Conflicts of Interest
References
- Wang, Y.; Cang, S.; Yu, H. A survey on wearable sensor modality centred human activity recognition in health care. Expert Syst. Appl. 2019, 137, 167–190. [Google Scholar] [CrossRef]
- Nasiri, S.; Khosravani, M.R. Progress and challenges in fabrication of wearable sensors for health monitoring. Sens. Actuators A Phys. 2020, 312, 112105. [Google Scholar] [CrossRef]
- Li, Y.; Zheng, L.; Wang, X. Flexible and wearable healthcare sensors for visual reality health-monitoring. Virtual Real. Intell. Hardw. 2019, 1, 411–427. [Google Scholar] [CrossRef]
- Avci, A.; Bosch, S.; Marin-Perianu, M.; Marin-Perianu, R.; Havinga, P. Activity recognition using inertial sensing for healthcare, wellbeing and sports applications: A survey. In Proceedings of the 23th International Conference on Architecture of Computing Systems 2010, Hannover, Germany, 22–23 February 2010; pp. 1–10. [Google Scholar]
- Levanon, Y.; Gefen, A.; Lerman, Y.; Givon, U.; Ratzon, N.Z. Reducing musculoskeletal disorders among computer operators: Comparison between ergonomics interventions at the workplace. Ergonomics 2012, 55, 1571–1585. [Google Scholar] [CrossRef] [PubMed]
- Donoghue, M.F.; O’Reilly, D.S.; Walsh, M.T. Wrist postures in the general population of computer users during a computer task. Appl. Ergon. 2013, 44, 42–47. [Google Scholar] [CrossRef]
- Randolph, S.A. Computer vision syndrome. Workplace Health Saf. 2017, 65, 328. [Google Scholar] [CrossRef]
- Destyanto, T.Y.R.; Lin, R.F. Detecting computer activities using eye-movement features. J. Ambient. Intell. Humaniz. Comput. 2020, 1–11. [Google Scholar] [CrossRef]
- Bulling, A.; Ward, J.A.; Gellersen, H.; Tröster, G. Eye movement analysis for activity recognition. In Proceedings of the 11th International Conference on Ubiquitous Computing, Orlando, FL, USA, 30 September–3 October 2009; pp. 41–50. [Google Scholar]
- Nweke, H.F.; Teh, Y.W.; Al-Garadi, M.A.; Alo, U.R. Deep learning algorithms for human activity recognition using mobile and wearable sensor networks: State of the art and research challenges. Expert Syst. Appl. 2018, 105, 233–261. [Google Scholar] [CrossRef]
- Costa, M.; Goldberger, A.L.; Peng, C.-K. Multiscale entropy analysis of biological signals. Phys. Rev. E 2005, 71, 021906. [Google Scholar] [CrossRef] [Green Version]
- Schleicher, R.; Galley, N.; Briest, S.; Galley, L. Blinks and saccades as indicators of fatigue in sleepiness warnings: Looking tired? Ergonomics 2008, 51, 982–1010. [Google Scholar] [CrossRef]
- Yamada, Y.; Kobayashi, M. Detecting mental fatigue from eye-tracking data gathered while watching video: Evaluation in younger and older adults. Artif. Intell. Med. 2018, 91, 39–48. [Google Scholar] [CrossRef] [PubMed]
- Carver, R.P. Pupil dilation and its relationship to information processing during reading and listening. J. Appl. Psychol. 1971, 55, 126. [Google Scholar] [CrossRef] [PubMed]
- Smith, T.J.; Whitwell, M.; Lee, J. Eye movements and pupil dilation during event perception. In Proceedings of the 2006 Symposium on Eye Tracking Research & Applications, San Diego, CA, USA, 27–29 March 2006; p. 48. [Google Scholar]
- Yan, C.; Li, P.; Yang, M.; Li, Y.; Li, J.; Zhang, H.; Liu, C. Entropy Analysis of Heart Rate Variability in Different Sleep Stages. Entropy 2022, 24, 379. [Google Scholar] [CrossRef] [PubMed]
- Chacón, M.; Rojas-Pescio, H.; Peñaloza, S.; Landerretche, J. Machine Learning Models and Statistical Complexity to Analyze the Effects of Posture on Cerebral Hemodynamics. Entropy 2022, 24, 428. [Google Scholar] [CrossRef]
- Liu, W.-M.; Liu, H.-R.; Chen, P.-W.; Chang, H.-R.; Liao, C.-M.; Liu, A.-B. Novel Application of Multiscale Cross-Approximate Entropy for Assessing Early Changes in the Complexity between Systolic Blood Pressure and ECG RR Intervals in Diabetic Rats. Entropy 2022, 24, 473. [Google Scholar] [CrossRef] [PubMed]
- Laudańska, Z.; López Pérez, D.; Radkowska, A.; Babis, K.; Malinowska-Korczak, A.; Wallot, S.; Tomalski, P. Changes in the Complexity of Limb Movements during the First Year of Life across Different Tasks. Entropy 2022, 24, 552. [Google Scholar] [CrossRef]
- Liu, L.; He, J.; Ren, K.; Lungu, J.; Hou, Y.; Dong, R. An Information Gain-Based Model and an Attention-Based RNN for Wearable Human Activity Recognition. Entropy 2021, 23, 1635. [Google Scholar] [CrossRef]
- Liu, Q.; Wei, Q.; Fan, S.-Z.; Lu, C.-W.; Lin, T.-Y.; Abbod, M.F.; Shieh, J.-S. Adaptive computation of multiscale entropy and its application in EEG signals for monitoring depth of anesthesia during surgery. Entropy 2012, 14, 978–992. [Google Scholar] [CrossRef] [Green Version]
- Cheng, C.-H.; Liang, R.-D.; Zhang, J.-S.; Fang, I.-J. The impact of product placement strategy on the placement communication effect: The case of a full-service restaurant. J. Hosp. Mark. Manag. 2014, 23, 424–444. [Google Scholar] [CrossRef]
- Brand, J.; Diamond, S.G.; Thomas, N.; Gilbert-Diamond, D. Evaluating the data quality of the Gazepoint GP3 low-cost eye tracker when used independently by study participants. Behav. Res. Methods 2021, 53, 1502–1514. [Google Scholar] [CrossRef]
- Cuve, H.C.; Stojanov, J.; Roberts-Gaal, X.; Catmur, C.; Bird, G. Validation of Gazepoint low-cost eye-tracking and psychophysiology bundle. Behav. Res. Methods 2022, 54, 1027–1049. [Google Scholar] [CrossRef] [PubMed]
- Yoshimura, K.; Kise, K.; Kunze, K. The eye as the window of the language ability: Estimation of English skills by analyzing eye movement while reading documents. In Proceedings of the 2015 13th International Conference on Document Analysis and Recognition (ICDAR), Tunis, Tunisia, 23–26 August 2015; pp. 251–255. [Google Scholar]
- Sanches, C.L.; Augereau, O.; Kise, K. Using the eye gaze to predict document reading subjective understanding. In Proceedings of the 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR), Kyoto, Japan, 9–15 November 2017; pp. 28–31. [Google Scholar]
- Liao, H.; Dong, W.; Huang, H.; Gartner, G.; Liu, H. Inferring user tasks in pedestrian navigation from eye movement data in real-world environments. Int. J. Geogr. Inf. Sci. 2019, 33, 739–763. [Google Scholar] [CrossRef]
- Huang, N.E.; Shen, Z.; Long, S.R.; Wu, M.C.; Shih, H.H.; Zheng, Q.; Yen, N.-C.; Tung, C.C.; Liu, H.H. The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis. Proc. R. Soc. London. Ser. A Math. Phys. Eng. Sci. 1998, 454, 903–995. [Google Scholar] [CrossRef]
- Wu, Z.; Huang, N.E. A study of the characteristics of white noise using the empirical mode decomposition method. Proc. R. Soc. London. Ser. A Math. Phys. Eng. Sci. 2004, 460, 1597–1611. [Google Scholar] [CrossRef]
- Boudraa, A.; Cexus, J.; Saidi, Z. EMD-based signal noise reduction. Int. J. Signal Processing 2004, 1, 33–37. [Google Scholar]
- Costa, M.D.; Goldberger, A.L. Generalized multiscale entropy analysis: Application to quantifying the complex volatility of human heartbeat time series. Entropy 2015, 17, 1197–1203. [Google Scholar] [CrossRef]
- Richman, J.S.; Moorman, J.R. Physiological time-series analysis using approximate entropy and sample entropy. Am. J. Physiol.-Heart Circ. Physiol. 2000, 278, H2039–H2049. [Google Scholar] [CrossRef] [Green Version]
- Wolf, A.; Swift, J.B.; Swinney, H.L.; Vastano, J.A. Determining Lyapunov exponents from a time series. Phys. D Nonlinear Phenom. 1985, 16, 285–317. [Google Scholar] [CrossRef] [Green Version]
- Pincus, S.M.; Goldberger, A.L. Physiological time-series analysis: What does regularity quantify? Am. J. Physiol. -Heart Circ. Physiol. 1994, 266, H1643–H1656. [Google Scholar] [CrossRef]
- Cahn, B.R.; Polich, J. Meditation states and traits: EEG, ERP, and neuroimaging studies. Psychol. Bull. 2006, 132, 180. [Google Scholar] [CrossRef]
- Costa, M.; Goldberger, A.L.; Peng, C.-K. Multiscale entropy analysis of complex physiologic time series. Phys. Rev. Lett. 2002, 89, 068102. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Scikit-Learn. Sklearn.Svm.SVC. Available online: https://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html (accessed on 18 April 2022).
- Pedregosa, F.; Varoquaux, G.; Gramfort, A.; Michel, V.; Thirion, B.; Grisel, O.; Blondel, M.; Prettenhofer, P.; Weiss, R.; Dubourg, V. Scikit-learn: Machine learning in Python. J. Mach. Learn. Res. 2011, 12, 2825–2830. [Google Scholar]
- Ramadhan, M.M.; Sitanggang, I.S.; Nasution, F.R.; Ghifari, A. Parameter tuning in random forest based on grid search method for gender classification based on voice frequency. DEStech Trans. Comput. Sci. Eng. 2017, 10, 625–629. [Google Scholar] [CrossRef] [Green Version]
- Busa, M.A.; van Emmerik, R.E. Multiscale entropy: A tool for understanding the complexity of postural control. J. Sport Health Sci. 2016, 5, 44–51. [Google Scholar] [CrossRef] [Green Version]
- Wyawahare, M.V.; Patil, P.M. Feature selection and classification for automatic detection of retinal nerve fibre layer thinning in retinal fundus images. Int. J. Biomed. Eng. Technol. 2015, 19, 205–219. [Google Scholar] [CrossRef]
- Su, C.-T.; Yang, C.-H. Feature selection for the SVM: An application to hypertension diagnosis. Expert Syst. Appl. 2008, 34, 754–763. [Google Scholar] [CrossRef]
- Hermes, L.; Buhmann, J.M. Feature selection for support vector machines. In Proceedings of the 15th International Conference on Pattern Recognition. ICPR-2000, Barcelona, Spain, 3–7 September 2000; pp. 712–715. [Google Scholar]
- Maldonado, S.; López, J. Dealing with high-dimensional class-imbalanced datasets: Embedded feature selection for SVM classification. Appl. Soft Comput. 2018, 67, 94–105. [Google Scholar] [CrossRef]
- Sun, L.; Fu, S.; Wang, F. Decision tree SVM model with Fisher feature selection for speech emotion recognition. EURASIP J. Audio Speech Music. Processing 2019, 2019, 2. [Google Scholar] [CrossRef] [Green Version]
- Wu, C.-H.; Lee, C.-H.; Jiang, B.C.; Sun, T.-L. Multiscale entropy analysis of postural stability for estimating fall risk via domain knowledge of Timed-Up-And-Go Accelerometer data for elderly people living in a community. Entropy 2019, 21, 1076. [Google Scholar] [CrossRef] [Green Version]
- Lee, C.-H.; Sun, T.-L. Multi-scale entropy analysis of body sway for investigating balance ability during exergame play under different parameter settings. Entropy 2015, 17, 7608–7627. [Google Scholar] [CrossRef] [Green Version]
- Lee, C.-H.; Sun, T.-L.; Jiang, B.C.; Choi, V.H. Using wearable accelerometers in a community service context to categorize falling behavior. Entropy 2016, 18, 257. [Google Scholar] [CrossRef]
- Wu, J.; Li, H.; Geng, Z.; Wang, Y.; Wang, X.; Zhang, J. Subtypes of nurses’ mental workload and interaction patterns with fatigue and work engagement during coronavirus disease 2019 (COVID-19) outbreak: A latent class analysis. BMC Nurs. 2021, 20, 206. [Google Scholar] [CrossRef] [PubMed]
- Lin, R.F.; Cheng, S.-H.; Liu, Y.-P.; Chen, C.-P.; Wang, Y.-J.; Chang, S.-Y. Predicting Emotional Valence of People Living with the Human Immunodeficiency Virus Using Daily Voice Clips: A Preliminary Study. Healthcare 2021, 9, 1148. [Google Scholar] [CrossRef] [PubMed]
No. | ML Method | Special Parameters |
---|---|---|
1 | SVM | Default |
2 | DT |
|
3 | RF |
|
Features | Statistic | ||||||
---|---|---|---|---|---|---|---|
Mean | STD | Var | Median | Max | Min | Skew | |
FPOGD | *** | *** | *** | *** | *** | ** | |
LPD | *** | *** | *** | *** | *** | *** | |
LPMM | *** | *** | *** | *** | *** | *** | * |
RPD | *** | *** | *** | *** | *** | *** | |
RPMM | *** | *** | *** | *** | *** | *** | *** |
BKDUR | *** | *** | *** | *** | *** | *** | *** |
BKPMIN | |||||||
SAC_MAG | *** | *** | *** | *** | *** | *** |
#of IMF Using CI | Eye-Movement Complexity Features | ||||||||
---|---|---|---|---|---|---|---|---|---|
FPOGD | FPOGX | FPOGY | LPCX | LPCY | LPD | RPD | LPMM | RPMM | |
IMF 1 | *** | *** | *** | *** | *** | *** | *** | *** | |
IMF 2 | * | *** | *** | *** | *** | *** | *** | *** | *** |
IMF 3 | *** | *** | *** | *** | *** | *** | *** | *** | *** |
IMF 4 | *** | *** | *** | *** | *** | ** | *** | *** | |
IMF 5 | *** | ** | *** | ** | *** | *** | *** | ||
IMF 6 | *** | *** | * | ** |
RF All Conventional Eye-Movement Features (56) | RF Important Conventional Eye-Movement Features (45) | ||||||||
---|---|---|---|---|---|---|---|---|---|
Mean (SD) | Predicted | Mean (SD) | Predicted | ||||||
Reading | Typing | Watching | Reading | Typing | Watching | ||||
Actual | Reading | 1.90 (0.00)% | 1.37 (0.00)% | 26.89 (0.01)% | Actual | Reading | 11.32 (0.03)% | 0.27 (0.00)% | 18.57 (0.04)% |
Typing | 0.53 (0.00)% | 29.31 (0.01)% | 0.32 (0.00)% | Typing | 0.44 (0.01)% | 28.63 (0.07)% | 1.08 (0.00)% | ||
Watching | 0.32 (0.00)% | 0.78 (0.01)% | 29.05 (0.01)% | Watching | 4.97 (0.01)% | 0.60 (0.00)% | 24.59 (0.01)% |
DT All Conventional Eye-movement Features (550) | DT Important Complexity Eye-movement Features (379) | ||||||||
Mean (SD) | Predicted | Mean (SD) | Predicted | ||||||
Reading | Watching | Reading | Reading | Watching | Typing | ||||
Actual | Reading | 18.81 (0.05)% | 8.90 (0.02)% | 2.46 (0.01)% | Actual | Reading | 19.97 (0.05)% | 8.57 (0.02)% | 1.61 (0.00)% |
Watching | 7.15 (0.02)% | 20.80 (0.05)% | 2.20 (0.01)% | Watching | 7.76 (0.02)% | 20.96 (0.05)% | 1.44 (0.01)% | ||
Typing | 1.62 (0.00)% | 2.08 (0.01)% | 26.46 (0.01)% | Typing | 1.97 (0.01)% | 1.42 (0.01)% | 26.77 (0.07)% | ||
RF All Conventional Eye-movement Features (550) | RF Important Complexity Eye-movement Features (379) | ||||||||
Mean (SD) | Predicted | Mean (SD) | Predicted | ||||||
Reading | Watching | Typing | Reading | Watching | Typing | ||||
Actual | Reading | 22.18 (0.06)% | 6.85 (0.01)% | 1.13 (0.00)% | Actual | Reading | 23.89 (0.06)% | 5.48 (0.48)% | 0.73 (0.01)% |
Typing | 5.03 (0.01)% | 24.42 (0.07)% | 0.71 (0.00)% | Typing | 4.58 (0.01)% | 24.69 (0.07)% | 0.91 (0.00)% | ||
Watching | 0.15 (0.00)% | 0.34 (0.00)% | 29.67 (0.07)% | Watching | 0.23 (0.02)% | 0.20 (0.00)% | 29.75 (0.08)% |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Destyanto, T.Y.R.; Lin, R.F. Evaluating the Effectiveness of Complexity Features of Eye Movement on Computer Activities Detection. Healthcare 2022, 10, 1016. https://doi.org/10.3390/healthcare10061016
Destyanto TYR, Lin RF. Evaluating the Effectiveness of Complexity Features of Eye Movement on Computer Activities Detection. Healthcare. 2022; 10(6):1016. https://doi.org/10.3390/healthcare10061016
Chicago/Turabian StyleDestyanto, Twin Yoshua R., and Ray F. Lin. 2022. "Evaluating the Effectiveness of Complexity Features of Eye Movement on Computer Activities Detection" Healthcare 10, no. 6: 1016. https://doi.org/10.3390/healthcare10061016
APA StyleDestyanto, T. Y. R., & Lin, R. F. (2022). Evaluating the Effectiveness of Complexity Features of Eye Movement on Computer Activities Detection. Healthcare, 10(6), 1016. https://doi.org/10.3390/healthcare10061016