Real-Time Machine Learning for Accurate Mexican Sign Language Identification: A Distal Phalanges Approach
Abstract
:1. Introduction
- i.
- only six measurements (angles) are used as feature descriptors;
- ii.
- efficient and accurate prediction;
- iii.
- relatively small training dataset;
- iv.
- high efficiency in letter prediction.
2. Related Work
2.1. The Taxonomy of Sign Language Recognition
2.2. Machine Learning Models in Sign Languages
3. Materials and Methods
3.1. Software and Hardware Characteristics
3.2. Data Loading and Dataset Preparation
3.3. The Performance Evaluation of Trained Models
3.4. Selection of Algorithms to Compare
- SVM: data classification is achieved by identifying the optimal separating hyperplane between classes in a multidimensional space;
- Naive Bayes: this approach is based on Bayes’ theorem and assumes independence between characteristics and the classes;
- KNN: data classification is performed by assigning labels based on nearest neighbor labels;
- Decision trees: this method classifies data using a decision tree, where each node represents a feature and each leaf a label;
- Random forests: combine multiple decision trees to classify data, reducing overfitting;
- XGBoost: implements gradient boosting to improve model accuracy using sequential decision trees with regularization and parallelization;
- LightGBM is another efficient implementation of gradient boosting. it uses sampling techniques to build decision trees faster and with lower memory usage;
- CatBoost is an ML algorithm developed by Yandex for gradient boosting on decision trees. It is particularly effective at handling categorical features directly, preventing overfitting, and offering high performance with both CPU and GPU support;
- RNNs: recurrent neural networks are designed for processing sequential data by maintaining a hidden state that captures information from previous time steps. They are widely used in applications like time-series forecasting and natural language processing.
3.5. Proposed Work
3.5.1. Training Stage
3.5.2. Validation Phase
3.6. Decision Tree
3.6.1. Entropy
3.6.2. Information Gain
3.6.3. Algorithm C4.5
3.6.4. Measure of Gini Impurity
4. Results
4.1. Evaluation Metrics and Their Relevance
- i.
- Random forest: Although random forest is known for its ability to handle large datasets with many features, it appears to be more time-consuming in training and prediction compared to other models. Nonetheless, it provides good accuracy on both training and test sets;
- ii.
- SVM: it has longer training and prediction times than other models, and its accuracy is not the highest on this dataset;
- iii.
- Naive Bayes: Although it has shorter training and prediction times, its accuracy is lower than other models. However, it could be a good choice if speed is a priority, and the required accuracy is reasonably high but not critical;
- iv.
- KNN: It shows very short training times, but the prediction times are longer. Its accuracy is relatively high, but its distance-based nature may not be optimal for large datasets;
- v.
- DT C4.5: it shows shorter training and prediction times compared to other models, and its accuracy is comparable to and even better than that of other more complex models on this dataset;
- vi.
- XGBoost and LightGBM: These gradient-boosting models have good accuracy results but longer training times than DT C4.5. However, their prediction times are shorter than those of random forest and SVM;
- vii.
- CatBoost: It performs excellently, with an accuracy of 1.00. However, its training time is significantly higher (2.42 s), and the cross-validation time is also longer (11.17 s);
- viii.
- Neural networks: although neural networks have good accuracy and recall (0.96), their training times (7.09 s) and cross-validation (29.00 s) are much longer compared to the decision tree and boosting models.
4.2. Confusion Matrix of the Compared Models
4.3. Performance of the DT C4.5 Classification Model
- Accuracy: This represents the proportion of correct predictions from the total predictions made for each class, providing a general measure of model performance. Values range from 0.99 to 1.0, indicating that the model is highly accurate for most classes;
- Precision: Indicates the proportion of instances correctly classified as positive out of all instances classified as positive. It measures the model’s ability to avoid misclassifying a negative instance as positive. Values range from 0.94 to 1.0, indicating that the model has a low false-positive rate for most classes;
- Recall: Represents the proportion of positive instances correctly identified by the model out of all true-positive instances. It measures the model’s ability to identify all relevant instances in a dataset. Values between 0.94 and 1.0 indicate that the model correctly identifies the most positive instances in each class;
- F1 Score: Measures model accuracy by considering both precision and recall. The harmonic mean of precision and recall balances the two metrics. Values range from 0.96 to 1.0, indicating a good balance between precision and recall for most classes;
- Support: values vary by class and represent the number of instances of each class in the test dataset.
4.4. Structure of Prediction in DT C4.5
4.5. The Results of the Characteristics of DT. C4.5
- i.
- Feature Names: In the context of a C4.5 DT, these are the features used to make decisions at each tree node. It is almost always specified that default feature names be used;
- ii.
- Gini: The Gini impurity measures how impure a node is. Determining which features and split values are best for dividing the dataset into smaller subsets is critical. In the decision-tree graph, nodes are split based on the Gini impurity value to minimize impurity in the resulting nodes;
- iii.
- Examples: In the context of DT C4.5, this refers to the number of data instances that arrive at a particular node during the tree’s training process. The number of samples arriving at each node can be shown in the decision-tree graph, providing information about the data distribution in the tree;
- iv.
- Value: The value at a node represents the class distribution of the samples arriving at that node. In the decision-tree graph, the value of a node can be visualized as a list showing how many samples of each class are present at that node;
- v.
- Class: The leaf nodes of the decision tree represent the predominant class of samples arriving at that node. In the decision-tree graph, the class of a leaf node can be displayed as a label indicating the predominant class at that node.
5. Discussion
6. Conclusions
Supplementary Materials
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Amrutha, K.; Prabu, P. ML Based Sign Language Recognition System. In Proceedings of the 2021 International Conference on Innovative Trends in Information Technology (ICITIIT), Kottayam, India, 11–12 February 2021; pp. 1–6. [Google Scholar] [CrossRef]
- Hekmat, A.; Abbas, H.; Shahadi, H. Sign Language Recognition and Hand Gestures Review. Kerbala J. Eng. Sci. 2022, 2, 209–234. [Google Scholar]
- Younas, F.; Nadir, J.; Usman, M.; Khan, M.A.; Khan, S.A.; Kadry, S.; Nam, Y. An Artificial Intelligence Approach for Word Semantic Similarity Measure of Hindi Language. KSII Trans. Internet Inf. Syst. 2021, 15, 2049–2068. [Google Scholar]
- Mahesh, B. Machine Learning Algorithms—A Review. Int. J. Sci. Res. 2019, 9, 381–386. [Google Scholar] [CrossRef]
- Napier, J. Sign Language Interpreter Training, Testing, and Accreditation: An International Comparison. Am. Ann. Deaf 2004, 149, 350–359. [Google Scholar] [CrossRef] [PubMed]
- Alaghband, M.; Maghroor, H.R.; Garibay, I. A survey on sign language literature. Mach. Learn. Appl. 2023, 14, 100504. [Google Scholar] [CrossRef]
- Escobar, L. Gestualidad y lengua en la lengua de señas mexicana. Lingüíst. Mex. Nueva Época 2019, 1, 141–166. [Google Scholar] [CrossRef]
- Valli, C.; Lucas, C. Linguistics of American Sign Language: An Introduction, 3rd ed.; Gallaudet University Press: Washington, DC, USA, 2000; ISBN 978-1-56368-097-7. [Google Scholar]
- Lugaresi, C.; Tang, J.; Nash, H.; McClanahan, C.; Uboweja, E.; Hays, M.; Zhang, F.; Chang, C.-L.; Yong, M.G.; Lee, J.; et al. MediaPipe: A Framework for Building Perception Pipelines. arXiv 2019, arXiv:1906.08172. [Google Scholar]
- Zhang, F.; Bazarevsky, V.; Vakunov, A.; Tkachenka, A.; Sung, G.; Chang, C.-L.; Grundmann, M. MediaPipe Hands: On-device Real-time Hand Tracking. arXiv 2020, arXiv:2006.10214. [Google Scholar] [CrossRef]
- Zelinsky, A. Learning OpenCV—Computer Vision with the OpenCV Library (Bradski, G.R. et al.; 2008) [On the Shelf]. IEEE Robot. Autom. Mag. IEEE Robot Autom. 2009, 16, 100. [Google Scholar] [CrossRef]
- González-Rodríguez, J.-R.; Córdova-Esparza, D.-M.; Terven, J.; Romero-González, J.-A. Towards a Bidirectional Mexican Sign Language–Spanish Translation System: A Deep Learning Approach. Technologies 2024, 12, 7. [Google Scholar] [CrossRef]
- Sosa-Jimenez, C.O.; Rios-Figueroa, H.V.; Solis-Gonzalez-Cosio, A.L. A Prototype for Mexican Sign Language Recognition and Synthesis in Support of a Primary Care Physician. IEEE Access 2022, 10, 127620–127635. [Google Scholar] [CrossRef]
- Li, Z.; Liu, F.; Yang, W.; Peng, S.; Zhou, J. A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects. IEEE Trans. Neural Netw. Learn. Syst. 2022, 33, 6999–7019. [Google Scholar] [CrossRef] [PubMed]
- Wangchuk, K.; Riyamongkol, P.; Waranusast, R. Real-time Bhutanese Sign Language digits recognition system using Convolutional Neural Network. ICT Express 2021, 7, 215–220. [Google Scholar] [CrossRef]
- Kasapbaşi, A.; Elbushra, A.; Al-Hardanee, O.; Yilmaz, A. DeepASLR: A CNN based Human Computer Interface for American Sign Language Recognition for Hearing-Impaired Individuals. Comput. Methods Programs Biomed. Update 2022, 2, 100048. [Google Scholar] [CrossRef]
- Arooj, S.; Altaf, S.; Ahmad, S.; Mahmoud, H.; Mohamed, A.S.N. Enhancing sign language recognition using CNN and SIFT: A case study on Pakistan sign language. J. King Saud Univ. Comput. Inf. Sci. 2024, 36, 101934. [Google Scholar] [CrossRef]
- Ameen, S.; Vadera, S. A convolutional neural network to classify American Sign Language fingerspelling from depth and colour images. Expert Syst. 2017, 34, e12197. [Google Scholar] [CrossRef]
- Thongtawee, A.; Pinsanoh, O.; Kitjaidure, Y. A Novel Feature Extraction for American Sign Language Recognition Using Webcam. In Proceedings of the 2018 11th Biomedical Engineering International Conference (BMEiCON), Chiang Mai, Thailand, 21–24 November 2018; pp. 1–5. [Google Scholar] [CrossRef]
- Rastgoo, R.; Kiani, K.; Escalera, S. Video-based isolated hand sign language recognition using a deep cascaded model. Multimed. Tools Appl. 2020, 79, 22965–22987. [Google Scholar] [CrossRef]
- Rastgoo, R.; Kiani, K.; Escalera, S. Real-time isolated hand sign language recognition using deep networks and SVD. J. Ambient Intell. Humaniz. Comput. 2022, 13, 591–611. [Google Scholar] [CrossRef]
- Sharma, P.; Anand, R.S. A comprehensive evaluation of deep models and optimizers for Indian sign language recognition. Graph. Vis. Comput. 2021, 5, 200032. [Google Scholar] [CrossRef]
- Katoch, S.; Singh, V.; Tiwary, U.S. Indian Sign Language recognition system using SURF with SVM and CNN. Array 2022, 14, 100141. [Google Scholar] [CrossRef]
- Tripathi, S.; Singh, S.K.; Kuan, L.H. Bag of Visual Words (BoVW) with Deep Features—Patch Classification Model for Limited Dataset of Breast Tumours. arXiv 2022, arXiv:2202.10701. [Google Scholar] [CrossRef]
- Tian, Y.; Shi, Y.; Liu, X. Recent advances on support vector machines research. Technol. Econ. Dev. Econ. 2012, 18, 5–33. [Google Scholar] [CrossRef]
- Subramanian, B.; Olimov, B.; Naik, S.M.; Kim, S.; Park, K.-H.; Kim, J. An integrated mediapipe-optimized GRU model for Indian sign language recognition. Sci. Rep. 2022, 12, 11964. [Google Scholar] [CrossRef]
- Sak, H.; Senior, A.; Beaufays, F. Long Short-Term Memory Based Recurrent Neural Network Architectures for Large Vocabulary Speech Recognition. arXiv 2014, arXiv:1402.1128. [Google Scholar] [CrossRef]
- Sundar, B.; Bagyammal, T. American Sign Language Recognition for Alphabets Using MediaPipe and LSTM. Procedia Comput. Sci. 2022, 215, 642–651. [Google Scholar] [CrossRef]
- Pathan, R.K.; Biswas, M.; Yasmin, S.; Khandaker, M.U.; Salman, M.; Youssef, A.A.F. Sign language recognition using the fusion of image and hand landmarks through multi-headed convolutional neural network. Sci. Rep. 2023, 13, 16975. [Google Scholar] [CrossRef]
- Ruiz, D.S.; Olvera-López, J.A.; Olmos-Pineda, I. Word Level Sign Language Recognition via Handcrafted Features. IEEE Lat. Am. Trans. 2023, 21, 839–848. [Google Scholar] [CrossRef]
- Mohsin, S.; Salim, B.W.; Mohamedsaeed, A.K.; Ibrahim, B.F.; Zeebaree, S.R.M. American Sign Language Recognition Based on Transfer Learning Algorithms. Int. J. Intell. Syst. Appl. Eng. 2024, 12, 390–399. [Google Scholar]
- Amangeldy, N.; Ukenova, A.; Bekmanova, G.; Razakhova, B.; Milosz, M.; Kudubayeva, S. Continuous Sign Language Recognition and Its Translation into Intonation-Colored Speech. Sensors 2023, 23, 6383. [Google Scholar] [CrossRef]
- Wali, A.; Shariq, R.; Shoaib, S.; Amir, S.; Farhan, A.A. Recent progress in sign language recognition: A review. Mach. Vis. Appl. 2023, 34, 127. [Google Scholar] [CrossRef]
- Farooq, U.; Rahim, M.S.M.; Sabir, N.; Hussain, A.; Abid, A. Advances in machine translation for sign language: Approaches, limitations, and challenges. Neural Comput. Appl. 2021, 33, 14357–14399. [Google Scholar] [CrossRef]
- Zhang, M.-L.; Zhou, Z.-H. ML-KNN: A lazy learning approach to multi-label learning. Pattern Recognit. 2007, 40, 2038–2048. [Google Scholar] [CrossRef]
- Louppe, G. Understanding Random Forests: From Theory to Practice. arXiv 2014, arXiv:1407.7502. [Google Scholar] [CrossRef]
- Hinton, G.E.; Osindero, S.; Teh, Y.-W. A Fast Learning Algorithm for Deep Belief Nets. Neural Comput. 2006, 18, 1527–1554. [Google Scholar] [CrossRef]
- Bajaj, Y.; Malhotra, P. American Sign Language Identification Using Hand Trackpoint Analysis. arXiv 2020, arXiv:2010.10590. [Google Scholar] [CrossRef]
- Sahoo, A.K.; Mishra, G.S.; Ravulakollu, K.K. Sign Language Recognition: State of the Art. 2014.
- Maebatake, M.; Suzuki, I.; Nishida, M.; Horiuchi, Y.; Kuroiwa, S. Sign Language Recognition Based on Position and Movement Using Multi-Stream HMM. In Proceedings of the 2008 Second International Symposium on Universal Communication, Osaka, Japan, 15–16 December 2008; pp. 478–481. [Google Scholar] [CrossRef]
- Athitsos, V.; Alon, J.; Sclaroff, S.; Kollios, G. BoostMap: An Embedding Method for Efficient Nearest Neighbor Retrieval. IEEE Trans. Pattern Anal. Mach. Intell. 2008, 30, 89–104. [Google Scholar] [CrossRef] [PubMed]
- Rahim, A.; Hossain, N.; Wahid, T.; Azam, S. Face Recognition using Local Binary Patterns (LBP). Glob. J. Comput. Sci. Technol. 2013, 13, 1–8. [Google Scholar]
- Huang, C.; Huang, J. A Fast HOG Descriptor Using Lookup Table and Integral Image. arXiv 2017, arXiv:1703.06256. [Google Scholar] [CrossRef]
- Verma, R.; Kaur, M.R. Enhanced Character Recognition Using Surf Feature and Neural Network Technique. 2014. Available online: https://www.semanticscholar.org/paper/Enhanced-Character-Recognition-Using-Surf-Feature-Verma-Kaur/49f3939df922881dd857faac71aa5c7b873a606a (accessed on 18 May 2024).
- Hussain, M.; Shaoor, A.; Alsuhibany, S.; Ghadi, Y.; Shloul, T.; Jalal, A.; Park, J. Intelligent Sign Language Recognition System for E-Learning Context. Comput. Mater. Contin. 2022, 72, 5327–5343. [Google Scholar] [CrossRef]
- Zhang, H. The Optimality of Naive Bayes. In Proceedings of the Seventeenth International Florida Artificial Intelligence Research Society Conference (FLAIRS ), Fredericton, NB, Canada, 1 January 2004. [Google Scholar]
- Chen, T.; Guestrin, C. XGBoost: A Scalable Tree Boosting System. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA, 13–17 August 2016; ACM: New York, NY, USA, 2006; pp. 785–794. [Google Scholar] [CrossRef]
- Ke, G.; Meng, Q.; Finley, T.; Wang, T.; Chen, W.; Ma, W.; Ye, Q.; Liu, T.Y. LightGBM: A Highly Efficient Gradient Boosting Decision Tree. In Advances in Neural Information Processing Systems; Curran Associates, Inc.: New York, HY, USA, 2017; Available online: https://proceedings.neurips.cc/paper_files/paper/2017/hash/6449f44a102fde848669bdd9eb6b76fa-Abstract.html (accessed on 18 May 2024).
- Lewis, R. An Introduction to Classification and Regression Tree (CART) Analysis. In Proceedings of the Annual Meeting of the Society for Academic Emergency Medicine, San Francisco, CA, USA, 22–25 May 2000. [Google Scholar]
- Pedregosa, F.; Varoquaux, G.; Gramfort, A.; Michel, V.; Thirion, B.; Grisel, O.; Blondel, M.; Prettenhofer, P.; Weiss, R.; Dubourg, V.; et al. Scikit-learn: Machine Learning in Python. Mach. Learn. Python 2011, 12, 2825–2830. [Google Scholar]
- Hubert, M.; Van der Veeken, S. Outlier detection for skewed data. J. Chemom. 2008, 22, 235–246. [Google Scholar] [CrossRef]
- Butcher, B.; Smith, B.J. Feature Engineering and Selection: A Practical Approach for Predictive Models. In The American Statistician; Kuhn, M., Johnson, K., Eds.; Chapman & Hall/CRC Press: Boca Raton, FL, USA, 2020; Volume 74, pp. 308–309. ISBN 978-1-13-807922-9. [Google Scholar] [CrossRef]
- Hastie, T.; Tibshirani, R.; Friedman, J. The Elements of Statistical Learning; Springer Series in Statistics; Springer: New York, NY, USA, 2009. [Google Scholar] [CrossRef]
- Shin, J.; Matsuoka, A.; Hasan, M.A.M.; Srizon, A.Y. American Sign Language Alphabet Recognition by Extracting Feature from Hand Pose Estimation. Sensors 2021, 21, 5856. [Google Scholar] [CrossRef]
- Obi, Y.; Claudio, K.S.; Budiman, V.M.; Achmad, S.; Kurniawan, A. Sign language recognition system for communicating to people with disabilities. Procedia Comput. Sci. 2023, 216, 13–20. [Google Scholar] [CrossRef]
- Joksimoski, B.; Zdravevski, E.; Lameski, P.; Pires, I.M.; Melero, F.J.; Martinez, T.P.; Garcia, N.M.; Mihajlov, M.; Chorbev, I.; Trajkovik, V. Technological Solutions for Sign Language Recognition: A Scoping Review of Research Trends, Challenges, and Opportunities. IEEE Access 2022, 10, 40979–40998. [Google Scholar] [CrossRef]
- Fang, G.; Gao, W.; Zhao, D. Large Vocabulary Sign Language Recognition Based on Fuzzy Decision Trees. IEEE Trans. Syst. Man Cybern. Part A Syst. Hum. 2004, 34, 305–314. [Google Scholar] [CrossRef]
- Quinlan, J.R. Induction of decision trees. Mach. Learn. 1986, 1, 81–106. [Google Scholar] [CrossRef]
- Quinlan, J.R. Improved Use of Continuous Attributes in C4.5. J. Artif. Intell. Res. 1996, 4, 77–90. [Google Scholar] [CrossRef]
- Li, X.; Yi, S.; Cundy, A.B.; Chen, W. Sustainable decision-making for contaminated site risk management: A decision tree model using machine learning algorithms. J. Clean. Prod. 2022, 371, 133612. [Google Scholar] [CrossRef]
- Lyu, Y.; Huang, X. Road Segmentation Using CNN with GRU. arXiv 2018, arXiv:1804.05164. [Google Scholar]
Disclaimer/Publisher’s Note: The statements, opinions, and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions, or products referred to in the content. |
Actual Class | ||||
---|---|---|---|---|
Positive | Negative | |||
Predicted Class | Negative | False Negative (FN) | True Negative (TN) | Neg Pre. Value TN/(TN + FN) |
Positive | True Positive (TP) | False Positive (FP) | Precision TP/(TP + FP) | |
F1-score (2xPxR)/P + R | Recall TP/(TP + FN) | Specificity TN/(TN + FP) | Accuracy TP + TN/(TP + TN + FP + FN) |
Pseudocode 1 Hand Angle Data Collection and Training |
|
Pseudocode 2 Real-Time Hand Gesture Recognition Validation |
1: Start 2: Import necessary libraries (cv2, mediapipe, matplotlib, numpy, pandas, sklearn) 3: Configure mediapipe for hand detection and capture video from the camera 4: Read hand gesture data from an Excel file 5: Initialize necessary variables and data structures 6: With mediapipe.Hands( 7: static_image_mode = False, 8: max_num_hands = 2, 9: min_detection_confidence = 0.5) as hands: 10: While True: 11: Read a frame from the camera 12: If the frame was read successfully: 13: Process the frame with the hand detection model 14: If hands are detected in the frame: 15: For each detected hand: 16: Extract coordinates of finger landmarks and wrist 17: Calculate angles between finger landmarks and wrist 18: Visualize landmarks and hand lines on the frame 19: Display calculated angles near the landmarks 20: Store the calculated angles 21: Show the frame with detected hands and calculated angles 22: If the ‘ESC’ key is pressed: 23: Perform gesture classification using a decision tree model 24: Display the predicted gesture label on the frame 25: If needed, save the output video with gesture labels 26: Exit the loop 27: Release camera resources and close all windows 28: End |
Model Results | |||||||||
---|---|---|---|---|---|---|---|---|---|
Metric/Method | D T C4.5 | SVM | N B | k-NN | RF | XGBoost | LightGBM | CatBoost | RNNs |
Training Accuracy | 1.00 | 0.98 | 0.97 | 0.99 | 1.00 | 1.00 | 1.00 | 1.00 | 0.92 |
Training Loss | 0.00 | 0.02 | 0.03 | 0.01 | 0.00 | 0.00 | 0.00 | 0.00 | 0.27 |
Testing Accuracy | 0.99 | 0.96 | 0.96 | 0.98 | 0.99 | 0.99 | 0.99 | 1.00 | 0.96 |
Testing Loss | 0.01 | 0.04 | 0.04 | 0.02 | 0.01 | 0.01 | 0.01 | 0.00 | 0.04 |
Accuracy | 0.99 | 0.96 | 0.96 | 0.98 | 0.99 | 0.99 | 0.99 | 1.00 | 0.96 |
Precision | 0.99 | 0.96 | 0.97 | 0.98 | 1.00 | 0.99 | 0.99 | 1.00 | 0.96 |
Recall | 0.99 | 0.96 | 0.96 | 0.98 | 0.99 | 0.99 | 0.99 | 1.00 | 0.96 |
F1 Score | 0.99 | 0.96 | 0.96 | 0.98 | 0.99 | 0.99 | 0.99 | 1.00 | 0.96 |
Execution Times | |||||||||
---|---|---|---|---|---|---|---|---|---|
Time/Method | D T C4.5 | SVM | NB | k-NN | RF | XGBoost | LightGBM | CatBoost | RNNs |
Training Time | 0.01 s | 0.05 s | 0.00 s | 0.02 s | 0.36 s | 0.31 s | 0.51 s | 2.42 s | 7.09 s |
Prediction Time | 0.00 s | 0.03 s | 0.01 s | 0.03 s | 0.02 s | 0.00 s | 0.10 s | 0.00 s | 0.12 s |
Cross-Validation Scores | |||||||||
---|---|---|---|---|---|---|---|---|---|
Metric/Method | D T C4.5 | SVM | NB | k-NN | RF | XGBoost | LightGBM | CatBoost | RNNs |
Cross-Validation Time | 0.07 s | 0.45 s | 0.02 s | 0.24 s | 1.78 s | 1.58 s | 2.41 s | 11.17 s | 29.00 s |
CV Scores | [0.977, 0.985, 0.984, 0.984, 0.985] | [0.946, 0.960, 0.932, 0.952, 0.951] | [0.954, 0.963, 0.936, 0.926, 0.976] | [0.965, 0.982, 0.954, 0.944, 0.973] | [0.984, 0.992, 0.971, 0.974, 0.976] | [0.937, 0.988, 0.967, 0.976, 0.981] | [0.937, 0.989, 0.967, 0.976, 0.981] | [0.990, 0.992, 0.991, 0.995, 0.993] | [0.967, 0.970, 0.967, 0.968, 0.980] |
CV Mean | 0.98 | 0.96 | 0.95 | 0.96 | 0.98 | 0.97 | 0.97 | 0.99 | 0.97 |
CV Standard Deviation | 0.00 | 0.01 | 0.02 | 0.01 | 0.01 | 0.02 | 0.02 | 0.00 | 0.01 |
Letter | Accuracy | Precisión | Recall | F1-Score | Support |
---|---|---|---|---|---|
A | 0.99 | 0.98 | 1 | 0.99 | 82 |
B | 0.99 | 0.98 | 1 | 0.99 | 47 |
C | 0.99 | 1 | 1 | 1 | 54 |
D | 0.99 | 1 | 0.95 | 0.98 | 42 |
E | 0.99 | 1 | 0.99 | 0.99 | 70 |
F | 0.99 | 0.94 | 0.97 | 0.96 | 34 |
G | 0.99 | 1 | 1 | 1 | 67 |
H | 0.99 | 1 | 0.97 | 0.99 | 39 |
I | 0.99 | 1 | 1 | 1 | 62 |
L | 0.99 | 1 | 1 | 1 | 80 |
M | 0.99 | 0.98 | 0.94 | 0.96 | 52 |
N | 1 | 0.97 | 1 | 0.99 | 71 |
O | 0.99 | 0.97 | 0.98 | 0.98 | 62 |
P | 1 | 1 | 1 | 1 | 47 |
R | 0.99 | 1 | 0.96 | 0.98 | 50 |
S | 0.99 | 0.97 | 0.94 | 0.96 | 35 |
T | 1 | 1 | 1 | 1 | 54 |
U | 1 | 0.98 | 0.97 | 0.97 | 60 |
V | 0.99 | 0.95 | 0.98 | 0.97 | 64 |
W | 1 | 1 | 1 | 1 | 36 |
Y | 1 | 0.97 | 1 | 0.98 | 30 |
Weighted Avg | 0.99 | 0.99 | 0.99 | 0.99 | 1138 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
García-Gil, G.; López-Armas, G.d.C.; Sánchez-Escobar, J.J.; Salazar-Torres, B.A.; Rodríguez-Vázquez, A.N. Real-Time Machine Learning for Accurate Mexican Sign Language Identification: A Distal Phalanges Approach. Technologies 2024, 12, 152. https://doi.org/10.3390/technologies12090152
García-Gil G, López-Armas GdC, Sánchez-Escobar JJ, Salazar-Torres BA, Rodríguez-Vázquez AN. Real-Time Machine Learning for Accurate Mexican Sign Language Identification: A Distal Phalanges Approach. Technologies. 2024; 12(9):152. https://doi.org/10.3390/technologies12090152
Chicago/Turabian StyleGarcía-Gil, Gerardo, Gabriela del Carmen López-Armas, Juan Jaime Sánchez-Escobar, Bryan Armando Salazar-Torres, and Alma Nayeli Rodríguez-Vázquez. 2024. "Real-Time Machine Learning for Accurate Mexican Sign Language Identification: A Distal Phalanges Approach" Technologies 12, no. 9: 152. https://doi.org/10.3390/technologies12090152