Few-Shot Conditional Learning: Automatic and Reliable Device Classification for Medical Test Equipment
Abstract
:1. Introduction
2. Data Acquisition and Preparation
2.1. Medical Equipment Dataset
2.2. Model Equipment Test Dataset
2.3. BAIR Robot Pushing Dataset
3. Materials and Methods
3.1. Few-Shot Problem Definition
3.2. Proposal
3.3. Define a Reliability Score
3.4. Experiments
3.4.1. ResNet
3.4.2. Masks Encoder
3.4.3. Few-Shot Experiments
3.4.4. Inference
4. Results
4.1. Masks Encoder Pre-Training
4.2. Meta-Validation Results
4.3. Details on Equipment and Reliability
4.4. Test Results by Preparatory Table
5. Discussion and Conclusions
Author Contributions
Funding
Data Availability Statement
Conflicts of Interest
Abbreviations
FSL | Few-shot learning |
STD | Standard deviation |
IQR | Interquartile range |
CV | Cross-validation |
RS | Reliability score |
TS | Trust score |
ET | Endotracheal Tube |
AUROC | Area Under the Receiver Operating Characteristic Curve |
AUPRC | Area Under the Precision–Recall Curve |
References
- Cheng, G.; Xie, X.; Han, J.; Guo, L.; Xia, G.S. Remote Sensing Image Scene Classification Meets Deep Learning: Challenges, Methods, Benchmarks, and Opportunities. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2020, 13, 3735–3756. [Google Scholar] [CrossRef]
- Hussain, E.; Hasan, M.; Rahman, M.A.; Lee, I.; Tamanna, T.; Parvez, M.Z. CoroDet: A deep learning based classification for COVID-19 detection using chest X-ray images. Chaos Solitons Fractals 2021, 142, 110495. [Google Scholar] [CrossRef]
- Khan, F.; Zafar, N.; Tahir, M.N.; Aqib, M.; Waheed, H.; Haroon, Z. A mobile-based system for maize plant leaf disease detection and classification using deep learning. Front. Plant Sci. 2023, 14, 1079366. [Google Scholar] [CrossRef]
- Serrano, C.; Lazo, M.; Serrano, A.; Toledo-Pastrana, T.; Barros-Tornay, R.; Acha, B. Clinically Inspired Skin Lesion Classification through the Detection of Dermoscopic Criteria for Basal Cell Carcinoma. J. Imaging 2022, 8, 197. [Google Scholar] [CrossRef]
- Hüllermeier, E.; Waegeman, W. Aleatoric and epistemic uncertainty in machine learning: An introduction to concepts and methods. Mach. Learn. 2021, 110, 457–506. [Google Scholar] [CrossRef]
- Kuleshov, V.; Liang, P.S. Calibrated structured prediction. Adv. Neural Inf. Process. Syst. 2015, 28, 3474–3482. [Google Scholar]
- Guo, C.; Pleiss, G.; Sun, Y.; Weinberger, K.Q. On calibration of modern neural networks. In Proceedings of the International Conference on Machine Learning, PMLR, Sydney, Australia, 6–11 August 2017; pp. 1321–1330. [Google Scholar]
- Goodfellow, I.J.; Shlens, J.; Szegedy, C. Explaining and harnessing adversarial examples. arXiv 2014, arXiv:1412.6572. [Google Scholar]
- Nguyen, A.; Yosinski, J.; Clune, J. Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, 7–12 June 2015; pp. 427–436. [Google Scholar]
- Wang, Y.; Yao, Q.; Kwok, J.T.; Ni, L.M. Generalizing from a few examples: A survey on few-shot learning. ACM Comput. Surv. 2020, 53, 63. [Google Scholar] [CrossRef]
- Wang, Z.; Li, J.; Wang, W.; Dong, Z.; Zhang, Q.; Guo, Y. Review of few-shot learning application in CSI human sensing. Artif. Intell. Rev. 2024, 57, 195. [Google Scholar] [CrossRef]
- Sun, J.; Cao, W.; Fu, X.; Ochi, S.; Yamanaka, T. Few-shot learning for plant disease recognition: A review. Agron. J. 2024, 116, 1204–1216. [Google Scholar] [CrossRef]
- Galán-Cuenca, A.; Gallego, A.J.; Saval-Calvo, M.; Pertusa, A. Few-shot learning for COVID-19 chest X-ray classification with imbalanced data: An inter vs. intra domain study. Pattern Anal. Appl. 2024, 27, 69. [Google Scholar] [CrossRef]
- Sun, L.; Zhang, M.; Wang, B.; Tiwari, P. Few-shot class-incremental learning for medical time series classification. IEEE J. Biomed. Health Inform. 2023, 28, 1872–1882. [Google Scholar] [CrossRef]
- Mihretie, G.N.; Liyeh, T.M.; Ayele, A.D.; Kassa, B.G.; Belay, H.G.; Aytenew, T.M.; Sewuye, D.A.; Birhane, B.M.; Misk, A.D.; Alemu, B.K. Knowledge and skills of newborn resuscitation among health care professionals in East Africa. A systematic review and meta-analysis. PLoS ONE 2024, 19, e0290737. [Google Scholar] [CrossRef] [PubMed]
- Patel, A.B.; Bang, A.; Kurhe, K.; Bhargav, S.; Hibberd, P.L. What Helping Babies Breathe knowledge and skills are formidable for healthcare workers? Front. Pediatr. 2023, 10, 891266. [Google Scholar] [CrossRef]
- Tolsgaard, M.; Konge, L.; Bjerrum, F.; Thomsen, A.S.S.; Gustafsson, A. Implementation of evidence-based simulation training. Ugeskr. Laeger 2019, 181, V02190086. [Google Scholar]
- Vadla, M.S.; Moshiro, R.; Mdoe, P.; Eilevstjønn, J.; Kvaløy, J.T.; Hhoki, B.H.; Ersdal, H. Newborn resuscitation simulation training and changes in clinical performance and perinatal outcomes: A clinical observational study of 10,481 births. Adv. Simul. 2022, 7, 38. [Google Scholar] [CrossRef]
- Palmer, E.; Labant, A.L.; Edwards, T.F.; Boothby, J. A collaborative partnership for improving newborn safety: Using simulation for neonatal resuscitation training. J. Contin. Educ. Nurs. 2019, 50, 319–324. [Google Scholar] [CrossRef]
- Bardelli, S.; Del Corso, G.; Ciantelli, M.; Del Pistoia, M.; Lorenzoni, F.; Fossati, N.; Scaramuzzo, R.T.; Cuttano, A. Improving pediatric/neonatology residents’ newborn resuscitation skills with a digital serious game: DIANA. Front. Pediatr. 2022, 10, 842302. [Google Scholar] [CrossRef]
- Hinder, M.; Tracy, M. Newborn resuscitation devices: The known unknowns and the unknown unknowns. Semin. Fetal Neonatal Med. 2021, 26, 101233. [Google Scholar] [CrossRef]
- Ebert, F.; Finn, C.; Lee, A.X.; Levine, S. Self-Supervised Visual Planning with Temporal Skip Connections. CoRL 2017, 12, 16. [Google Scholar]
- American Academy of Pediatrics and American Heart Association. Textbook of Neonatal Resuscitation (NRP); American Academy of Pediatrics and American Heart Association: Itasca, IL, USA, 2016. [Google Scholar]
- Zhao, X.; Ding, W.; An, Y.; Du, Y.; Yu, T.; Li, M.; Tang, M.; Wang, J. Fast segment anything. arXiv 2023, arXiv:2306.12156. [Google Scholar]
- Kirillov, A.; Mintun, E.; Ravi, N.; Mao, H.; Rolland, C.; Gustafson, L.; Xiao, T.; Whitehead, S.; Berg, A.C.; Lo, W.Y.; et al. Segment anything. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Paris, France, 2–3 October 2023; pp. 4015–4026. [Google Scholar]
- Snell, J.; Swersky, K.; Zemel, R. Prototypical networks for few-shot learning. In Advances in Neural Information Processing Systems; Curran Associates Inc.: Red Hook, NY, USA, 2017; Volume 30. [Google Scholar]
- Jiang, H.; Kim, B.; Guan, M.; Gupta, M. To trust or not to trust a classifier. In Advances in Neural Information Processing Systems; Curran Associates Inc.: Red Hook, NY, USA, 2018; Volume 31. [Google Scholar]
- Tenenbaum, J. Mapping a manifold of perceptual observations. In Advances in Neural Information Processing Systems; Curran Associates Inc.: Red Hook, NY, USA, 1997; Volume 10. [Google Scholar]
- McInnes, L.; Healy, J.; Melville, J. Umap: Uniform manifold approximation and projection for dimension reduction. arXiv 2018, arXiv:1802.03426. [Google Scholar]
- Mahmud, M.S.; Huang, J.Z.; Fu, X. Variational autoencoder-based dimensionality reduction for high-dimensional small-sample data classification. Int. J. Comput. Intell. Appl. 2020, 19, 2050002. [Google Scholar] [CrossRef]
- Del Corso, G. Uncertainty Analysis of Biological Systems: Towards a Digital Twin of the Human Heart. Ph.D. Thesis, Gran Sasso Science Institute, L’Aquila, Italy, 2022. [Google Scholar]
- Janssen, H. Monte-Carlo based uncertainty analysis: Sampling efficiency and sampling convergence. Reliab. Eng. Syst. Saf. 2013, 109, 123–132. [Google Scholar] [CrossRef]
- Alibrahim, H.; Ludwig, S.A. Hyperparameter optimization: Comparing genetic algorithm against grid search and bayesian optimization. In Proceedings of the 2021 IEEE Congress on Evolutionary Computation (CEC), Kraków, Poland, 28 June–1 July 2021; pp. 1551–1559. [Google Scholar]
- Wang, Z.; Bovik, A.; Sheikh, H.; Simoncelli, E. Image quality assessment: From error visibility to structural similarity. IEEE Trans. Image Process. 2004, 13, 600–612. [Google Scholar] [CrossRef]
- Zhou, Z.H. Ensemble Methods: Foundations and Algorithms; CRC Press: Boca Raton, FL, USA, 2012. [Google Scholar]
- Wertheimer, D.; Hariharan, B. Few-shot learning with localization in realistic settings. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA, 15–20 June 2019; pp. 6558–6567. [Google Scholar]
- Xie, J.; Long, F.; Lv, J.; Wang, Q.; Li, P. Joint distribution matters: Deep brownian distance covariance for few-shot classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and PATTERN Recognition, New Orleans, LA, USA, 18–24 June 2022; pp. 7972–7981. [Google Scholar]
Test Set | SSIM | Dice |
---|---|---|
Internal (BAIR) | 0.983 (0.005) | 0.935 (0.044) |
External (Medical Equipment) | 0.961 (0.008) | 0.991 (0.005) |
Backbone | CV Fold | 1-Shot | 3-Shot | 5-Shot |
---|---|---|---|---|
ResNet-18 | Fold 1 | 89.25 (4.28) | 81.60 (5.02) | 88.50 (4.72) |
Fold 2 | 73.40 (6.81) | 80.55 (4.98) | 77.80 (5.07) | |
Fold 3 | 96.65 (3.59) | 100.00 (0.00) | 100.00 (0.00) | |
Fold 4 | 98.30 (2.47) | 97.55 (2.67) | 100.00 (0.00) | |
Fold 5 | 99.40 (1.28) | 99.55 (0.96) | 99.70 (0.81) | |
Median [IQR] | 96.65 [9.05] | 97.55 [17.95] | 99.70 [11.5] | |
ResNet-50 | Fold 1 | 92.80 (3.83) | 89.15 (4.02) | 91.25 (4.01) |
Fold 2 | 76.55 (6.50) | 77.45 (5.88) | 79.40 (6.01) | |
Fold 3 | 98.45 (2.05) | 100.00 (0.00) | 100.00 (0.00) | |
Fold 4 | 89.25 (5.79) | 99.90 (0.49) | 99.50 (1.12) | |
Fold 5 | 84.70 (7.79) | 96.75 (2.93) | 98.75 (1.68) | |
Median [IQR] | 89.25 [8.10] | 96.75 [10.75] | 98.75 [8.25] |
CV Fold | Table 1 | Table 2 | Table 3 | Table 4 | Table 5 | Table 6 | Table 7 | Table 8 |
---|---|---|---|---|---|---|---|---|
Fold 1 | 87.50 | 75.00 | 75.00 | 75.00 | 87.50 | 87.50 | 87.50 | 87.50 |
Fold 2 | 50.00 | 75.00 | 62.50 | 62.50 | 87.50 | 75.00 | 87.50 | 75.00 |
Fold 3 | 75.00 | 75.00 | 87.50 | 50.00 | 87.50 | 87.50 | 100.00 | 100.00 |
Fold 4 | 75.00 | 87.50 | 87.50 | 62.50 | 87.50 | 87.50 | 100.00 | 87.50 |
Fold 5 | 87.50 | 75.00 | 87.50 | 62.50 | 87.50 | 87.50 | 100.00 | 100.00 |
Median [IQR] | 75.00 [12.50] | 75.00 [0.00] | 87.00 [12.50] | 62.5 [0.00] | 87.50 [0.00] | 87.50 [0.00] | 100.00 [12.50] | 87.50 [12.50] |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Pachetti, E.; Del Corso, G.; Bardelli, S.; Colantonio, S. Few-Shot Conditional Learning: Automatic and Reliable Device Classification for Medical Test Equipment. J. Imaging 2024, 10, 167. https://doi.org/10.3390/jimaging10070167
Pachetti E, Del Corso G, Bardelli S, Colantonio S. Few-Shot Conditional Learning: Automatic and Reliable Device Classification for Medical Test Equipment. Journal of Imaging. 2024; 10(7):167. https://doi.org/10.3390/jimaging10070167
Chicago/Turabian StylePachetti, Eva, Giulio Del Corso, Serena Bardelli, and Sara Colantonio. 2024. "Few-Shot Conditional Learning: Automatic and Reliable Device Classification for Medical Test Equipment" Journal of Imaging 10, no. 7: 167. https://doi.org/10.3390/jimaging10070167