Deep-Learning-Based Detection of Infants with Autism Spectrum Disorder Using Auto-Encoder Feature Representation
Abstract
:1. Introduction
2. Proposed Method
2.1. Data Collection and Acoustic Feature Extraction
2.2. Pre-Trained AE for Acoustic Features
2.3. Establishing and Training the Deep Learning Model for ASD Detection
3. Performance Evaluation
4. Discussion
5. Conclusions
Author Contributions
Funding
Conflicts of Interest
References
- National Institute of Mental Health. Autism Spectrum Disorder. Available online: https://www.nimh.nih.gov/health/topics/autism-spectrum-disorders-asd/index.shtml (accessed on 26 October 2020).
- American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders: DSM-5; American Psychiatric Publishing: Washington, DC, USA, 2013. [Google Scholar]
- Centers for Disease Control and Prevention (CDC). Data & Statistics on Autism Spectrum Disorder. Available online: https://www.cdc.gov/ncbddd/autism/data.html (accessed on 26 October 2020).
- Fenske, E.C.; Zalenski, S.; Krantz, P.J.; McClannahan, L.E. Age at intervention and treatment outcome for autistic children in a comprehensive intervention program. Anal. Interv. Devel. Disabil. 1985, 5, 49–58. [Google Scholar] [CrossRef]
- Falkmer, T.; Anderson, K.; Falkmer, M.; Horlin, C. Diagnostic procedures in autism spectrum disorders: A systematic literature review. Eur. Child Adolesc. Psychiatry 2013, 22, 329–340. [Google Scholar] [CrossRef] [PubMed]
- Bailey, A.; Le Couteur, A.; Gottesman, I.; Bolton, P.; Simonoff, E.; Yuzda, E.; Rutter, M. Autism as a strongly genetic disorder: Evidence from a British twin study. Physiol. Med. 1995, 25, 63–77. [Google Scholar] [CrossRef] [PubMed]
- Duffy, F.H.; Als, H. A stable pattern of EEG spectral coherence distinguishes children with autism from neuro-typical controls—A large case control study. BMC Med. 2012, 10, 64. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Chaspari, T.; Lee, C.-C.; Narayanan, S.S. Interplay between verbal response latency and physiology of children with autism during ECA interactions. In Proceedings of the Annual Conference of the International Speech Communication Association (Interspeech), Portland, OR, USA, 9–13 September 2012; pp. 1319–1322. [Google Scholar]
- Baron-Cohen, S. Social and pragmatic deficits in autism: Cognitive or affective? J. Autism Dev. Disord. 1988, 18, 379–402. [Google Scholar] [CrossRef] [PubMed]
- Bonneh, Y.S.; Levanon, Y.; Dean-Pardo, O.; Lossos, L.; Adini, Y. Abnormal speech spectrum and increased pitch variability in young autistic children. Front. Hum. Neurosci. 2011, 4, 237. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Chericoni, N.; de Brito Wanderley, D.; Costanzo, V.; Diniz-Gonçalves, A.; Gille, M.L.; Parlato, E.; Cohen, D.; Apicella, F.; Calderoni, S.; Muratori, F. Pre-linguistic vocal trajectories at 6–18 months of age as early markers of autism. Front. Psychol. 2016, 7, 1595. [Google Scholar] [CrossRef] [PubMed]
- Alom, M.Z.; Taha, T.M.; Yakopcic, C.; Westberg, S.; Sidike, P.; Nasrin, M.S.; Hasan, M.; van Essen, B.C.; Awwal, A.A.S.; Asari, V.K. A state-of-the-art survey on deep learning theory and architectures. Electronics 2019, 8, 292. [Google Scholar] [CrossRef] [Green Version]
- Song, D.-Y.; Kim, S.Y.; Bong, G.; Kim, J.M.; Yoo, H.J. The use of artificial intelligence in screening and diagnosis of autism spectrum disorder: A literature review. J. Korean Acad. Child. Adolesc. Psychiatry 2019, 30, 145–152. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Santos, J.F.; Brosh, N.; Falk, T.H.; Zwaigenbaum, L.; Bryson, S.E.; Roberts, W.; Smith, I.M.; Szatmari, P.; Brian, J.A. Very early detection of autism spectrum disorders based on acoustic analysis of pre-verbal vocalizations of 18-month old toddlers. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Vancouver, BC, Canada, 26–31 May 2013; pp. 7567–7571. [Google Scholar]
- Li, M.; Tang, D.; Zeng, J.; Zhou, T.; Zhu, H.; Chen, B.; Zou, X. An automated assessment framework for atypical prosody and stereotyped idiosyncratic phrases related to autism spectrum disorder. Comput. Speech Lang. 2019, 56, 80–94. [Google Scholar] [CrossRef]
- Eyben, F.; Scherer, K.R.; Schuller, B.W.; Sundberg, J.; André, E.; Busso, C.; Devillers, L.Y.; Epps, J.; Laukka, P.; Narayanan, S.S.; et al. The Geneva minimalistic acoustic parameter set (GeMAPS) for voice research and affective computing. IEEE Trans. Affect. Comput. 2016, 7, 190–202. [Google Scholar] [CrossRef] [Green Version]
- Pokorny, F.B.; Schuller, B.W.; Marschik, P.B.; Brueckner, R.; Nyström, P.; Cummins, N.; Bölte, S.; Einspieler, C.; Falck-Ytter, T. Earlier identification of children with autism spectrum disorder: An automatic vocalisation-based approach. In Proceedings of the Annual Conference of the International Speech Communication Association (Interspeech), Stockholm, Sweden, 20–24 August 2017; pp. 309–313. [Google Scholar]
- Xing, C.; Ma, L.; Yang, X. Stacked denoise autoencoder based feature extraction and classification for hyperspectral images. J. Sens. 2016, 2016, 3632943. [Google Scholar] [CrossRef] [Green Version]
- Bong, G.; Kim, J.; Hong, Y.; Yoon, N.; Sunwoo, H.; Jang, J.; Oh, M.; Lee, K.; Jung, S.; Yoo, H. The feasibility and validity of autism spectrum disorder screening instrument: Behavior development screening for toddlers (BeDevel)—A pilot study. Autism Res. 2019, 12, 1112–1128. [Google Scholar] [CrossRef] [PubMed]
- Center for Autism Research. Social Communication Questionnaire (SCQ). Available online: https://www.carautismroadmap.org/social-communication-questionnaire-scq/?print=pdf (accessed on 26 October 2020).
- Center for Autism Research. Childhood Autism Rating Scale, 2nd Edition (CARS2). Available online: https://www.carautismroadmap.org/childhood-autism-rating-scale/?print=pdf (accessed on 26 October 2020).
- Center for Autism Research. Social Responsiveness Scale, 2nd Edition (SRS-2). Available online: https://www.carautismroadmap.org/social-responsiveness-scale/?print=pdf (accessed on 26 October 2020).
- Eyben, F.; Wöllmer, M.; Schuller, B. OpenSMILE—The Munich versatile and fast open-source audio feature extractor. In Proceedings of the 18th ACM International Conference on Multimedia, Firenze, Italy, 25–29 October 2010; pp. 1459–1462. [Google Scholar]
- Masci, J.; Meier, U.; Cireşan, D.; Schmidhuber, J. Stacked Convolutional Auto-Encoders for Hierarchical Feature Extraction. In Artificial Neural Networks and Machine-ICANN 2011; Honkela, T., Duch, W., Girolami, M., Kaski, S., Eds.; Springer: Berlin/Heidelberg, Germany, 2011; pp. 52–59. [Google Scholar]
- Sainath, T.; Kingsbury, B.; Ramabhadran, B. Auto-encoder bottleneck features using deep belief networks. In Proceedings of the 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Kyoto, Japan, 25–30 March 2012. [Google Scholar]
- Nachar, N. The Mann-Whitney U: A test for assessing whether two independent samples come from the same distribution. Tutor. Quant. Methods Psychol. 2008, 4, 13–20. [Google Scholar] [CrossRef]
- Le, L.; Patterson, A.; White, M. Supervised autoencoders: Improving generalization performance with unsupervised regularizers. In Advances in Neural Information Processing Systems; Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., Garnett, R., Eds.; Curran Associates, Inc.: New York, NY, USA, 2018; pp. 107–117. [Google Scholar]
- van Laarhoven, T. L2 regularization versus batch and weight normalization. arXiv 2017, arXiv:1706.05350. [Google Scholar]
- Ioffe, S.; Szegedy, C. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Proceedings of the International Conference on Machine Learning, Lille, France, 6–11 July 2015; pp. 448–456. [Google Scholar]
- Nair, V.; Hinton, G.E. Rectified linear units improve restricted Boltzmann machines. In Proceedings of the 27th International Conference on Machine Learning, Haifa, Israel, 21–24 June 2010; pp. 807–814. [Google Scholar]
- Srivastava, N.; Hinton, G.; Krizhevsky, A.; Sutskever, I.; Salakhutdinov, R. Dropout: A simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 2014, 15, 929–1958. [Google Scholar]
- Kingma, D.P.; Ba, J.L. ADAM: A method for stochastic optimization. In Proceedings of the 3rd International Conference on Learning Representations, San Diego, CA, USA, 7–9 May 2015; pp. 1–15. [Google Scholar]
- Abadi, M.; Barham, P.; Chen, J.; Chen, Z.; Davis, A.; Dean, J.; Devin, M.; Ghemawat, S.; Irving, G.; Isard, M.; et al. TensorFlow: A system for large-scale machine learning. In Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation, Savannah, GA, USA, 2–4 November 2016; pp. 265–283. [Google Scholar]
- Schuller, B.; Steidl, S.; Batliner, A. The Interspeech 2009 emotion challenge. In Proceedings of the Annual Conference of the International Speech Communication Association (Interspeech), Brighton, UK, 6–10 September 2009; pp. 312–315. [Google Scholar]
- van der Maaten, L.; Hinton, G. Visualizing data using t-SNE. J. Mach. Learn. Res. 2008, 9, 2579–2605. [Google Scholar]
Ages (Month) | No. of Subjects Diagnosed as ASD | No. of Subjects Diagnosed as TD | No. of Infant Subjects |
---|---|---|---|
6–12 months | 0 | 5 M/1 F | 5 M/1 F |
12–18 months | 1 M/3 F | 14 M/9 F | 15 M/12 F |
18–24 months | 3 M/3 F | 0 | 3 M/3 F |
Age (average ± SD) | 19.20 ± 2.52 | 14.72 ± 2.45 | 15.92 ± 3.17 |
Infant ID | Age (Months) on Initial Diagnosis Date | Gender | Initial Diagnosis Date (Year/Month/Day) | Definite Final Diagnosis Date (Year/Month/Day) | ASD/TD |
---|---|---|---|---|---|
1 | 18 | Male | 2018/07/28 | 2018/08/28 | TD |
2 | 18 | Male | 2017/07/27 | 2017/08/27 | TD |
3 | 10 | Male | 2018/08/10 | 2018/09/10 | TD |
4 | 13 | Male | 2017/06/10 | 2017/07/10 | TD |
5 | 22 | Female | 2018/01/31 | 2018/02/28 | ASD |
6 | 16 | Male | 2018/03/17 | 2018/04/17 | TD |
7 | 17 | Female | 2018/06/30 | 2018/07/30 | TD |
8 | 14 | Female | 2018/01/06 | 2018/02/06 | TD |
9 | 18 | Male | 2018/07/17 | 2018/08/17 | TD |
10 | 14 | Male | 2017/11/04 | 2017/12/04 | TD |
11 | 17 | Female | 2017/06/29 | 2017/07/29 | ASD |
12 | 12 | Female | 2018/01/20 | 2018/02/20 | TD |
13 | 9 | Male | 2017/02/18 | 2017/03/18 | TD |
14 | 18 | Female | 2017/03/04 | 2017/04/04 | ASD |
15 | 18 | Male | 2018/05/19 | 2018/06/19 | TD |
16 | 24 | Female | 2018/08/08 | 2018/09/08 | ASD |
17 | 19 | Male | 2018/02/24 | 2018/03/24 | ASD |
18 | 19 | Male | 2017/04/18 | 2017/05/18 | ASD |
19 | 18 | Female | 2017/03/04 | 2017/04/04 | TD |
20 | 12 | Male | 2016/12/31 | 2017/01/31 | TD |
21 | 16 | Female | 2018/03/16 | 2018/04/16 | TD |
22 | 20 | Male | 2017/10/14 | 2017/11/14 | ASD |
23 | 15 | Male | 2018/05/09 | 2018/06/09 | ASD |
24 | 17 | Female | 2017/02/04 | 2017/03/04 | TD |
25 | 16 | Male | 2018/03/17 | 2018/04/17 | TD |
26 | 12 | Male | 2018/03/29 | 2018/04/29 | TD |
27 | 17 | Female | 2017/01/25 | 2017/02/25 | TD |
28 | 17 | Male | 2018/02/08 | 2018/03/08 | ASD |
29 | 14 | Male | 2018/01/13 | 2018/02/13 | TD |
30 | 16 | Male | 2016/11/30 | 2016/12/30 | TD |
31 | 12 | Male | 2017/03/22 | 2017/04/22 | TD |
32 | 15 | Male | 2017/03/11 | 2017/04/11 | TD |
33 | 16 | Male | 2017/12/05 | 2018/01/05 | TD |
34 | 13 | Female | 2017/12/13 | 2018/01/13 | TD |
35 | 15 | Female | 2017/03/25 | 2018/04/25 | TD |
36 | 13 | Male | 2018/08/25 | 2018/09/25 | TD |
37 | 21 | Male | 2017/06/24 | 2017/07/24 | ASD |
38 | 14 | Male | 2017/02/22 | 2017/03/22 | TD |
39 | 14 | Male | 2018/01/27 | 2018/02/27 | TD |
Vocal Label | ASD | TD |
---|---|---|
0 | 80.134 (0.104) | 267.897 (0.250) |
1 | 314.405 (0.409) | 443.498 (0.414) |
2 | 33.241 (0.043) | 34.766 (0.032) |
3 | 8.311 (0.011) | 57.286 (0.054) |
4 | 333.400 (0.433) | 266.794 (0.249) |
Total | 769.491 | 1070.241 |
Models | SVM | BLSTM (eGeMAPS-54) | BLSTM (eGeMAPS-88) | BLSTM (AE-Encoded) | ||||
---|---|---|---|---|---|---|---|---|
Predicted To | ASD | TD | ASD | TD | ASD | TD | ASD | TD |
ASD | 62 | 18 | 170 | 103 | 196 | 99 | 215 | 98 |
TD | 413 | 632 | 305 | 547 | 279 | 551 | 260 | 552 |
Accuracy | 0.6178 | 0.6373 | 0.6640 | 0.6818 | ||||
Precision | 0.1305 | 0.3579 | 0.4126 | 0.4526 | ||||
Recall | 0.7750 | 0.6227 | 0.6644 | 0.6869 | ||||
F1 score | 0.2234 | 0.4545 | 0.5091 | 0.5457 | ||||
UAR | 0.5514 | 0.5997 | 0.6302 | 0.6509 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Lee, J.H.; Lee, G.W.; Bong, G.; Yoo, H.J.; Kim, H.K. Deep-Learning-Based Detection of Infants with Autism Spectrum Disorder Using Auto-Encoder Feature Representation. Sensors 2020, 20, 6762. https://doi.org/10.3390/s20236762
Lee JH, Lee GW, Bong G, Yoo HJ, Kim HK. Deep-Learning-Based Detection of Infants with Autism Spectrum Disorder Using Auto-Encoder Feature Representation. Sensors. 2020; 20(23):6762. https://doi.org/10.3390/s20236762
Chicago/Turabian StyleLee, Jung Hyuk, Geon Woo Lee, Guiyoung Bong, Hee Jeong Yoo, and Hong Kook Kim. 2020. "Deep-Learning-Based Detection of Infants with Autism Spectrum Disorder Using Auto-Encoder Feature Representation" Sensors 20, no. 23: 6762. https://doi.org/10.3390/s20236762