Next Article in Journal
Enhanced Sensing Behavior of Three-Dimensional Microfluidic Paper-Based Analytical Devices (3D-μPADs) with Evaporation-Free Enclosed Channels for Point-of-Care Testing
Previous Article in Journal
Prospective Observational COVID-19 Screening and Monitoring of Asymptomatic Cancer Center Health-Care Workers with a Rapid Serological Test
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Can the MMPI Predict Adult ADHD? An Approach Using Machine Learning Methods

1
Department of Psychiatry, Hanyang University Medical Center, Seoul 04763, Korea
2
Department of Nursing, College of Nursing and Health, Kongju National University, Gongju 32588, Korea
*
Author to whom correspondence should be addressed.
Diagnostics 2021, 11(6), 976; https://doi.org/10.3390/diagnostics11060976
Submission received: 15 March 2021 / Revised: 25 April 2021 / Accepted: 27 May 2021 / Published: 28 May 2021
(This article belongs to the Section Machine Learning and Artificial Intelligence in Diagnostics)

Abstract

:
(1) Background: Adult attention-deficit/hyperactivity disorder (ADHD) symptoms cause various social difficulties due to attention deficit and impulsivity. In addition, in contrast to ADHD in childhood, ADHD in adulthood is difficult to diagnose due to mixed psychopathologies. This study aimed to determine whether it is possible to predict ADHD symptoms in adults using the Minnesota Multiphasic Personality Inventory-2 (MMPI-2) with machine learning (ML) techniques; (2) Methods: Data collected from 5726 college students were analyzed. The MMPI-2-Restructured Form (MMPI-2-RF) was used, and ADHD symptoms in adults were evaluated using the Attention-Deficit/Hyperactivity Disorder Self-Report Scale (ASRS). For statistical analysis, three ML algorithms were used, i.e., K-nearest neighbors (KNN), linear discriminant analysis (LDA), and random forest, with the ASRS evaluation result as the dependent variable and the 50 MMPI-2-RF scales as predictors; (3) Results: When the KNN, LDA, and random forest techniques were applied, the accuracy was 93.1%, 91.2%, and 93.6%, respectively, and the area under the curve (AUC) was 0.722, 0.806, and 0.790, respectively. The AUC of the LDA method was the largest, with an excellent level of diagnostic accuracy; (4) Conclusions: ML using the MMPI-2 in a large group could provide reliable accuracy in screening for adult ADHD.

1. Introduction

Attention-deficit/hyperactivity disorder (ADHD) is a neurodevelopmental disorder characterized by inattention, hyperactivity, and impulsivity [1]. Around 50–80% of patients diagnosed with ADHD in childhood report that symptoms persist until adulthood [2,3].
ADHD has been regarded as a childhood-specific disorder for a long time because it is thought that ADHD symptoms improve as children develop [4]; however, long-term follow-up studies report that patients diagnosed with ADHD in childhood meet the diagnostic criteria even as adults [5,6,7,8,9]. Adult ADHD can generally be defined as late adolescents and adults (17 years of age or older). It is a mental disorder that includes a combination of problems, such as inattention, hyperactivity, and impulsiveness, and these problems cause a social functional impairment. The main symptoms of ADHD in adults appear to be carelessness and impulsiveness. Nevertheless, ADHD in adults show marked improvement in hyperactivity symptoms [1,10].
Adults with ADHD may have difficulties in interpersonal relationships, and it is known that they face problems at work, such as the inability to organize and process work efficiently [11,12]. ADHD in adults has a different pattern compared with that of ADHD in children; thus, it is often underdiagnosed in adults and not recognized by clinicians [13]. The main diagnostic features of ADHD are carelessness and hyperactivity/impulsivity; however, in adult ADHD, symptoms such as problems with executive function, emotion regulation, self-concept, self-esteem, and interpersonal relationships are the main features in addition to the main diagnostic characteristics [14,15,16,17,18,19,20,21,22]. Consequently, it is difficult to accurately screen adult ADHD patients only using the diagnostic criteria of the Diagnostic and Statistical Manual of Mental Disorders (DSM) or International Classification of Diseases (ICD) system in the clinical field. Therefore, there is a need for an effective screening tool for adult ADHD [23].
The number of adults diagnosed with ADHD in the United States has steadily increased over the past 20 years [24,25]. The Centers for Disease Control and Prevention (CDC) examined the treatment of adult ADHD from 2002 to 2005 and reported that the percentage of people receiving stimulants for ADHD treatment increased by 90.0% [26], and they were often misused or abused for leisure and not used for therapeutic purposes [27,28,29,30]. Moreover, patients diagnosed with ADHD are given extra time to complete tasks for exams and classes and are provided with special benefits such as lighter workloads, preferred seating, and no penalties for misspellings [31]. Therefore, it is important to identify individuals who falsely report symptoms of ADHD.
Currently, the Adult ADHD Self-Report Scale (ASRS) developed by the World Health Organization can be used to screen for adult ADHD. It is a self-reported scale consisting of 18 items suitable for adulthood based on the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV) diagnostic criteria. It has adequate sensitivity (68.7%), excellent specificity (99.5%), and excellent total classification accuracy (97.9%) [32]. However, it is difficult to confirm the reliability of self-reports due to insincere reports.
There have been efforts to use the Minnesota Multiphasic Personality Inventory-2 (MMPI-2) as a detection tool for exaggerated or hidden ADHD [31,33]. The MMPI-2 is one of the most widely used objective personality tests in the world for psychopathology and emotional function evaluation [34,35,36]. Among them, the MMPI-2-Restructured Form (MMPI-2-RF) is a psychometrically improved and more efficient short version that emphasizes the clinical scales reconstructed in MMPI-2 [37]. The MMPI-2 is widely used in psychological evaluation and can simultaneously evaluate response validity during clinical evaluation [31]; thus, it is most commonly used for psychopathological evaluation in medical and forensic fields [38,39,40].
Machine learning (ML) is defined as a computational strategy that automatically determines methods and parameters to arrive at an optimal solution to a problem [41]. A machine learns from data with minimal human intervention, recognizes patterns in data, and proposes to improve diagnostic and prognostic accuracy. This approach appears to be particularly useful in predicting human behavior, including high-risk behavior, and it can be applied to improve the effectiveness and goals of prevention programs and interventions [41]. In comparison with conventional statistical approaches, ML technology has advantages in accuracy and scalability in terms of prediction [42]; thus, several recent studies have applied ML technology to differentiate ADHD from control groups. These studies showed moderate accuracy using linear classifiers [43,44,45,46,47], and it seems that a greater number of more precise studies are needed to predict ADHD through ML.
This study aimed to differentiate risk groups with significant adult ADHD symptoms by applying the latest ML algorithms using the MMPI-2. In order to compensate for the under-assessment of symptoms due to mixed pathologies and self-reports of adult ADHD by using a valid scale that can detect the examinee’s psychopathology exaggeration and clinical scales to identify personality traits related to psychopathology with the MMPI-2, this study aimed to overcome the problem of screening for adult ADHD.

2. Materials and Methods

We used part of the dataset of a survey conducted at Kongju National University [48,49,50]. The subjects were informed of the guarantee of anonymity and the use of the survey results for research purposes, and written consent was obtained. The data came from 5806 respondents who completed both the MMPI and ASRS. Data from 5726 respondents were analyzed, excluding the data of 51 subjects who had many omissions and 29 subjects who did not meet the validity criteria of the MMPI-2-RF. This study was approved by the Kongju National University Institutional Review Board (approval No. KNU2015-38, 28 July 2015).
For the MMPI-2-RF, a total of 50 scales that can effectively measure the clinical significance of MMPI-2 items were developed and consisted of 8 validity scales and 42 major scales. The validity criteria for the MMPI were not responding to 30 or more non-response questions, a Variable Response Inconsistency (VRIN) t-score of 80 or more, and a True Response Inconsistency (TRIN) t-score of 80 or more according to the nonclinical scene guidelines in the MMPI-2 manual [51]. In this study, we used the Korean version of the MMPI-2-RF, and its reliability and validity were verified [52].
The ASRS is a screening tool consisting of 18 questions about the latest frequency of symptoms based on the DSM-IV criteria symptoms of adult ADHD developed by the World Health Organization [4,32]. The ASRS questions are expressed slightly differently from those on the ADHD rating scale for children, and as a screening test with high sensitivity and specificity including the content of ADHD symptoms in adults, the tool has great applicability in clinical and research settings [32].
Each item evaluates the frequency of occurrence of specific symptoms of ADHD over the past 6 months on a 5-point scale, and response options range from 0 to 4 points per question. In our study, the ASRS (+) group was defined as subjects whose total score in the first 9 questions (part 1) or the last 9 questions (part 2) exceeded 21 [53].
In this study, the MMPI-2-RF and ASRS were inputted into the ML algorithm. We used three ML classification methods: K-nearest neighbors (KNN), linear discriminant analysis (LDA), and random forest. Of the ML algorithms, K-nearest neighbors (KNN) algorithm is non-parametric classification method, which collects the existing classes and classifies new classes based on the comparison measure [54]. Linear discriminant analysis (LDA) is the correlation between categorical variable and an interrelated variable is considered [55]. Additionally, random forest algorithm is the grouping of tree predictors such that each tree is influenced by the values of a random vector experimented individually and with the same dissemination for all trees in the forest [56]. Outcome variables were analyzed by dividing the subjects into ASRS (+) and ASRS (−) groups, and 50 scales of the MMPI-RF were used as explanatory variables. To assess diagnostic accuracy, we used the area under the curve (AUC), which reflects how good the test is at distinguishing between patients with disease and those without disease. An AUC of 0.5 suggests no discrimination, 0.7–0.8 is considered acceptable, 0.8–0.9 is considered excellent, and more than 0.9 is considered outstanding [57].
Of the total sample, 20% was used as the test dataset, and the remaining 80% were used as the validation dataset; the data were randomly separated into the training dataset and test dataset. To prevent machine learning from overfitting, a validation dataset was added as a verification step. If the model was overfitting, the prediction rate or the error rate would fall, so 20% of the entire dataset was designated as the test dataset and 80% as the validation dataset [58,59]. All statistical analyses were performed using JASP v0.14.1 (Amsterdam, The Netherlands. Released on 17 December 2020) [60]. All p values were obtained using a two-sided test, and p < 0.05 was considered statistically significant.

3. Results

3.1. Experimental Results

3.1.1. General Characteristics

A total of 347 subjects (6.0%) were above the cutoff point in the ASRS part 1, and 109 subjects (1.9%) were above the cutoff point in the ASRS part 2. A total of 381 patients (6.6%) were classified into the ASRS (+) group as patients exceeding the cutoff point in part 1 or 2. There were 2851 males (49.8%), and the mean age was 19.8 years (SD = 1.3). There was no significant difference in age between the two groups (t = 0.428, p = 0.669). The total ASRS score was 4.53 (SD = 0.93) in ASRS (+) group and 4.69 (SD = 1.41) in ASRS (-) group. However, all restructured clinical (RC) scales showed high scores in the ASRS (+) group (Table 1).

3.1.2. Prediction Accuracy of ML Model

Of the 5726 subjects for ML, 3664 subjects were used as the training dataset, 917 subjects were used as the validation dataset, and 1145 subjects were used as the test dataset. In this study, the prediction accuracy of the KNN, LDA, and random forest methods was 93.1%, 91.2%, and 93.6%, respectively (Table 2).

3.1.3. AUC of ML Model

The AUC of the KNN, LDA, and random forest methods was 0.722 (recall = 0.931, precision = 0.900), 0.806 (recall = 0.912, precision = 0.899), and 0.790 (recall = 0.936, precision = 0.916), respectively. The AUC of the LDA method was the largest, with an excellent level of diagnostic accuracy, and the rest had an acceptable level of diagnostic accuracy (Table 3, Figure 1).

4. Discussion

This study aimed to predict and report adult ADHD symptoms using ML with 50 scales of the MMPI-2-RF, the most commonly used self-report evaluation tool. The three ML algorithms showed high accuracy (91.2–93.6%). Although there were differences according to each method, ADHD symptoms in adults were predicted with an excellent level of accuracy. Therefore, using a common screening tool, the MMPI-2-RF, risk factors related to poor concentration, a symptom of ADHD in adults, may be predicted using ML algorithms.
In our study, most of the clinical scales were high in the ASRS (+) group. This indicates emotional difficulties in adult ADHD, which can be screened using tools that measure existing psychopathology. Although several emotional difficulties may be attributed to concentration disorders and behavioral symptoms, a general increase in psychopathology symptoms is not specific to adult ADHD.
A major difficulty in screening for adult ADHD is that it often has multiple comorbidities and coexistence pathologies [61]. These symptoms of ADHD may be mistaken to be part of other mental disorders requiring clinical attention [62]. Therefore, the ASRS is used as a differential diagnosis tool for various psychopathologies (e.g., substance use disorder) that are mixed with ADHD and thus are difficult to diagnose [32].
In particular, it is important to differentiate the diagnoses of various mental disorders in ADHD including adult ADHD; for this purpose, discrimination studies through ML are being conducted [54,55,56,57]. However, there are various diagnostic tools and methods for ADHD diagnosis, and although ML techniques may be used to diagnose ADHD through neurological tests, a screening tool for adult ADHD diagnosis (ASRS, Conners rating scale, etc.) is still used in the clinical field. Studies that distinguish the symptoms of ADHD using psychological measurement tools are insufficient. As a comprehensive evaluation is required to diagnose adult ADHD, it is possible to initially screen for adult ADHD using efficient screening tools such as the ASRS, which has the advantage of allowing clinicians to discriminate various psychopathological characteristics in advance. A study has demonstrated the clinical use of a screening tool for discriminating autism and ADHD with an accuracy of 82% [63]. Similarly, the accuracy level of the three algorithms in this study was high (91.2–93.6%), and in the case of the LDA technique, an AUC of 0.806 with an accuracy of 91.2% is expected in clinical settings.
However, ML has the disadvantage of not being able to accurately describe the relationship between input and output [64]; thus, it is difficult to precisely determine the complex influence of selected characteristics in classification models. Therefore, if a positive finding is found in the screening test using the ASRS, clinicians should determine whether there is another psychiatric diagnosis, and in adults, it is necessary to confirm the presence of other psychiatric symptoms based on the developmental history [65]. In the case of adult ADHD, concomitant diseases are common, and the evaluation itself is difficult and complex; thus, adult ADHD diagnosis is challenging with the existing criteria, and there is a need to develop specific diagnostic criteria [66].
This study has some limitations. First, the results may not be representative of the entire population as the survey was conducted at one university. In particular, age homogeneity may limit the applicability of the results. Second, the study consisting of a self-reported questionnaire on adult ADHD symptoms in a non-clinical group lacked clinical diagnosis. In addition, there was no information on coexisting diseases or psychiatric treatment history. Considering that this was a retrospective analysis using data in a large university subjects, it was difficult to obtain the relevant information. In the future, the findings will need to be confirmed through follow-up studies that continuously evaluate the symptoms of ADHD in various populations, including clinical evaluation and analysis (factor analysis, principal component analysis, etc.). In addition, this study selected adult ADHD using KNN, LDA, and random forest among machine learning techniques based on clinical theory, but we will suggest extended techniques that use various machine learning classification techniques (vector machine or neural network).
The strength of this study is that it is a large-scale, multi-faceted analysis of the same group of college students. In particular, the limitations of self-report tests were offset using the valid scale of the MMPI-2-RF. In the future, it may be possible to evaluate adult ADHD symptoms in a setting where only the MMPI-2 is used, such as assessment for employment and physical examination for military service.
Thus far, studies that discriminate or classify ADHD through ML algorithms have mostly been based on neurological tests (fMRI, EEG, etc.) [67,68,69]. This study has great significance in that ML was used with the ASRS, which is a screening tool for ADHD, and the MMPI-2, which discriminates psychopathological personality characteristics.
The screening of adult ADHD symptoms is not conclusive with only the MMPI-2, and the use of the MMPI-2-RF makes it possible to obtain valid test results for various aspects of psychopathology. If used together with ML, it will be helpful as an auxiliary tool. In the future, if clinical data are added and analyzed for a wider variety of groups, the possibility of using the MMPI-2 in the evaluation of ADHD symptoms in adults is expected to increase.

5. Conclusions

This study demonstrated that ML using the MMPI-2-RF could provide reliable accuracy in classifying and predicting adult ADHD symptoms. This will help clinicians to detect and treat adult ADHD symptoms early in clinical settings.

Author Contributions

K.L. and S.K. contributed to the design of the study, interpretation of the results, and preparation of the manuscript. K.L. and H.-K.L. coordinated the data acquisition. K.L. and S.K. wrote the draft manuscript. K.L., H.-K.L. and S.K. contributed to the critical revision of the article for important intellectual content. All authors have read and agreed to the published version of the manuscript.

Funding

This research was supported by the Basic Science Research Program of the National Research Foundation of Korea (NRF) funded by the Ministry of Education (NRF-2018R1D1A1B07050245), and the Technology Innovation Program (20012931) funded by the Ministry of Trade, Industry and Energy (MOTIE, Korea). This funding source had no role in the design of this study and will not have any role during its execution, analyses, interpretation of the data, or decision to submit results.

Institutional Review Board Statement

The study was conducted according to the guidelines of the Declaration of Helsinki and approved by the Institutional Review Board of Kongju National University Institutional Review Board (approval No. KNU2015-38, 28 July 2015).

Informed Consent Statement

Written informed consent has been obtained from the patient(s) to publish this paper.

Data Availability Statement

The data presented in this study are available on request from the corresponding author. The data are not publicly available due to ethical restrictions.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Association, A.P. Diagnostic and Statistical Manual of Mental Disorders (DSM-5®); American Psychiatric Pub: Washington, DC, USA, 2013. [Google Scholar]
  2. Barkley, R.A.; Fischer, M.; Smallish, L.; Fletcher, K. The persistence of attention-deficit/hyperactivity disorder into young adulthood as a function of reporting source and definition of disorder. J. Abnorm. Psychol. 2002, 111, 279. [Google Scholar] [CrossRef]
  3. Wender, P.H. Attention-deficit hyperactivity disorder in adults. Psychiatr. Clin. N. Am. 1998, 21, 761–774. [Google Scholar] [CrossRef]
  4. Kessler, R.C.; Adler, L.A.; Gruber, M.J.; Sarawate, C.A.; Spencer, T.; Van Brunt, D.L. Validity of the World Health Organization Adult ADHD Self-Report Scale (ASRS) Screener in a representative sample of health plan members. Int. J. Methods Psychiatr. Res. 2007, 16, 52–65. [Google Scholar] [CrossRef] [PubMed]
  5. De Graaf, R.; Kessler, R.C.; Fayyad, J.; ten Have, M.; Alonso, J.; Angermeyer, M.; Borges, G.; Demyttenaere, K.; Gasquet, I.; de Girolamo, G.; et al. The prevalence and effects of adult attention-deficit/hyperactivity disorder (ADHD) on the performance of workers: Results from the WHO World Mental Health Survey Initiative. Occup. Environ. Med. 2008, 65, 835–842. [Google Scholar] [CrossRef] [Green Version]
  6. Montejano, L.; Sasane, R.; Hodgkins, P.; Russo, L.; Huse, D. Adult ADHD: Prevalence of diagnosis in a US population with employer health insurance. Curr. Med. Res. Opin. 2011, 27 (Suppl. 2), 5–11. [Google Scholar] [CrossRef]
  7. Montes, L.G.A.; García, A.O.H.; Ricardo-Garcell, J. ADHD prevalence in adult outpatients with nonpsychotic psychiatric illnesses. J. Atten. Disord. 2007, 11, 150–156. [Google Scholar] [CrossRef]
  8. Park, S.; Cho, M.J.; Chang, S.M.; Jeon, H.J.; Cho, S.J.; Kim, B.S.; Bae, J.N.; Wang, H.R.; Ahn, J.H.; Hong, J.P. Prevalence, correlates, and comorbidities of adult ADHD symptoms in Korea: Results of the Korean epidemiologic catchment area study. Psychiatry Res. 2011, 186, 378–383. [Google Scholar] [CrossRef] [PubMed]
  9. Weiss, M.; Murray, C.; Weiss, G. Adults with attention-deficit/hyperactivity disorder: Current concepts. J. Psychiatr. Pract. 2002, 8, 99–111. [Google Scholar] [CrossRef]
  10. Kessler, R.C.; Green, J.G.; Adler, L.A.; Barkley, R.A.; Chatterji, S.; Faraone, S.V.; Finkelman, M.; Greenhill, L.L.; Gruber, M.J.; Jewell, M.; et al. Structure and diagnosis of adult attention-deficit/hyperactivity disorder: Analysis of expanded symptom criteria from the Adult ADHD Clinical Diagnostic Scale. Arch. Gen. Psychiatry 2010, 67, 1168–1178. [Google Scholar] [CrossRef] [Green Version]
  11. Sibley, M.H.; Pelham, W.E.; Molina, B.S.G.; Gnagy, E.M.; Waschbusch, D.A.; Garefino, A.C.; Kuriyan, A.B.; Babinski, D.E.; Karch, K.M. Diagnosing ADHD in adolescence. J. Consult. Clin. Psychol. 2012, 80, 139–150. [Google Scholar] [CrossRef] [Green Version]
  12. Ward, M.F.; Wender, P.H.; Reimherr, F.W. The Wender Utah Rating Scale: An aid in the retrospective diagnosis of childhood attention deficit hyperactivity disorder. Am. J. Psychiatry 1993, 150, 885–890. [Google Scholar] [CrossRef]
  13. Adler, L.A.; Spencer, T.; Faraone, S.V.; Kessler, R.C.; Howes, M.J.; Biederman, J.; Secnik, K. Validity of pilot Adult ADHD Self- Report Scale (ASRS) to Rate Adult ADHD symptoms. Ann. Clin. Psychiatry 2006, 18, 145–148. [Google Scholar] [CrossRef]
  14. Barkley, R.A. Behavioral inhibition, sustained attention, and executive functions: Constructing a unifying theory of ADHD. Psychol. Bull. 1997, 121, 65–94. [Google Scholar] [CrossRef] [Green Version]
  15. Canu, W.H.; Carlson, C.L. Rejection sensitivity and social outcomes of young adult men with ADHD. J. Atten. Disord. 2007, 10, 261–275. [Google Scholar] [CrossRef] [Green Version]
  16. Conners, C.K.; Erhardt, D.; Sparrow, E.P. Conners’ Adult ADHD Rating Scales (CAARS): Technical Manual; Multi-Health Systems: North Tonawanda, NY, USA, 1999. [Google Scholar]
  17. Corbisiero, S.; Morstedt, B.; Bitto, H.; Stieglitz, R.D. Emotional Dysregulation in Adults with Attention-Deficit/Hyperactivity Disorder-Validity, Predictability, Severity, and Comorbidity. J. Clin. Psychol. 2017, 73, 99–112. [Google Scholar] [CrossRef]
  18. Faraone, S.; Biederman, J.; Spencer, T. Diagnostic efficiency of symptom items for identifying adult ADHD. J. Adhd. Relat. Disord. 2010, 1, 38–48. [Google Scholar]
  19. Musser, E.D.; Nigg, J.T. Emotion Dysregulation Across Emotion Systems in Attention Deficit/Hyperactivity Disorder. J. Clin. Child Adolesc. Psychol. 2019, 48, 153–165. [Google Scholar] [CrossRef]
  20. Safren, S.A.; Sprich, S.E.; Cooper-Vince, C.; Knouse, L.E.; Lerner, J.A. Life impairments in adults with medication-treated ADHD. J. Atten. Disord. 2010, 13, 524–531. [Google Scholar] [CrossRef]
  21. Shaw-Zirt, B.; Popali-Lehane, L.; Chaplin, W.; Bergman, A. Adjustment, social skills, and self-esteem in college students with symptoms of ADHD. J. Atten. Disord. 2005, 8, 109–120. [Google Scholar] [CrossRef]
  22. Willcutt, E.G.; Doyle, A.E.; Nigg, J.T.; Faraone, S.V.; Pennington, B.F. Validity of the executive function theory of attention-deficit/hyperactivity disorder: A meta-analytic review. Biol. Psychiatry 2005, 57, 1336–1346. [Google Scholar] [CrossRef]
  23. Yoo, A.K.E.-H.L.; Sang, S.J.S.-T.H.; Kim, H.H.J.-H. Validation of the Korean Version of Barkley Deficits in Executive Functioning Scale Short-Form. Korean J. Clin. Psychol. 2019, 38, 247–256. [Google Scholar]
  24. McCabe, S.E.; Teter, C.J.; Boyd, C.J. The use, misuse and diversion of prescription stimulants among middle and high school students. Subst. Use Misuse 2004, 39, 1095–1116. [Google Scholar] [CrossRef] [PubMed]
  25. Rushton, J.L.; Whitmire, J.T. Pediatric stimulant and selective serotonin reuptake inhibitor prescription trends: 1992 to 1998. Arch. Pediatr. Adolesc. Med. 2001, 155, 560–565. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  26. Centers for Disease and Prevention. Attention Deficit/Hyperactivity Disorder (ADHD): Data and Statistics; Centers for Disease and Prevention: Washington, DC, USA, 2013.
  27. Babcock, Q.; Byrne, T. Student perceptions of methylphenidate abuse at a public liberal arts college. J. Am. Coll. Health 2000, 49, 143–145. [Google Scholar] [CrossRef] [PubMed]
  28. DeSantis, A.D.; Webb, E.M.; Noar, S.M. Illicit use of prescription ADHD medications on a college campus: A multimethodological approach. J. Am. Coll. Health 2008, 57, 315–324. [Google Scholar] [CrossRef] [PubMed]
  29. McCabe, S.E.; Teter, C.J.; Boyd, C.J.; Knight, J.R.; Wechsler, H. Nonmedical use of prescription opioids among US college students: Prevalence and correlates from a national survey. Addict. Behav. 2005, 30, 789–805. [Google Scholar] [CrossRef] [PubMed]
  30. DeSantis, A.D.; Hane, A.C. “Adderall is definitely not a drug”: Justifications for the illegal use of ADHD stimulants. Subst. Use Misuse 2010, 45, 31–46. [Google Scholar] [CrossRef]
  31. Harp, J.P.; Jasinski, L.J.; Shandera-Ochsner, A.L.; Mason, L.H.; Berry, D.T. Detection of malingered ADHD using the MMPI-2-RF. Psychol. Inj. Law 2011, 4, 32–43. [Google Scholar] [CrossRef]
  32. Kessler, R.C.; Adler, L.; Ames, M.; Demler, O.; Faraone, S.; Hiripi, E.; Howes, M.J.; Jin, R.; Secnik, K.; Spencer, T. The World Health Organization Adult ADHD Self-Report Scale (ASRS): A short screening scale for use in the general population. Psychol. Med. 2005, 35, 245. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  33. Robinson, E.V.; Rogers, R. Detection of feigned ADHD across two domains: The MMPI-2-RF and CAARS for faked symptoms and TOVA for simulated attention deficits. J. Psychopathol. Behav. Assess. 2018, 40, 376–385. [Google Scholar] [CrossRef]
  34. Greene, R.L. The MMPI-2: An Interpretive Manual; Allyn & Bacon: Boston, MA, USA, 2000. [Google Scholar]
  35. Lubin, B.; Larsen, R.M.; Matarazzo, J.D. Patterns of psychological test usage in the United States: 1935–1982. Am. Psychol. 1984, 39, 451. [Google Scholar] [CrossRef]
  36. Lubin, B.; Larsen, R.M.; Matarazzo, J.D.; Seever, M. Psychological test usage patterns in five professional settings. Am. Psychol. 1985, 40, 857. [Google Scholar] [CrossRef]
  37. Tellegen, A.; Ben-Porath, Y.; McNulty, J.; Arbisi, P.; Graham, J.; Kaemmer, B. MMPI-2 Restructured Clinical (RC) Scales: Development, Validation, and Interpretation; University of Minnesota Press: Minneapolis, MN, USA, 2003. [Google Scholar]
  38. Ben-Porath, Y.S. Interpreting the MMPI-2-RF; University of Minnesota Press: Minneapolis, MN, USA, 2012. [Google Scholar]
  39. Ben-Porath, Y.S.; Butcher, J.N. The comparability of MMPI and MMPI-2 scales and profiles. Psychol. Assess. J. Consult. Clin. Psychol. 1989, 1, 345. [Google Scholar] [CrossRef]
  40. Pope, K.S.; Butcher, J.N.; Seelen, J. The MMPI, MMPI-2, & MMPI-A in Court: A Practical Guide for Expert Witnesses and Attorneys; American Psychological Association: Washington, DC, USA, 2006. [Google Scholar]
  41. Dwyer, D.B.; Falkai, P.; Koutsouleris, N. Machine Learning Approaches for Clinical Psychology and Psychiatry. Annu. Rev. Clin. Psychol. 2018, 14, 91–118. [Google Scholar] [CrossRef]
  42. Linthicum, K.P.; Schafer, K.M.; Ribeiro, J.D. Machine learning in suicide science: Applications and ethics. Behav. Sci. Law 2019, 37, 214–222. [Google Scholar] [CrossRef] [PubMed]
  43. Barry, R.J.; Clarke, A.R.; Johnstone, S.J. A review of electrophysiology in attention-deficit/hyperactivity disorder: I. Qualitative and quantitative electroencephalography. Clin. Neurophysiol. 2003, 114, 171–183. [Google Scholar] [CrossRef]
  44. Buchsbaum, M.; Wender, P. Average evoked responses in normal and minimally brain dysfunctioned children treated with amphetamine: A preliminary report. Arch. Gen. Psychiatry 1973, 29, 764–770. [Google Scholar] [CrossRef] [PubMed]
  45. Robaey, P.; Breton, F.; Dugas, M.; Renault, B. An event-related potential study of controlled and automatic processes in 6-8-year-old boys with attention deficit hyperactivity disorder. Electroencephalogr. Clin. Neurophysiol. 1992, 82, 330–340. [Google Scholar] [CrossRef]
  46. Satterfield, J.H.; Braley, B.W. Evoked potentials and brain maturation in hyperactive and normal children. Electroencephalogr. Clin. Neurophysiol. 1977, 43, 43–51. [Google Scholar] [CrossRef]
  47. Smith, J.L.; Johnstone, S.J.; Barry, R.J. Aiding diagnosis of attention-deficit/hyperactivity disorder and its subtypes: Discriminant function analysis of event-related potential data. J. Child Psychol. Psychiatry 2003, 44, 1067–1075. [Google Scholar] [CrossRef] [PubMed]
  48. Kim, S.; Lee, H.-K.; Lee, K. Assessment of suicidal risk using Minnesota multiphasic personality inventory-2 restructured form. BMC Psychiatry 2020, 20, 81. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  49. Kim, S.; Lee, H.-K.; Lee, K. Which PHQ-9 items can effectively screen for suicide? Machine learning approaches. Int. J. Environ. Res. Public Health 2021, 18, 3339. [Google Scholar] [CrossRef]
  50. Lee, K.; Lee, H.-K.; Kim, S.; Kim, S.H. The relationship between circadian typology and lifetime experiences of hypomanic symptoms. Psychiatry Res. 2021, 298, 113788. [Google Scholar] [CrossRef]
  51. Han, K.H.; Moon, K.J.; Lim, J.Y.; Kim, J.S. MMPI-2-RF: Manual for Korean Version of MMPI-2-RF; Maumsarang Ltd.: Seoul, Korea, 2011. [Google Scholar]
  52. Kim, J.H.; Lee, E.H.; Joung, Y.S. The WHO Adult ADHD Self-Report Scale: Reliability and Validity of the Korean Version. Psychiatry Investig. 2013, 10, 41–46. [Google Scholar] [CrossRef] [Green Version]
  53. Instanes, J.T.; Haavik, J.; Halmoy, A. Personality Traits and Comorbidity in Adults with ADHD. J. Atten. Disord. 2016, 20, 845–854. [Google Scholar] [CrossRef]
  54. Hand, D.J. Principles of data mining. Drug Saf. 2007, 30, 621–622. [Google Scholar] [CrossRef]
  55. McLachlan, G.J. Discriminant Analysis and Statistical Pattern Recognition; John Wiley & Sons: Hoboken, NJ, USA, 2004; Volume 544. [Google Scholar]
  56. Breiman, L. Random forests. Mach. Learn. 2001, 45, 5–32. [Google Scholar] [CrossRef] [Green Version]
  57. Hosmer, D.W., Jr.; Lemeshow, S.; Sturdivant, R.X. Applied Logistic Regression; John Wiley & Sons: Hoboken, NJ, USA, 2013; Volume 398. [Google Scholar]
  58. Nguyen, Q.H.; Ly, H.-B.; Ho, L.S.; Al-Ansari, N.; Le, H.V.; Tran, V.Q.; Prakash, I.; Pham, B.T. Influence of Data Splitting on Performance of Machine Learning Models in Prediction of Shear Strength of Soil. Math. Probl. Eng. 2021, 2021, 4832864. [Google Scholar] [CrossRef]
  59. Deo, R.C. Machine Learning in Medicine. Circulation 2015, 132, 1920–1930. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  60. Team, J. JASP; 0.14.1; University of Amsterdam: Amsterdam, The Netherlands, 2020. [Google Scholar]
  61. Faraone, S.V.; Biederman, J.; Mick, E. The age-dependent decline of attention deficit hyperactivity disorder: A meta-analysis of follow-up studies. Psychol. Med. 2006, 36, 159–165. [Google Scholar] [CrossRef]
  62. Gizer, I.R.; Ficks, C.; Waldman, I.D. Candidate gene studies of ADHD: A meta-analytic review. Hum. Genet. 2009, 126, 51–90. [Google Scholar] [CrossRef] [PubMed]
  63. Duda, M.; Haber, N.; Daniels, J.; Wall, D.P. Crowdsourced validation of a machine-learning classification system for autism and ADHD. Transl. Psychiatry 2017, 7, e1133. [Google Scholar] [CrossRef]
  64. Singh, Y.; Bhatia, P.K.; Sangwan, O. A review of studies on machine learning techniques. Int. J. Comput. Sci. Secur. 2007, 1, 70–84. [Google Scholar]
  65. Lee, S.; Choi, J.-W.; Kim, K.-M.; Kim, J.W.; Kim, S.; Kang, T.; Kim, J.I.; Lee, Y.S.; Kim, B.; Han, D.H. The guideline of diagnosis and treatment of attention-deficit hyperactivity disorder: Developed by ADHD translational research center. J. Korean Acad. Child Adolesc. Psychiatry 2016, 27, 236–266. [Google Scholar] [CrossRef]
  66. Davidson, M.A. ADHD in adults: A review of the literature. J. Atten. Disord. 2008, 11, 628–641. [Google Scholar] [CrossRef] [PubMed]
  67. Lenartowicz, A.; Loo, S.K. Use of EEG to diagnose ADHD. Curr. Psychiatry Rep. 2014, 16, 498. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  68. O’Mahony, N.; Florentino-Liano, B.; Carballo, J.J.; Baca-García, E.; Rodríguez, A.A. Objective diagnosis of ADHD using IMUs. Med. Eng. Phys. 2014, 36, 922–926. [Google Scholar] [CrossRef]
  69. Peng, X.; Lin, P.; Zhang, T.; Wang, J. Extreme learning machine-based classification of ADHD using brain structural MRI data. PLoS ONE 2013, 8, e79476. [Google Scholar] [CrossRef]
Figure 1. ROC curve plots for each machine learning methods; LDA: linear discriminant analysis; RF: random forest classification; KNN: K-nearest neighbors classification.
Figure 1. ROC curve plots for each machine learning methods; LDA: linear discriminant analysis; RF: random forest classification; KNN: K-nearest neighbors classification.
Diagnostics 11 00976 g001
Table 1. The differences between groups.
Table 1. The differences between groups.
DimensionScaleASRS (−)ASRS (+)tpCohen’s d
Validity
Indicators
VRIN_r4.19±2.375.27±2.69−8.51<0.001−0.45
TRIN_r10.37±1.9711.25±2.27−8.28<0.001−0.44
F_r3.57±3.357.59±4.64−21.99<0.001−1.17
Fp_r2.18±2.023.86±2.88−15.12<0.001−0.80
Fs2.15±1.944.14±2.47−18.90<0.001−1.00
FBS_r9.11±3.4812.01±4.07−15.50<0.001−0.82
L_r4.46±2.243.25±2.0910.24<0.0010.54
K_r7.24±2.674.77±2.2917.54<0.0010.93
Higher-
Order (H-O)
EID13.99±6.6020.87±7.22−19.55<0.001−1.04
THD2.55±2.704.86±3.95−15.61<0.001−0.83
BXD6.16±2.968.47±3.04−14.68<0.001−0.78
Restructured
Clinical (RC)
RCd7.38±5.1713.54±5.40−22.41<0.001−1.19
RC16.62±3.9310.17±4.83−16.75<0.001−0.89
RC25.53±3.116.57±3.20−6.32<0.001−0.34
RC34.76±2.586.69±2.81−13.99<0.001−0.74
RC44.03±2.606.21±3.03−15.60<0.001−0.83
RC61.46±1.832.99±2.78−15.07<0.001−0.80
RC77.78±4.3112.59±4.47−20.98<0.001−1.11
RC83.18±2.595.72±3.22−18.17<0.001−0.96
RC911.28±4.6114.71±4.01−14.19<0.001−0.75
Content, Clinical SubscaleMLS3.02±1.603.97±1.77−11.08<0.001−0.59
GIC0.64±1.041.21±1.36−9.99<0.001−0.53
HPC1.31±1.372.25±1.74−12.71<0.001−0.67
NUC2.99±1.694.02±1.74−11.56<0.001−0.61
COG3.06±2.135.68±2.16−23.18<0.001−1.23
SUI0.29±0.740.82±1.25−12.72<0.001−0.67
HLP1.00±1.071.82±1.28−14.23<0.001−0.76
SFD1.41±1.262.45±1.28−15.54<0.001−0.82
NFC4.01±2.165.73±1.98−15.13<0.001−0.80
STW2.76±1.724.20±1.76−15.77<0.001−0.84
AXY0.42±0.821.16±1.25−16.50<0.001−0.88
ANP2.37±1.643.78±1.75−16.07<0.001−0.85
BRF2.00±1.552.73±1.77−8.75<0.001−0.46
MSF3.97±2.314.03±2.32−0.450.66−0.02
JCP0.83±1.071.48±1.39−11.35<0.001−0.60
SUB0.70±0.951.16±1.32−8.79<0.001−0.47
AGG2.51±1.853.90±1.99−14.19<0.001−0.75
ACT2.36±1.743.61±1.73−13.57<0.001−0.72
FML2.07±1.853.73±2.22−16.72<0.001−0.89
IPP4.37±2.184.50±2.16−1.120.26−0.06
SAV4.16±2.584.54±2.67−2.730.01−0.15
SHY3.33±2.014.27±1.92−8.91<0.001−0.47
DSF0.78±1.121.37±1.46−9.61<0.001−0.51
AES2.94±1.693.12±1.76−2.040.04−0.11
MEC2.22±1.812.41±1.88−1.950.05−0.10
Personality Psychopathology Five (PSY-5)AGGR_r7.85±3.238.56±3.14−4.16<0.001−0.22
PSYC_r3.07±2.795.60±3.89−16.58<0.001−0.88
DISC_r5.73±2.547.16±2.67−10.61<0.001−0.56
NEGE_r7.88±3.8711.43±3.72−17.36<0.001−0.92
INTR_r8.06±3.828.26±3.74−1.010.31−0.05
ASRS (+): Adult ADHD symptoms group; the subjects whose total score of part 1 or part 2 exceeded 21; values are presented as mean ± SD; ASRS (−): control group. df: 5724.00.
Table 2. Accuracy in each predictive model.
Table 2. Accuracy in each predictive model.
Trees or Nearest NeighborsValidation AccuracyTest AccuracyOOB Accuracy
K-Nearest Neighbors Classification70.9270.931
Linear Discriminant Analysis 0.912
Random Forest Classification400.9440.9360.078
The random forest models are optimized with respect to the out-of-bag accuracy. The KNN model is optimized with respect to the validation set accuracy.
Table 3. Total average evaluation metrics in each predictive model.
Table 3. Total average evaluation metrics in each predictive model.
PrecisionRecallF1 ScoreAUC
K-Nearest Neighbors Classification0.9000.9310.9090.722
Linear Discriminant Analysis0.8990.9120.9050.806
Random Forest Classification0.9160.9360.9090.790
Area under curve (AUC) is calculated for every class against all other classes.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Kim, S.; Lee, H.-K.; Lee, K. Can the MMPI Predict Adult ADHD? An Approach Using Machine Learning Methods. Diagnostics 2021, 11, 976. https://doi.org/10.3390/diagnostics11060976

AMA Style

Kim S, Lee H-K, Lee K. Can the MMPI Predict Adult ADHD? An Approach Using Machine Learning Methods. Diagnostics. 2021; 11(6):976. https://doi.org/10.3390/diagnostics11060976

Chicago/Turabian Style

Kim, Sunhae, Hye-Kyung Lee, and Kounseok Lee. 2021. "Can the MMPI Predict Adult ADHD? An Approach Using Machine Learning Methods" Diagnostics 11, no. 6: 976. https://doi.org/10.3390/diagnostics11060976

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop