Next Article in Journal
Optimizing Soil Health and Sorghum Productivity through Crop Rotation with Quinoa
Previous Article in Journal
Unveiling Drought-Resilient Latin American Popcorn Lines through Agronomic and Physiological Evaluation
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Life
Volume 14
Issue 6
10.3390/life14060744
life-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Preliminary External Validation Results of the Artificial Intelligence-Based Headache Diagnostic Model: A Multicenter Prospective Observational Study

by
Mariko Okada
1,
Masahito Katsuki
2,
Tomokazu Shimazu
3,
Takao Takeshima
4,
Takashi Mitsufuji
1,
Yasuo Ito
1,
Katsumi Ohbayashi
5,
Noboru Imai
6,
Junichi Miyahara
4,
Yasuhiko Matsumori
7,
Yoshihiko Nakazato
1,
Kazuki Fujita
8,
Eri Hoshino
3 and
Toshimasa Yamamoto
1,*
1
Department of Neurology, Saitama Medical University, 38 Morohongo, Moroyama-machi, Iruma-gun, Saitama 350-0495, Japan
2
Physical Education and Health Center, Nagaoka University of Technology, Niigata 940-2137, Japan
3
Department of Neurology, Saitama Neuropsychiatric Institute, Saitama 338-8577, Japan
4
Headache Center and Department of Neurology, Tominaga Hospital, Osaka 556-0017, Japan
5
Ohbayashi Clinic, Tochigi 321-0933, Japan
6
Department of Neurology, Japanese Red Cross Shizuoka Hospital, Shizuoka 420-0853, Japan
7
Sendai Headache and Neurology Clinic, Sendai 982-0014, Japan
8
Department of Neurology, Jichi Medical University Saitama Medical Center, Saitama 330-8503, Japan
*
Author to whom correspondence should be addressed.
Life 2024, 14(6), 744; https://doi.org/10.3390/life14060744
Submission received: 7 May 2024 / Revised: 5 June 2024 / Accepted: 8 June 2024 / Published: 11 June 2024
(This article belongs to the Section Medical Research)
Versions Notes

Abstract

:
The misdiagnosis of headache disorders is a serious issue, and AI-based headache model diagnoses with external validation are scarce. We previously developed an artificial intelligence (AI)-based headache diagnosis model using a database of 4000 patients’ questionnaires in a headache-specializing clinic and herein performed external validation prospectively. The validation cohort of 59 headache patients was prospectively collected from August 2023 to February 2024 at our or collaborating multicenter institutions. The ground truth was specialists’ diagnoses based on the initial questionnaire and at least a one-month headache diary after the initial consultation. The diagnostic performance of the AI model was evaluated. The mean age was 42.55 ± 12.74 years, and 51/59 (86.67%) of the patients were female. No missing values were reported. Of the 59 patients, 56 (89.83%) had migraines or medication-overuse headaches, and 3 (5.08%) had tension-type headaches. No one had trigeminal autonomic cephalalgias or other headaches. The models’ overall accuracy and kappa for the ground truth were 94.92% and 0.65 (95%CI 0.21–1.00), respectively. The sensitivity, specificity, precision, and F values for migraines were 98.21%, 66.67%, 98.21%, and 98.21%, respectively. There was disagreement between the AI diagnosis and the ground truth by headache specialists in two patients. This is the first external validation of the AI headache diagnosis model. Further data collection and external validation are required to strengthen and improve its performance in real-world settings.

1. Introduction

Primary headaches represent a prevalent issue in public health [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16]. The International Classification of Headache Disorders, 3rd edition (ICHD-3) [17], indexes migraine [18,19,20], tension-type headache (TTH) [21,22,23], and trigeminal autonomic cephalalgias (TACs) [24] as typical examples of primary headaches, and they significantly impede patients’ everyday tasks. A correct diagnosis based on ICHD-3 criteria is essential for prescribing both preventative and acute treatment for headaches [25,26,27,28,29,30,31]. However, in 2023, only 982 headache specialists were practicing in Japan for more than 10 million people [18,19,20,32,33] experiencing migraines. Because of the low number of headache specialists, delays in headache diagnosis are likely, which can raise the risk of chronic migraines, treatment resistance, comorbidities, and medication-overuse headaches (MOHs) [32,34,35,36,37,38,39,40,41,42,43,44,45]. Furthermore, late diagnosis and treatment initiation for migraines sometimes result in migraine prognoses ranging from episodic to chronic, which are relatively difficult to treat [46,47,48,49]. In Japan, primary care physicians often handle the initial consultation for headache disorders, despite lacking specific training in this area. This situation places significant stress on primary care physicians and results in unmet medical needs. Similar challenges may also arise in other countries [50,51,52,53].
Artificial intelligence (AI)-based medical devices are gradually coming into the world. This is expected to reduce medical treatment costs and time [54,55,56,57]. AI-based solutions for headache diagnosis can address this issue because they can save time that would otherwise be taken up by medical interviews while increasing diagnostic performance [58,59,60]. If this AI becomes widely used, it could increase rational and efficient actions, such as patients who have not seen a doctor going to a hospital when they realize they have migraines, or non-specialists accurately diagnosing migraines in a short time and referring them to a specialist. However, only seven models [60,61,62,63,64,65,66] can diagnose primary headache subtypes and have been evaluated in both training and test datasets during development [67]. Furthermore, no models have performed external validation so far, and there are no reports examining whether AI performs the same capabilities in different domains as its training data.
Domain shifting is often a problem in AI development. AI that performs well on training data and target data used in the development environment will perform poorly in a different operating environment. This can occur when the probability distribution of target data in the operational environment differs from that in the development environment.
We previously developed an AI headache diagnosis model based on a headache-specialized outpatient dataset [65]. Because this AI was created using only data from headache outpatient clinics at one hospital, it is unknown whether it would perform similarly outside of that hospital. This means that the possibility of an inherent domain shift cannot be ruled out. In this study, we prospectively recruit headache patients and evaluate the AI’s performance compared to the headache specialists’ diagnoses as external validation. The objective is to prove that the diagnostic performance of AI is equivalent to that of a headache specialist.

2. Materials and Methods

2.1. AI Diagnosis Model

We utilized the AI headache diagnosis model that was previously reported and developed [65]. We will now provide a brief description of the model. It was constructed using data from 4000 consecutive patients aged 15 and over who visited the Tominaga Hospital Headache Center between March 2013 and December 2021. The patients’ questionnaire responses and diagnoses from their initial appointments were analyzed. Three headache specialists diagnosed all the patients. The 4000 patients were divided into training and test datasets randomly. The model was trained using 70% (2800) of the patients’ questionnaire data, which contained no missing information. To assess the model’s performance, it was tested on a separate dataset comprising responses from 1200 patients (30% of the total).
The headache questionnaire sheet had 17 variables. From the questionnaire sheet of each patient, we acquired 17 variables: age, sex, height, weight, age at headache onset, headache frequency, headache duration, site of headache, headache characteristics, headache severity, presence of aggravation by exercise, concomitant symptoms (including nausea, vomiting, photophobia, phonophobia, osmophobia, autonomic symptoms, paralysis), presence of aura, times when the headache is most likely to occur, inducement of headache, use of acute medication, and family history.
Headache diagnoses were classified into five categories as follows:
  • Class 1: Migraines and MOHs (ICHD-3 codes 1 and 8.2);
  • Class 2: TTHs (code 2);
  • Class 3: TACs (code 3);
  • Class 4: Other primary headaches (code 4);
  • Class 5: Other headaches (other codes; secondary).
The average accuracy, sensitivity (recall), specificity, precision, and F values (Definition A: macro-average of F value for each class; Definition B: harmonic mean of average recall and average precision) were 76.25%, 56.26%, 92.16%, 61.24%, 56.88%, and 73.58%, respectively. The micro-average area under the curve of the receiver operating characteristic curve (AUC of the ROC) was 0.95.

2.2. Validation Cohort and Procedure

To test the model’s efficacy as external validation, we prospectively collected 59 headache patients from August 2023 to February 2024. Patient recruitment was carried out in 8 clinics and hospitals, and all of the patients consulted headache specialists in each facility. We validated the AI’s performance using 59 of the new patients’ data, including questionnaires and diagnoses by the specialists.
The recruitment procedure was as follows: Patients aged 20 or above with recurrent primary headaches who had undergone consultations at our institution or collaborating institutions for at least three months and consented to participate were included. A one-month observation period followed for electronic headache diary assessment before formal registration, and headache specialists confirmed their diagnoses based on both the initial headache questionnaire sheets and headache diary. The exclusion criteria were (i) pregnant or lactating individuals, those potentially pregnant during the study, or those wishing to become pregnant during the trial; (ii) patients with hemiplegic migraines; (iii) patients with severe cardiac, hepatic, renal, hematologic, or pulmonary disorders, or any condition deemed life-threatening by the investigator, especially if the patients were taking a medication that has a side effect of headache or has a preventive effect of headaches, which may modify the nature of the headache characteristics; (iv) patients with psychiatric disorders, including depression, following the Diagnostic and Statistical Manual of Mental Disorders (DSM)-5 diagnostic criteria; and (v) patients consuming more than 400 mg of caffeine per day, such as drinking 4 to 5 cups of coffee or regularly consuming caffeine-containing energy drinks.
The ground truth referred to the diagnoses provided by the headache specialists, and the agreement rates between headache specialists and AI diagnoses were assessed. Performance metrics such as accuracy, sensitivity (recall), specificity, precision, F-values (using definitions A and B), and kappa were evaluated to gauge performance.

2.3. Statistical Analysis

Variables with a normal distribution are presented as means (standard deviation), whereas those with a non-normal distribution are presented as medians (interquartile range). Unweighted Cohen’s kappa statistics were employed to gauge the agreement in diagnoses between the model and the raters. Unweighted kappa was chosen due to the nominal nature of the outcomes. The interpretation of kappa values followed Cohen’s guidelines, classifying agreement as “no agreement” for kappa ≤ 0, “none-to-slight agreement” for kappa = 0.01–0.20, “fair agreement” for kappa = 0.21–0.40, “moderate agreement” for kappa = 0.41–0.60, “substantial agreement” for kappa = 0.61–0.80, and “almost perfect agreement” for kappa = 0.81–1.00. The performance metrics of both the AI model and the raters were evaluated against specialist diagnoses, considered the reference standard (ground truth). Statistical analyses were conducted using SPSS 28.0.0 (IBM Corp., Armonk, NY, USA), Python 3.9.0, PyCaret 3.0.0, and Matplotlib 3.5.1.2.4.

2.4. Ethics

The Central Ethics Committee of Saitama Medical University approved this study (approval number: 2021-001). The requirement for written informed consent was obtained from all the patients. Opt-out consent documents were presented at the Tominaga Hospital and its website (https://www.tominaga.or.jp/about/registration_and_trial/ (accessed on 15 April 2024)) for patients who did not wish to participate. All methods in this study were performed following the relevant guidelines and regulations of the Declaration of Helsinki. This study was performed under the Strengthening the Reporting of Observational Studies in Epidemiology guidelines [68] and the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis guidelines [69]. We also complied with the guidelines regarding medical AI development [67].

3. Results

3.1. Validation Cohort’s Characteristics

The diagnoses of the training, test [65], and external validation data are summarized in Table 1. The mean patient age was 42.55 (standard deviation 12.74) years, and 51/59 (86.67%) of the patients were female. No missing values were reported. Of the 59 patients, 56 (94.92%) had migraines or MOHs and 3 (5.08%) had TTHs. No one had TACs or other headaches (secondary headaches).

3.2. External Validation Performance

As the specialists’ diagnoses were ground truths, the AI’s performance was evaluated using the same patients’ questionnaire sheets. The overall accuracy and kappa for the ground truth were 94.92% and 0.65 (95%CI 0.21–1.00), respectively. The other indices, such as sensitivity, specificity, precision, and F values for migraine, were 98.21%, 66.67%, 98.21%, and 98.21%, respectively (Table 2).
There was disagreement between the AI diagnosis and the ground truth by the headache specialists in two patients. Among those, the AI misdiagnosed one TTH case as a migraine case and one migraine case as TACs.
The former case was a 23-year-old female with intermittent headaches for the past 6 years. The headache persisted throughout the day and every day. The center of her head ached. Sometimes, the headache pulsated, and sometimes it was persistent and tight. The pain intensity was moderate, and the headache was not aggravated by physical activity. It was accompanied by nausea, vomiting, and light–sound sensitivity. There was no aura. Stress increased the headaches’ intensity, and analgesics were used about three times a month. The AI raised migraine as a diagnosis because the headache was moderate and slightly interfering with her life, but the doctor diagnosed it as TTHs because there was no vomiting and no significant interference with her life. The patient’s medical questionnaire indicated that she had nausea and vomiting, but in fact the headache was not related to nausea and vomiting, and other illnesses such as chronic gastritis were suspected.
The latter case was a 32-year-old male with intermittent headaches for the past 10 years. Their frequency was several times a year, but the presence or absence of clustering could not be determined from the questionnaire alone. The headache attack lasted about a day. The headache was painful around the left side of the eye, the back of the head, and the neck. There was gouging and cracking pain in the eyes, and the headache was pulsatile. There was nausea and sound sensitivity, as well as redness of the eyes, running tears, and nasal discharge. There was stiffness in the shoulders and flashes of light as an aura. The AI diagnosed TACs based on factors such as nasal discharge and redness of the eyes on the medical questionnaire. However, the specialist diagnosed migraines in this person based on the presence of a visual aura, exacerbation with body movement, and the fact that triptans were effective for their headaches.

4. Discussion

We externally validated the previously developed AI model’s performance using prospectively collected diagnoses of 59 patients with headache specialists. The accuracy and kappa for the ground truth were 94.92% and 0.65 (95%CI 0.21–1.00), respectively, suggesting that this AI has the same diagnostic capabilities as headache specialists. To our knowledge, this is the first report on the external validation of an AI headache diagnosis model.
The number of headache specialists in Japan is low, at 982 doctors for a population of 130 million in 2023, so non-headache specialists often treat headache patients. It is also true that even specialists sometimes have difficulty diagnosing headaches; the diagnosis rate of repetitive migraines that interfere with daily life is 86.7% even among headache specialists, but 24.6% for intractable chronic migraines [70]. Furthermore, among non-specialists, the accuracy of headache diagnosis is as low as 46% [65]. Therefore, there is a need to develop a system to efficiently make correct headache diagnoses. In the realm of headache medicine, AI has the potential to improve diagnostic accuracy significantly. Our previous research indicates that an AI algorithm can boost diagnostic precision even among non-specialists in headache disorders [65]. With the widespread occurrence of migraines and other types of headaches and the scarcity of specialized headache doctors, such technological aids could enable patients to obtain accurate diagnoses, alleviating the impact of severe headaches and potentially lessening the need for referrals to neurologists.
In Japan, the percentage of headache patients visiting a medical institution is as low as 30% [19], and many patients purchase OTC painkillers depending on their own headache self-diagnosis and the possibility of developing MOHs. If migraine patients who especially need specialized treatment can be identified among these headache patients, they can visit a hospital and receive appropriate treatment. For example, a patient could be diagnosed by AI at home using a smartphone and realize that he or she has migraine headaches, which could lead to medical attention. It is also possible that a non-specialist physician, with the help of AI, could properly diagnose migraines and refer the patient to a specialist. Therefore, if AI diagnosis can be proven to be as accurate as that of headache specialists, it will improve the accuracy of migraine diagnosis by non-headache specialists and also increase the rate of outpatient visits by headache patients.
Our study shows that the diagnostic accuracy of the AI’s diagnosis is very high, with a kappa of 0.9. If a headache specialist examines a patient using this AI diagnosis, he/she does not need to check the entire headache diary but can accurately diagnose the remaining 10% that the AI diagnosis determines to be challenging to determine, thereby accurately ascertaining the number of migraine headaches and accurately determining the effectiveness of treatment. This is expected to reduce the burden on physicians, shorten patients’ treatment times, and improve the quality of medical care. High sensitivity and specificity are useful for screening and educating many migraineurs, but they may miss certain headaches. It is possible to adjust the output of the AI to achieve the desired level of sensitivity and specificity. Other screening tools, such as SNOOP10 [71], could also be used to screen for dangerous secondary headaches. How to use this AI in combination with other screeners needs to be explored in the future.
Our model’s sensitivity was as high as 98.21%, but its specificity was not as high at 66.67%. High sensitivity is suitable for screening migraines. The specialist’s diagnosis may not always be correct, and as noted in the ICHD-3, all applicable headaches should be considered and diagnosed. In particular, migraines coexisting with TTHs may be overlooked, so our model may be useful in terms of picking up as many migraine patients as possible.
Many AI-based headache diagnostic models have been reported (Table 3). However, not one of these has been externally validated. Therefore, overfitting cannot be ruled out. Compared to these, our study is significant because we are the first to examine the usefulness of our own AI model in prospective cases collected at other institutions.
In this report, we discuss AI models for diagnosis. However, other AI-related research in the headache area is also gaining momentum. AI models for predicting treatment efficacy are also being developed. As an illustration, extensive data gathered from a smartphone app designed for electronic headache diaries were used to assess the effectiveness of acute migraine medications as reported by patients. AI promises to enhance outcome predictions, such as forecasting migraine episodes, anticipating responses to treatments, and identifying which migraine patients may develop additional health conditions [72]. AI has also been used to predict the therapeutic effect of CGRP-related drugs [73]. An AI model for predicting outcomes of headache surgery was also developed [74]. An AI model that extracts the number of monthly headache days from electronic medical records using a large language model has also been reported [75]. In this way, AI is increasingly being developed around the world, not only for diagnosis but also for predicting treatment effects and supporting medical treatment. AI is expected to become a good partner in headache treatment in the future.
Screening tools, not AI, have been developed so far. These are based on statistical methods, but it is unknown which is better than decision tree-based AI. As famous screeners, the Three-Question Headache Screen [76], ID Migraine [77], and the Four-Item Migraine Screener [77] have been reported, and their sensitivity and specificity range from 0.75 to 0.90. It remains to be seen whether AI or these tools are better, as their performance is highly dependent on the target population and other factors. We plan to examine the usefulness of these screening tools in our next research project when we have more cases.
Reports on domain shifting are scarce. This is the flip side of the fact that AI is being actively developed, but its usefulness has not been tested against other populations. This suggests that most medical AI models, although they can be developed on a trial basis, are unlikely to make it onto the market as a product. Only 17 medical AI models have been approved in Japan as of 2021 (https://www.pmda.go.jp/files/000244149.pdf (accessed on 20 April 2024)). Many processes are needed to put medical AI on the market as a product. In the future, it may be necessary to develop a system that allows AI to be created on a trial basis to demonstrate how useful it can be and to implement it in society as soon as possible.
This study has some limitations. First is the lack of further generalizability. The results were derived from data obtained from some hospitals or clinics with headache specialists. It is still unclear what the diagnostic performance is like outside of Japan or in an environment where non-specialists examine patients. In other words, the possibility that a domain shift is still inherent cannot be ruled out. Second, due to the exclusive reliance on questionnaire sheets in developing this model, information regarding neurological symptoms, vital signs, or other medical histories was not obtained. High specificity for secondary headaches is crucial in a diagnostic tool to prevent the misdiagnosis of potentially life-threatening conditions [71,78]. Hence, there is a need to develop an AI-based diagnostic tool capable of gathering data on patients’ histories and neurological findings, as well as radiological and laboratory test results, to effectively rule out secondary headaches. Third, the validation cohort included many migraine patients and did not include patients with TACs or other conditions. Therefore, kappa’s 95% CIs are wide-ranging. This suggests that although a prospective patient population was obtained, an even larger validation with no bias in the dataset is needed. We will prospectively collect more cases to include more types of headaches and perform the analysis again in the future. Although the model created in our previous study was used [65], it is necessary to further tune the model based on the data obtained prospectively in this way and further validate how the sensitivity and specificity of the model should be set for clinical use. Finally, AI should be used as a screening tool and for diagnostic assistance. An experienced headache specialist is also indispensable, given the diagnosis of secondary headaches and comorbid headaches of multiple types.

5. Conclusions

We externally validated the AI model’s performance using prospectively collected data from 59 patients. The diagnostic performance was strong for migraines and MOHs; the sensitivity, specificity, precision, and F-values for migraines were 98.21%, 66.67%, 98.21%, and 98.21%, respectively. Overall, the accuracy and kappa for the ground truth were 94.92% and 0.65 (95%CI 0.21–1.00), respectively, suggesting that this AI model has the same diagnostic capabilities as headache specialists. This AI headache diagnosis model may resolve the issue of undiagnosed and undertreated headaches and help non-headache specialists.

Author Contributions

Conceptualization, T.Y. and T.S.; data curation, M.O. and M.K.; funding acquisition, M.O.; formal analysis, M.K.; resources, T.S., T.T., T.M., Y.I., K.O., N.I., J.M., and Y.M.; investigation, M.O., T.S., T.T., T.M., Y.I., K.O., N.I., J.M., Y.M., K.F., and E.H.; writing—original draft preparation, M.O. and M.K.; writing—review and editing, T.Y., T.S., T.T., and Y.N.; supervision, T.Y. and T.T.; project administration: T.Y. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by a “New researcher promoting research fee” with grant number 22-E-1-03, supported by Saitama Medical University.

Institutional Review Board Statement

The Central Ethics Committee of Saitama Medical University approved this study (approval number: 2021-001). The requirement for written informed consent was obtained from all the patients. Opt-out consent documents were presented at the Tominaga Hospital and on its website (https://www.tominaga.or.jp/about/registration_and_trial/ (accessed on 15 April 2024)) for patients who did not wish to participate. All methods were performed following the relevant guidelines and regulations of the Declaration of Helsinki. This study was performed under the Strengthening the Reporting of Observational Studies in Epidemiology guidelines and the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis guidelines.

Informed Consent Statement

Written informed consent has been obtained from the patients to publish this paper.

Data Availability Statement

The datasets used and/or analyzed during the current study are available from the corresponding author upon reasonable request.

Acknowledgments

We thank Makiko Hirayama, Yuichi Maruki, Hisaka Igarashi, Shoji Kikui, Daisuke Danno, Muneto Tastumoto, and Yuji Kato for their support.

Conflicts of Interest

The authors report no conflicts of interest concerning the materials or methods used in this study or the findings presented in this paper. No sources of financial or material support were given. This article and contents of this study were not published nor presented previously. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.

References

  1. Matsumori, Y.; Ueda, K.; Komori, M.; Zagar, A.J.; Kim, Y.; Jaffe, D.H.; Takeshima, T.; Hirata, K. Burden of Migraine in Japan: Results of the ObserVational Survey of the Epidemiology, TReatment, and Care Of MigrainE (OVERCOME [Japan]) Study. Neurol. Ther. 2022, 11, 205–222. [Google Scholar] [CrossRef]
  2. Katsuki, M.; Matsumori, Y.; Kawahara, J.; Yamagishi, C.; Koh, A.; Kawamura, S.; Kashiwagi, K.; Kito, T.; Oguri, M.; Mizuno, S.; et al. Headache Education by Leaflet Distribution during COVID-19 Vaccination and School-based On-demand E-learning: Itoigawa Geopark Headache Awareness Campaign. Headache J. Head Face Pain 2023, 63, 429–440. [Google Scholar] [CrossRef] [PubMed]
  3. Takeshima, T.; Ueda, K.; Komori, M.; Zagar, A.J.; Kim, Y.; Jaffe, D.H.; Matsumori, Y.; Hirata, K. Potential Unmet Needs in Acute Treatment of Migraine in Japan: Results of the OVERCOME (Japan) Study. Adv. Ther. 2022, 39, 5176–5190. [Google Scholar] [CrossRef]
  4. Lebedeva, E.R.; Olesen, J.; Osipova, V.V.; Volkova, L.I.; Tabeeva, G.R.; Steiner, T.J. The Yekaterinburg Headache Initiative: An Interventional Project, within the Global Campaign against Headache, to Reduce the Burden of Headache in Russia. J. Headache Pain 2013, 14, 101. [Google Scholar] [CrossRef] [PubMed]
  5. Bøe, M.G.; Thortveit, E.; Vatne, A.; Mygland, Å. Chronic Headache with Medication Overuse: Long-Term Prognosis after Withdrawal Therapy. Cephalalgia 2017, 37, 1215–1221. [Google Scholar] [CrossRef]
  6. Zidverc-Trajkovic, J.J.; Pekmezovic, T.; Jovanovic, Z.; Pavlovic, A.; Mijajlovic, M.; Radojicic, A.; Sternic, N. Long-Term Predictors of Remission in Patients Treated for Medication-Overuse Headache at a Specialized Headache Center: A Prospective Cohort Study. Cephalalgia 2018, 38, 265–273. [Google Scholar] [CrossRef]
  7. Diener, H.-C.; Dodick, D.; Evers, S.; Holle, D.; Jensen, R.H.; Lipton, R.B.; Porreca, F.; Silberstein, S.; Schwedt, T. Pathophysiology, Prevention, and Treatment of Medication Overuse Headache. Lancet Neurol. 2019, 18, 891–902. [Google Scholar] [CrossRef]
  8. Schembri, E.; Barrow, M.; McKenzie, C.; Dawson, A. The Evolving Classifications and Epidemiological Challenges Surrounding Chronic Migraine and Medication Overuse Headache: A Review. Korean J. Pain 2022, 35, 4–13. [Google Scholar] [CrossRef]
  9. Lipton, R.B.; Buse, D.C.; Serrano, D.; Holland, S.; Reed, M.L. Examination of Unmet Treatment Needs among Persons with Episodic Migraine: Results of the American Migraine Prevalence and Prevention (AMPP) Study. Headache 2013, 53, 1300–1311. [Google Scholar] [CrossRef] [PubMed]
  10. Hirata, K.; Ueda, K.; Komori, M.; Zagar, A.J.; Selzler, K.J.; Nelson, A.M.; Han, Y.; Jaffe, D.H.; Matsumori, Y.; Takeshima, T. Comprehensive Population-Based Survey of Migraine in Japan: Results of the ObserVational Survey of the Epidemiology, TReatment, and Care Of MigrainE (OVERCOME [Japan]) Study. Curr. Med. Res. Opin. 2021, 37, 1945–1955. [Google Scholar] [CrossRef]
  11. Ojo, A.; Zhang, S.; Bleibdrey, N.; Zimskind, D.; Keshvani, N.; Chalmers, R. Persistence and Switching Patterns of Migraine Prophylactic Medications in Canada: A Retrospective Claims Analysis Comparing Adherence and Evaluating the Economic Burden of Illness. J. Pharm. Pharm. Sci. 2023, 25, 402–417. [Google Scholar] [CrossRef] [PubMed]
  12. Genc, D.; Vaičienė-Magistris, N.; Zaborskis, A.; Şaşmaz, T.; Tunç, A.Y.; Uluduz, D.; Wöber, C.; Wöber-Bingöl, Ç.; Steiner, T.J. The Burden Attributable to Headache Disorders in Children and Adolescents in Lithuania: Estimates from a National Schools-Based Study. J. Headache Pain 2021, 22, 24. [Google Scholar] [CrossRef]
  13. Lipton, R.B.; Buse, D.C.; Sandoe, C.H.; Ford, J.H.; Hand, A.L.; Jedynak, J.P.; Port, M.D.; Detke, H.C. Changes in Migraine Interictal Burden Following Treatment with Galcanezumab: Results from a Phase III Randomized, Placebo-Controlled Study. Headache 2023, 63, 683–691. [Google Scholar] [CrossRef] [PubMed]
  14. Korolainen, M.A.; Kurki, S.; Lassenius, M.I.; Toppila, I.; Costa-Scharplatz, M.; Purmonen, T.; Nissilä, M. Burden of Migraine in Finland: Health Care Resource Use, Sick-Leaves and Comorbidities in Occupational Health Care. J. Headache Pain 2019, 20, 13. [Google Scholar] [CrossRef] [PubMed]
  15. Kikui, S.; Chen, Y.; Todaka, H.; Asao, K.; Adachi, K.; Takeshima, T. Burden of Migraine among Japanese Patients: A Cross-Sectional National Health and Wellness Survey. J. Headache Pain 2020, 21, 110. [Google Scholar] [CrossRef] [PubMed]
  16. Takeshima, T.; Wan, Q.; Zhang, Y.; Komori, M.; Stretton, S.; Rajan, N.; Treuer, T.; Ueda, K. Prevalence, Burden, and Clinical Management of Migraine in China, Japan, and South Korea: A Comprehensive Review of the Literature. J. Headache Pain 2019, 20, 111. [Google Scholar] [CrossRef] [PubMed]
  17. Headache Classification Committee of the International Headache Society (IHS) The International Classification of Headache Disorders, 3rd Edition. Cephalalgia 2018, 38, 1–211. [CrossRef]
  18. Katsuki, M.; Matsumori, Y.; Kawahara, J.; Yamagishi, C.; Koh, A.; Kawamura, S.; Kashiwagi, K.; Kito, T.; Oguri, M.; Mizuno, S.; et al. School-Based Online Survey on Chronic Headache, Migraine, and Medication-Overuse Headache Prevalence among Children and Adolescents in Japanese One City—Itoigawa Benizuwaigani Study. Clin. Neurol. Neurosurg. 2023, 226, 107610. [Google Scholar] [CrossRef]
  19. Sakai, F.; Igarashi, H. Prevalence of Migraine in Japan: A Nationwide Survey. Cephalalgia 1997, 17, 15–22. [Google Scholar] [CrossRef]
  20. Takeshima, T.; Ishizaki, K.; Fukuhara, Y.; Ijiri, T.; Kusumi, M.; Wakutani, Y.; Mori, M.; Kawashima, M.; Kowa, H.; Adachi, Y.; et al. Population-Based Door-to-Door Survey of Migraine in Japan: The Daisen Study. Headache J. Head Face Pain 2004, 44, 8–19. [Google Scholar] [CrossRef]
  21. Suzuki, N.; Ishikawa, Y.; Gomi, S.; Ito, N.; Watanabe, S.; Yokoyama, M.; Funatsu, K.; Miyake, H.; Yokoyama, H.; Shimizu, T.; et al. Prevalence and Characteristics of Headaches in a Socially Active Population Working in the Tokyo Metropolitan Area-Surveillance by an Industrial Health Consortium. Intern. Med. 2014, 53, 683–689. [Google Scholar] [CrossRef] [PubMed]
  22. Fumal, A.; Schoenen, J. Tension-Type Headache: Current Research and Clinical Management. Lancet Neurol. 2008, 7, 70–83. [Google Scholar] [CrossRef] [PubMed]
  23. Crystal, S.C.; Grosberg, B.M. Tension-Type Headache in the Elderly. Curr. Pain. Headache Rep. 2009, 13, 474–478. [Google Scholar] [CrossRef] [PubMed]
  24. Imai, N.; Yagi, N.; Kuroda, R.; Konishi, T.; Serizawa, M.; Kobari, M. Clinical Profile of Cluster Headaches in Japan: Low Prevalence of Chronic Cluster Headache, and Uncoupling of Sense and Behaviour of Restlessness. Cephalalgia 2011, 31, 628–633. [Google Scholar] [CrossRef] [PubMed]
  25. Headache Clinical Practice Guideline Development Committee, C. [Clinical Practice Guideline for Headache Disorders 2021] (Japanese); Japanese Society of Neurology, Japanese Headache Society, Japanese Society of Neurological Therapeutics, Eds.; Igaku-Shoin: Tokyo, Japan, 2021. [Google Scholar]
  26. Ailani, J.; Lipton, R.B.; Goadsby, P.J.; Guo, H.; Miceli, R.; Severt, L.; Finnegan, M.; Trugman, J.M.; ADVANCE Study Group Atogepant for the Preventive Treatment of Migraine. Atogepant for the Preventive Treatment of Migraine. N. Engl. J. Med. 2021, 385, 695–706. [Google Scholar] [CrossRef]
  27. Hirata, K.; Sakai, F.; Takeshima, T.; Imai, N.; Matsumori, Y.; Yoshida, R.; Numachi, Y.; Peng, C.; Mikol, D.D.; Cheng, S. Efficacy and Safety of Erenumab in Japanese Migraine Patients with Prior Preventive Treatment Failure or Concomitant Preventive Treatment: Subgroup Analyses of a Phase 3, Randomized Trial. J. Headache Pain 2021, 22, 110. [Google Scholar] [CrossRef]
  28. Takizawa, T.; Kitano, T.; Iijima, M.; Togo, K.; Yonemoto, N. Treatment Patterns and Characteristics of Patients with Migraine: Results from a Retrospective Database Study in Japan. J. Headache Pain 2024, 25, 19. [Google Scholar] [CrossRef] [PubMed]
  29. Chanchlani, R.; Agrawal, A.; Janjua, D.; Hafsa, S.N. The Efficacy of Different Triptans for the Treatment of Acute Headache in Pediatric Migraine: A Systematic Review. Indian Pediatr. 2023, 60, 663–671. [Google Scholar] [CrossRef] [PubMed]
  30. Thomsen, R.W.; Szépligeti, S.K.; Xue, F.; Pedersen, L.; Sørensen, H.T.; Ehrenstein, V. Patterns of Initial Migraine Treatment in Denmark: A Population-Based Study. Pharmacoepidemiol. Drug Saf. 2019, 28, 322–329. [Google Scholar] [CrossRef]
  31. Tatsumoto, M.; Suzuki, E.; Nagata, M.; Suzuki, K.; Hirata, K. Prophylactic Treatment for Patients with Migraine Using Blue Cut for Night Glass. Intern. Med. 2022; in press. [Google Scholar] [CrossRef]
  32. Katsuki, M.; Kawahara, J.; Matsumori, Y.; Yamagishi, C.; Koh, A.; Kawamura, S.; Kashiwagi, K.; Kito, T.; Entani, A.; Yamamoto, T.; et al. Questionnaire-Based Survey during COVID-19 Vaccination on the Prevalence of Elderly’s Migraine, Chronic Daily Headache, and Medication-Overuse Headache in One Japanese City—Itoigawa Hisui Study. J. Clin. Med. 2022, 11, 4707. [Google Scholar] [CrossRef] [PubMed]
  33. Katsuki, M.; Yamagishi, C.; Matsumori, Y.; Koh, A.; Kawamura, S.; Kashiwagi, K.; Kito, T.; Entani, A.; Yamamoto, T.; Ikeda, T.; et al. Questionnaire-Based Survey on the Prevalence of Medication-Overuse Headache in Japanese One City-Itoigawa Study. Neurol. Sci. 2022, 43, 3811–3822. [Google Scholar] [CrossRef]
  34. Buse, D.C.; Armand, C.E.; Charleston, L.; Reed, M.L.; Fanning, K.M.; Adams, A.M.; Lipton, R.B. Barriers to Care in Episodic and Chronic Migraine: Results from the Chronic Migraine Epidemiology and Outcomes Study. Headache 2021, 61, 628–641. [Google Scholar] [CrossRef] [PubMed]
  35. Katsuki, M.; Kashiwagi, K.; Kawamura, S.; Koh, A. The Efficacy of Japanese Herbal Kampo Medicine as an Acute and Prophylactic Medication to Treat Chronic Daily Headache and Medication Overuse Headache: -Single Arm Retrospective Study. Cureus 2022, 14, e25419. [Google Scholar] [CrossRef]
  36. Carlsen, L.N.; Westergaard, M.L.; Bisgaard, M.; Schytz, J.B.; Jensen, R.H. National Awareness Campaign to Prevent Medication-Overuse Headache in Denmark. Cephalalgia 2018, 38, 1316–1325. [Google Scholar] [CrossRef]
  37. Lai, J.T.F.; Dereix, J.D.C.; Ganepola, R.P.; Nightingale, P.G.; Markey, K.A.; Aveyard, P.N.; Sinclair, A.J. Should We Educate about the Risks of Medication Overuse Headache? J. Headache Pain 2014, 15, 10. [Google Scholar] [CrossRef]
  38. Tepper, S.J. Chronic Migraine and Medication Overuse Headache. Headache 2014, 54, 1249–1250. [Google Scholar] [CrossRef] [PubMed]
  39. Alstadhaug, K.B.; Ofte, H.K.; Kristoffersen, E.S. Preventing and Treating Medication Overuse Headache. Pain Rep. 2017, 2, e612. [Google Scholar] [CrossRef]
  40. Westergaard, M.L.; Lau, C.J.; Allesøe, K.; Gjendal, S.T.; Jensen, R.H. Monitoring Chronic Headache and Medication-Overuse Headache Prevalence in Denmark. Cephalalgia 2020, 40, 6–18. [Google Scholar] [CrossRef]
  41. Da Silva, A.N.; Lake, A.E. Clinical Aspects of Medication Overuse Headaches. Headache J. Head Face Pain 2014, 54, 211–217. [Google Scholar] [CrossRef]
  42. Jonsson, P.; Hedenrud, T.; Linde, M. Epidemiology of Medication Overuse Headache in the General Swedish Population. Cephalalgia 2011, 31, 1015–1022. [Google Scholar] [CrossRef] [PubMed]
  43. Viana, M.; De Icco, R.; Allena, M.; Sances, G.; Højland, J.R.; Katsarava, Z.; Lainez, M.J.A.; Fadic, R.; Goicochea, M.T.; Nappi, G.; et al. Clinical Subtypes of Medication Overuse Headache—Findings from a Large Cohort. Headache J. Head Face Pain 2019, 59, 1481–1491. [Google Scholar] [CrossRef]
  44. Thorlund, K.; Sun-Edelstein, C.; Druyts, E.; Kanters, S.; Ebrahim, S.; Bhambri, R.; Ramos, E.; Mills, E.J.; Lanteri-Minet, M.; Tepper, S. Risk of Medication Overuse Headache across Classes of Treatments for Acute Migraine. J. Headache Pain 2016, 17, 107. [Google Scholar] [CrossRef] [PubMed]
  45. Westergaard, M.L.; Glümer, C.; Hansen, E.H.; Jensen, R.H. Prevalence of Chronic Headache with and without Medication Overuse: Associations with Socioeconomic Position and Physical and Mental Health Status. Pain 2014, 155, 2005–2013. [Google Scholar] [CrossRef]
  46. Mathew, N.T. Transformed Migraine, Analgesic Rebound, and Other Chronic Daily Headaches. Neurol. Clin. 1997, 15, 167–186. [Google Scholar] [CrossRef] [PubMed]
  47. Silberstein, S.; Lipton, R. Chronic Daily Headache, Including Transformed Migraine, Chronic Tension-Type Headache and Medication Overuse. In Wolff’s Headache and Other Head Pain; Silberstein, S., Lipton, R., Dalessio, D., Eds.; Oxford University Press: New York, NY, USA, 2001; pp. 247–282. [Google Scholar]
  48. May, A.; Schulte, L.H. Chronic Migraine: Risk Factors, Mechanisms and Treatment. Nat. Rev. Neurol. 2016, 12, 455–464. [Google Scholar] [CrossRef] [PubMed]
  49. Schulte, L.H.; Allers, A.; May, A. Hypothalamus as a Mediator of Chronic Migraine: Evidence from High-Resolution FMRI. Neurology 2017, 88, 2011–2016. [Google Scholar] [CrossRef] [PubMed]
  50. Lipton, R.B.; Nicholson, R.A.; Reed, M.L.; Araujo, A.B.; Jaffe, D.H.; Faries, D.E.; Buse, D.C.; Shapiro, R.E.; Ashina, S.; Cambron-Mellott, M.J.; et al. Diagnosis, Consultation, Treatment, and Impact of Migraine in the US: Results of the OVERCOME (US) Study. Headache J. Head Face Pain 2022, 62, 122–140. [Google Scholar] [CrossRef] [PubMed]
  51. Katsarava, Z.; Mania, M.; Lampl, C.; Herberhold, J.; Steiner, T.J. Poor Medical Care for People with Migraine in Europe—Evidence from the Eurolight Study. J. Headache Pain 2018, 19, 10. [Google Scholar] [CrossRef] [PubMed]
  52. Roxas, A.; Quiles, L.E.; Wang, S.-J. Delivery of Care for Migraine in the Asian Oceanian Region: A Cross-Sectional Study. Cephalalgia 2021, 41, 1348–1358. [Google Scholar] [CrossRef]
  53. Woldeamanuel, Y.W. Headache in Resource-Limited Settings. Curr. Pain. Headache Rep. 2017, 21, 51. [Google Scholar] [CrossRef] [PubMed]
  54. Okiyama, S.; Fukuda, M.; Sode, M.; Takahashi, W.; Ikeda, M.; Kato, H.; Tsugawa, Y.; Iwagami, M. Examining the Use of an Artificial Intelligence Model to Diagnose Influenza: Development and Validation Study. J. Med. Internet Res. 2022, 24, e38751. [Google Scholar] [CrossRef] [PubMed]
  55. Hisaki, F.; Aga, M.; Tomitani, N.; Okawara, Y.; Harada, N.; Kario, K. Daily Self-Reported Behavioural Efficacy Records on Hypertension Digital Therapeutics as Digital Metrics Associated with the Reduction in Morning Home Blood Pressure: Post-Hoc Analysis of HERB-DH1 Trial. Hypertens. Res. 2024, 47, 120–127. [Google Scholar] [CrossRef] [PubMed]
  56. Kario, K.; Harada, N.; Okura, A. The First Software as Medical Device of Evidence-Based Hypertension Digital Therapeutics for Clinical Practice. Hypertens. Res. 2022, 45, 1899–1905. [Google Scholar] [CrossRef] [PubMed]
  57. Noda, Y.; So, R.; Sonoda, M.; Tabuchi, T.; Nomura, A. The Usefulness of a Smartphone App-Based Smoking Cessation Program for Conventional Cigarette Users, Heated Tobacco Product Users, and Dual Users: Retrospective Study. J. Med. Internet Res. 2023, 25, e42776. [Google Scholar] [CrossRef] [PubMed]
  58. Katsuki, M.; Narita, N.; Matsumori, Y.; Ishida, N.; Watanabe, O.; Cai, S.; Tominaga, T. Preliminary Development of a Deep Learning-Based Automated Primary Headache Diagnosis Model Using Japanese Natural Language Processing of Medical Questionnaire. Surg. Neurol. Int. 2020, 11, 475. [Google Scholar] [CrossRef] [PubMed]
  59. Woldeamanuel, Y.W.; Cowan, R.P. Computerized Migraine Diagnostic Tools: A Systematic Review. Ther. Adv. Chronic Dis. 2022, 13, 20406223211065236. [Google Scholar] [CrossRef]
  60. Cowan, R.P.; Rapoport, A.M.; Blythe, J.; Rothrock, J.; Knievel, K.; Peretz, A.M.; Ekpo, E.; Sanjanwala, B.M.; Woldeamanuel, Y.W. Diagnostic Accuracy of an Artificial Intelligence Online Engine in Migraine: A Multi-Center Study. Headache, 2022; online ahead. [Google Scholar] [CrossRef] [PubMed]
  61. Walters, A.B.; Smitherman, T.A. Development and Validation of a Four-Item Migraine Screening Algorithm among a Nonclinical Sample: The Migraine-4. Headache 2016, 56, 86–94. [Google Scholar] [CrossRef]
  62. Yin, Z.; Dong, Z.; Lu, X.; Yu, S.; Chen, X.; Duan, H. A Clinical Decision Support System for the Diagnosis of Probable Migraine and Probable Tension-Type Headache Based on Case-Based Reasoning. J. Headache Pain 2015, 16, 29. [Google Scholar] [CrossRef]
  63. Vandewiele, G.; De Backere, F.; Lannoye, K.; Vanden Berghe, M.; Janssens, O.; Van Hoecke, S.; Keereman, V.; Paemeleire, K.; Ongenae, F.; De Turck, F. A Decision Support System to Follow up and Diagnose Primary Headache Patients Using Semantically Enriched Data. BMC Med. Inform. Decis. Mak. 2018, 18, 98. [Google Scholar] [CrossRef] [PubMed]
  64. Kwon, J.; Lee, H.; Cho, S.; Chung, C.S.; Lee, M.J.; Park, H. Machine Learning-Based Automated Classification of Headache Disorders Using Patient-Reported Questionnaires. Sci. Rep. 2020, 10, 14062. [Google Scholar] [CrossRef]
  65. Katsuki, M.; Shimazu, T.; Kikui, S.; Danno, D.; Miyahara, J.; Takeshima, R.; Takeshima, E.; Shimazu, Y.; Nakashima, T.; Matsuo, M.; et al. Developing an Artificial Intelligence-Based Headache Diagnostic Model and Its Utility for Non-Specialists’ Diagnostic Accuracy. Cephalalgia 2023, 43, 033310242311569. [Google Scholar] [CrossRef] [PubMed]
  66. Katsuki, M.; Matsumori, Y.; Kawamura, S.; Kashiwagi, K.; Koh, A.; Tachikawa, S.; Yamagishi, F. Developing an Artificial Intelligence-Based Diagnostic Model of Headaches from a Dataset of Clinic Patients’ Records. Headache 2023, 63, 1097–1108. [Google Scholar] [CrossRef] [PubMed]
  67. Luo, W.; Phung, D.; Tran, T.; Gupta, S.; Rana, S.; Karmakar, C.; Shilton, A.; Yearwood, J.; Dimitrova, N.; Ho, T.B.; et al. Guidelines for Developing and Reporting Machine Learning Predictive Models in Biomedical Research: A Multidisciplinary View. J. Med. Internet Res. 2016, 18, e323. [Google Scholar] [CrossRef]
  68. Cuschieri, S. The STROBE Guidelines. Saudi J. Anaesth. 2019, 13, S31–S34. [Google Scholar] [CrossRef]
  69. Zamanipoor Najafabadi, A.H.; Ramspek, C.L.; Dekker, F.W.; Heus, P.; Hooft, L.; Moons, K.G.M.; Peul, W.C.; Collins, G.S.; Steyerberg, E.W.; van Diepen, M. TRIPOD Statement: A Preliminary Pre-Post Analysis of Reporting and Methods of Prediction Models. BMJ Open 2020, 10, e041537. [Google Scholar] [CrossRef]
  70. Roesch, A.; Dahlem, M.A.; Neeb, L.; Kurth, T. Validation of an Algorithm for Automated Classification of Migraine and Tension-Type Headache Attacks in an Electronic Headache Diary. J. Headache Pain 2020, 21, 75. [Google Scholar] [CrossRef]
  71. Do, T.P.; Remmers, A.; Schytz, H.W.; Schankin, C.; Nelson, S.E.; Obermann, M.; Hansen, J.M.; Sinclair, A.J.; Gantenbein, A.R.; Schoonman, G.G. Red and Orange Flags for Secondary Headaches in Clinical Practice: SNNOOP10 List. Neurology 2019, 92, 134–144. [Google Scholar] [CrossRef]
  72. Chiang, C.-C.; Fang, X.; Horvath, Z.; Cadiou, F.; Urani, A.; Poh, W.; Narimatsu, H.; Cheng, Y.; Dodick, D.W. Simultaneous Comparisons of 25 Acute Migraine Medications Based on 10 Million Users’ Self-Reported Records from a Smartphone Application. Neurology 2023, 101, e2560–e2570. [Google Scholar] [CrossRef]
  73. Gonzalez-Martinez, A.; Pagán, J.; Sanz-García, A.; García-Azorín, D.; Rodríguez-Vico, J.S.; Jaimes, A.; García, A.G.; de Terán, J.D.; González-García, N.; Quintas, S.; et al. Machine-Learning-Based Approach for Predicting Response to Anti-Calcitonin Gene-Related Peptide (CGRP) Receptor or Ligand Antibody Treatment in Patients with Migraine: A Multicenter Spanish Study. Eur. J. Neurol. 2022, 29, 3102–3111. [Google Scholar] [CrossRef] [PubMed]
  74. Chartier, C.; Gfrerer, L.; Knoedler, L.; Austen, W.G. Artificial Intelligence-Enabled Evaluation of Pain Sketches to Predict Outcomes in Headache Surgery. Plast. Reconstr. Surg. 2023, 151, 405–411. [Google Scholar] [CrossRef] [PubMed]
  75. Chiang, C.-C.; Luo, M.; Dumkrieger, G.; Trivedi, S.; Chen, Y.-C.; Chao, C.-J.; Schwedt, T.J.; Sarker, A.; Banerjee, I. A Large Language Model-Based Generative Natural Language Processing Framework Finetuned on Clinical Notes Accurately Extracts Headache Frequency from Electronic Health Records. medRxiv 2023. [Google Scholar] [CrossRef] [PubMed]
  76. Cady, R.K.; Borchert, L.D.; Spalding, W.; Hart, C.C.; Sheftell, F.D. Simple and Efficient Recognition of Migraine with 3-Question Headache Screen. Headache 2004, 44, 323–327. [Google Scholar] [CrossRef] [PubMed]
  77. Lipton, R.B.; Dodick, D.; Sadovsky, R.; Kolodner, K.; Endicott, J.; Hettiarachchi, J.; Harrison, W. ID Migraine validation study A Self-Administered Screener for Migraine in Primary Care: The ID Migraine Validation Study. Neurology 2003, 61, 375–382. [Google Scholar] [CrossRef]
  78. Aygun, D.; Bildik, F. Clinical Warning Criteria in Evaluation by Computed Tomography the Secondary Neurological Headaches in Adults. Eur. J. Neurol. 2003, 10, 437–442. [Google Scholar] [CrossRef]
Table 1. Patient characteristics (ground truth).
Table 1. Patient characteristics (ground truth).
Training Data *Test Data *Validation Data
n = 2800n = 1200n = 59
Mean age (standard deviation)41.27 (17.69)41.38 (18.27)42.55 (12.74)
Biological sex (%Female)64.80%63.20%86.67%
Class 1: Migraines or MOHs1597 (57.03%)708 (59.00%)56 (94.92%)
Class 2: TTHs522 (18.64%)197 (16.42%)3 (5.08%)
Class 3: TACs244 (8.71%)101 (8.42%)0
Class 4: Other primary headaches55 (1.96%)30 (2.50%)0
Class 5: Other headaches (secondary)382 (13.64%)164 (13.67%)0
Abbreviations: TACs, trigeminal autonomic cephalalgia; TTH, tension-type headache; MOH, medication-overuse headache; *: used for AI model, as previously reported in our other article [65].
Table 2. Confusion matrix.
Table 2. Confusion matrix.
Prediction by AI Performance Index
Class 1: Migraine or MOHClass 2: TTHClass 3; TACsClass 4: Other Primary HeadachesClass 5: Other Headache (Secondary)TotalAccuracySensitivity (Recall)PrecisionSpecificityF-Value
Ground truth by headache specialistClass 1: Migraines or MOHs5501005698.21%98.21%98.21%66.67%98.21%
Class 2: TTHs12000366.67%66.67%100.00%100.00%80.00%
Class 3: TACs000000---100%-
Class 4: Other primary headaches000000---100%-
Class 5: Other headaches (secondary)000000---100%-
Total5621005994.92%----
Free-marginal kappa = 0.65 (95% confidence interval: 0.21–1.00). Abbreviations: TACs, trigeminal autonomic cephalalgias; TTH, tension-type headache; MOH, medication-overuse headache.
Table 3. Previous reports on AI-based headache diagnosis models.
Table 3. Previous reports on AI-based headache diagnosis models.
AuthorYearData SourceOutput by the ModelMethodsVariablesTraining Sample NumberTest Sample NumberValidation Sample Number%MigraineAccuracySensitivity (Recall)SpecificityPrecisionF-Value
Yin [62]2015The International Headache Center in Chinese PLA General HospitalClass 2: migraine or TTHsCase-based reasoning + genetic algorithm81676222Not performed76.1%93.0%97.0%79.2%93.1%95.0%
Walters [61]2016University studentsClass 2: migraines or other headache disordersLogistic regression4887942Not performed9.4%92%94%92%64%93%
Vandewiele [63]2018Patients with headache at the Department of Neurology, Ghent University Hospital.
Migbase dataset.		Class 3: migraines, TTHs, TACsDecision treeNot described849-32Not described98%98%98%Not describedNot described
Kwon [64]2020Samsung Medical Center headache clinicClass 5: migraines, TTHs, TACs, thunderclap headaches, epicranial headacheseXtreme gradient boosting751286876Not performed68.5%58.6% †58.7% †85.6% †65.3% †58.6% †,‡
Cowan [60]2022Patients at three academic headache centersClass 2: migraines or other headache disordersDecision tree135--21262%92%89%97%98%93%
Katsuki [65]2023Headache Center, Tominaga HospitalClass 5: migraines or MOHs, TTHs, TACs, other primary headaches, or secondary headachesLight gradient-boosting machine17280012005060.0%90.0%68.6%95.0%96.4%88.1%
Katsuki [66]2023Sendai Headache and Neurology ClinicClass 5: migraines or MOHs, TTHs, TACs, other primary headaches, or other headachesGradient-boosting classifier2242401818Not performed.79.7%93.7% †40.6% †48.5% †88.7% †43.5% †,‡
Ours [65]2023Headache Center, Tominaga HospitalClass 5: migraines or MOHs, TTHs, TACs, other primary headaches, or secondary headachesLight gradient-boosting machine17280012005989.83%92.11% §92.11% §92.86% §33.33% §95.41% §
Abbreviations: AI, artificial intelligence; TACs, trigeminal autonomic cephalalgias; TTH, tension-type headache; MOH, medication-overuse headache; † Calculated as a macro-average; ‡ F-value by definition A; § only for migraine diagnosis.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Okada, M.; Katsuki, M.; Shimazu, T.; Takeshima, T.; Mitsufuji, T.; Ito, Y.; Ohbayashi, K.; Imai, N.; Miyahara, J.; Matsumori, Y.; et al. Preliminary External Validation Results of the Artificial Intelligence-Based Headache Diagnostic Model: A Multicenter Prospective Observational Study. Life 2024, 14, 744. https://doi.org/10.3390/life14060744

AMA Style

Okada M, Katsuki M, Shimazu T, Takeshima T, Mitsufuji T, Ito Y, Ohbayashi K, Imai N, Miyahara J, Matsumori Y, et al. Preliminary External Validation Results of the Artificial Intelligence-Based Headache Diagnostic Model: A Multicenter Prospective Observational Study. Life. 2024; 14(6):744. https://doi.org/10.3390/life14060744

Chicago/Turabian Style

Okada, Mariko, Masahito Katsuki, Tomokazu Shimazu, Takao Takeshima, Takashi Mitsufuji, Yasuo Ito, Katsumi Ohbayashi, Noboru Imai, Junichi Miyahara, Yasuhiko Matsumori, and et al. 2024. "Preliminary External Validation Results of the Artificial Intelligence-Based Headache Diagnostic Model: A Multicenter Prospective Observational Study" Life 14, no. 6: 744. https://doi.org/10.3390/life14060744

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop