Next Article in Journal
Apathy in Parkinson’s Disease: Defining the Park Apathy Subtype
Previous Article in Journal
Comprehensive Cortical Structural Features Predict the Efficacy of Cognitive Behavioral Therapy in Obsessive-Compulsive Disorder
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Brain Sciences
Volume 12
Issue 7
10.3390/brainsci12070922
brainsci-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Review

Latest Trends in Outcome Measures in Dementia and Mild Cognitive Impairment Trials

by
Divyani Garg
1,
Anu Gupta
2,
Ayush Agarwal
2,
Biswamohan Mishra
2,
Madakasira Vasantha Padma Srivastava
2,
Aneesh Basheer
3 and
Venugopalan Y. Vishnu
2,*
1
Department of Neurology, Vardhman Mahavir Medical College and Safdarjung Hospital, New Delhi 110029, India
2
Department of Neurology, All India Institute of Medical Sciences, New Delhi 110029, India
3
Department of Medicine, DM Wayanad Institute of Medical Sciences (DM WIMS), Wayanad 673577, India
*
Author to whom correspondence should be addressed.
Brain Sci. 2022, 12(7), 922; https://doi.org/10.3390/brainsci12070922
Submission received: 30 May 2022 / Revised: 12 July 2022 / Accepted: 12 July 2022 / Published: 14 July 2022
(This article belongs to the Section Behavioral Neuroscience)
Browse Figure
Versions Notes

Abstract: Background

Disease modification trials in dementia and mild cognitive impairment (MCI) have not met with success. One potential criticism of these trials is the lack of sensitive outcome measures. A large number of outcome measures have been employed in dementia and MCI trials. This review aims to describe and analyze the utility of cognitive/clinical outcome measures in Alzheimer’s disease (AD) and MCI trials. Methods: A PubMed search was conducted using relevant MeSH terms and exploded keywords. The search was confined to English language publications of human studies from the last five years which describe the latest trends in the use of outcome measures. Results: Despite broad use, the outcome measures employed are heterogeneous, with little data on correlations between scales. Another problem is that most studies are over-reliant on clinician/researcher assessment and cognitive outcomes, and there is a definite lack of stakeholder input. Finetuning of the paradigm is also required for people with early-stage disease, mild to moderate disease, and advanced dementia, as the outcome measures in these subgroups have varying relevance. Disease modification/prevention is an appropriate goal in early disease, whereas palliation and freedom from discomfort are paramount in later stages. The outcome measures selected must be suitable for and sensitive to these particular care goals. Although there is a shift to enrich MCI cohorts using a biomarker-based approach, the clinical relevance of such outcome measures remains uncertain. Conclusions: Outcome measures in dementia/MCI trials remain inhomogeneous and diverse, despite extensive use. Outcome measures fall within several paradigms, including cognitive, functional, quality-of-life, biomarker-based, and patient-reported outcome measures. The success of future disease-modifying trials is reliant to a large extent on the selection of outcome measures which combine all outcomes of clinical relevance as well as clinical meaning. Outcome measures should be tied to the type and stage of dementia and to the specific interventions employed.

1. Introduction

Around 47 million people live with dementia globally, with this number estimated to rise to 82 million by 2030 and 152 million by 2050 [1]. A number of common comorbidities are known to impact cognitive abilities, including diabetes, prediabetes [2,3] and hypertension [4]. Although the typical clinical dementia syndrome is readily recognizable, certain uncommon features, either in isolation or combination, may also herald the diagnosis. These include behavioural and psychological syndromes such as obsessive-compulsive disorder, repetitive questioning, minor accidents while driving, repeated falls, lack of personal attention, wandering, inappropriate use of objects, and rapid eye movement sleep behaviour disorder [5].
None of the disease modification therapy trials in Alzheimer’s disease (AD) or other forms of dementia have yielded success. One criticism of these trials has been the use of inappropriate outcome measures, including a perceived lack of stakeholder input and reliance on clinician/researcher assessments. Ref. [6] Outcome measures in dementia/MCI trials are numerous, largely heterogeneous, and are over-reliant on cognitive measures (Table 1). A review of 676 dementia and 129 MCI trial records found a large number of outcome measures, including several which were not standardised [7]. Multiple regulatory agencies have emphasized using cognitive, functional and global outcomes measures in AD, the dementia subtype most widely studied [8]. Selecting appropriate outcome measures in clinical trials depends on the study design and is crucial for ensuring the strength of the study. In dementia and mild cognitive impairment (MCI), both clinical and non-clinical outcomes have been employed (Figure 1). Non-clinical outcomes are based on radiological or other biomarker-based assessments. The reliability of the latter as surrogate outcome measures is debatable, as concordant results have not been achieved with clinical measures. Dementia outcomes focus on measuring either disease incidence and severity or the disease’s functional impact, which may be physical, cognitive, behavioural or psychological. Global assessment and quality of life-based scales may be appropriate, as dementia has a complex effect on the patient’s lived experience. Outcome measures that assess the impact on the caregiver/s are also increasingly encouraged. Ref. [9] In a similar vein, patient-reported outcome measures (PROMs) are also increasingly recommended, as they represent the patients’ perspectives and provide them an opportunity to share their concerns and viewpoints. However, PROMs continue to remain underutilized in dementia registries [10].
In this review, we aim to detail and analyze clinical and cognitive outcome measures that have been used in AD and MCI trials in order to document their strengths and weaknesses and utility as outcome measures in AD/MCI trials. We will not focus on biomarkers as outcome measures, since these require in-depth evaluation, which is beyond the scope of this review.

2. Search Methodology

We performed a search in the PubMed database on 30 April 2022 using the following MeSH terms: “dementia”, “cognitive impairment”, “cognitive dysfunction”, “outcome assessment, healthcare”, “patient reported outcome measures”, “Alzheimer disease”, and the keywords: dementia, mild cognitive impairment, minor neurocognitive disorder, outcome measure. The search was confined to studies on human subjects published in English and limited to the last five years. We included studies which reported the utility of outcome measures in dementia and MCI trials. The studies were screened based on title and abstract by two reviewers (DG, AG) and any conflict was resolved in concert with a third reviewer (VVY). After initial screening, the full text of eligible studies were accessed and assessed for their suitability. Relevant cross-references were also screened and assessed for eligibility. The initial search retrieved 1394 results. After screening the titles and abstracts, the full texts of 179 studies were accessed. A total of 53 studies were included in the final review.

3. Outcome Measures

3.1. Cognitive Outcome Measures

The majority of data on outcome measures in dementia emanates from studies on AD. Ref. [11] Less data is available for vascular dementia (VaD) and Dementia with Lewy Bodies (DLB). More than 300 cognitive measures have been utilized in AD/MCI studies. Some of the frequently used measures are described below. The psychometric properties of cognitive outcome measures are described in detail in a comprehensive review by Bossers et al. [11].

3.1.1. Global Cognitive Function

Mini-Mental State Examination (MMSE)
The Mini-Mental State Examination (MMSE) is among the most frequently used cognitive outcome measure in dementia/MCI trials [7,12]. It has good internal consistency (Cronbach’s alpha > 0.7). Its test-retest reliability (intraclass correlations ICC 0.95), construct and criterion validity are also established [13,14].
For conversion from MCI to dementia, the sensitivity of baseline MMSE scores ranges from 23 to 76% and specificity from 40 to 94% [15]. For conversion from MCI to AD specifically, the sensitivity ranges from 27 to 89% and specificity from 32 to 90%. Floor and ceiling effects with preclinical AD and MCI are known to occur with the MMSE. MMSE may not be able to differentiate MCI from healthy controls [16]. Stand-alone MMSE may not be sufficient to predict MCI progression and may not be an optimal outcome in predementia and MCI trials. Although MMSE is a popular outcome measure, it was not designed as such, and data on its psychometric properties is limited [17].
Link: https://www.sciencedirect.com/science/article/abs/pii/0022395675900266?via%3Dihub (accessed on 11 July 2022).
Alzheimer’s Disease Assessment Scale-Cognitive subscale (ADAS-COG)
The ADAS was designed as a rating scale to assess the extent of both cognitive and non-cognitive issues in patients with AD [18]. Similarly to MMSE, it is a frequently used cognitive outcome measure in trials [7,12]. It consists of two subscales: ADAS-Cog and ADAS-Noncog. The cognitive subscale, ADAS-Cog, comprises 11 parts which are both patient-completed and assessor-based and test episodic memory, language, orientation and praxis. ADAS-Cog is one of the most widely used outcome measures in mild to moderate AD trials and has established utility in trials on cholinesterase inhibitors in dementia.
Severe floor and ceiling effects have been demonstrated with ADAS-Cog in MCI, mild and early AD, and preclinical states, with some of the item categories not yielding the intended results according to a Rasch analysis [19]. For these reasons, it has been suggested that ADAS-Cog is not an appropriate measure in pre-dementia or early dementia trials [18].
Link: https://www.fda.gov/media/122843/download (accessed on 11 July 2022).
Montreal Cognitive Assessment (MoCA)
MoCA is a 30-point screening tool and takes around 10 min to administer. Scores range between 0 and 30, and scores above 26 are abnormal. It has a sensitivity of 90% for detection of MCI (compared with 18% using MMSE) and 100% for mild AD. It also has excellent sensitivity (87%) [20]. MoCA also extends the cognitive evaluation offered by MMSE by including working memory, orientation, immediate and delayed memory, executive function and visuospatial abilities.
Link: https://www.mocatest.org/ (accessed on 11 July 2022).
Both MMSE and MOCA have been monetized, but there are special considerations for research studies and data collection.
Saint Louis University Mental State (SLUMS)
SLUMS is a tool for the assessment of mild cognitive impairment and dementia [21]. SLUMS was designed to overcome some of the limitations of MMSE [22]. It assesses memory, orientation and attention. In addition, executive function is assessed using the clock-drawing test and the animal recall task.
Link: https://www.slu.edu/medicine/internal-medicine/geriatric-medicine/aging-successfully/assessment-tools/mental-status-exam.php (accessed on 11 July 2022).

3.1.2. Domain-Specific Cognitive Tests

These tests supplement the MMSE or the ADAS-Cog and cover specific cognitive subdomains. Some of the tests included in this category are the Trail Making Tests A and B, the Clock Drawing Test, the digit Span forward test, the digit Span backward test, word recognition tests, fluency tests, the Mohs Number Cancellation Test, the Rey Memory Test, the Digit Symbol Substitution Test, and the Buschke Selective Reminding test.

3.2. Functional Outcome Measures

Functional measures are underutilized in studies on MCI, with an estimated 16% using any measure of functionality [7].

3.2.1. Alzheimer Disease Cooperative Study Activities of Daily Living Scale (ADCS-ADL)

The ADCS-ADL assesses basic and instrumental ADLs among patients with AD [23]. It may be clinician-administered or completed by the caregiver. The responses are based on the preceding four-week period. For patients with MMSE 0–15, the Alzheimer Disease Cooperative Study ADL-sev scale was developed [24]. It has also been adapted for patients with MCI.
Link: https://pubmed.ncbi.nlm.nih.gov/9236950/ (accessed on 11 July 2022).

3.2.2. Barthel Index (BI)

BI is widely used in the assessment of basic ADLs, as it is simple to administer and score. Although it is validated and used extensively [25], validation in dementia is supported by limited evidence [26]. It consists of 10 items, and the cumulative score is calculated with scores ranging from 0 to 20. Although BI has good internal consistency, unidimensionality was not found to be robust. In a recent evaluation of BI in persons with dementia, multiple issues were found to interfere with its performance, including misfit items, item bias, measurement gaps etc. [26]. The authors suggested that BI needs further refinement before use with dementia patients.
Link: https://pubmed.ncbi.nlm.nih.gov/14258950/ (accessed on 11 July 2022).

3.2.3. Disability Assessment for Dementia (DAD)

DAD is a questionnaire comprising 46 items which is administered to the caregiver and assesses basic and instrumental ADLs. It has a high internal consistency (Cronbach’s alpha > 0.8), good test-retest (ICC = 0.96) and inter-rater reliability (ICC = 0.95) [27].
Link: https://research.aota.org/ajot/article-abstract/53/5/471/4350/Development-of-a-Functional-Measure-for-Persons?redirectedFrom=fulltext (accessed on 11 July 2022).

3.3. Quality of Life Outcome Measures

3.3.1. Quality of Life in Alzheimer’s Disease (QOL-AD)

The QOL-AD scale is a 13-item scale, with scores ranging from 13 to 52. Higher scores are indicative of better quality of life. The scale has high content validity [28,29]. and the interrater reliability is also good, with Cohen’s kappa values > 0.70. Likewise, it has good internal consistency, with a Cronbach’s alpha coefficient of 0.82.
Link: https://journals.lww.com/psychosomaticmedicine/Abstract/2002/05000/Assessing_Quality_of_Life_in_Older_Adults_With.16.aspx (accessed on 11 July 2022).

3.3.2. Quality of Life in Late-Stage Dementia (QUALID)

QUALID is an 11-item scale, with total scores in the range 11–55. It was created from a subset of a larger number of items by Albert et al. [30]. Lower scores are consistent with a better quality of life [31]. The items are rated on a five-point Likert scale. It can be used for persons with an MMSE score ≥ 3 and was initially designed for use in patients with severe dementia in long-term care facilities. It is an assessor-administered questionnaire, which requires contact with the patient for at least 30 h. It is administered over an observation period of one week. It is modulated by the effect of antidepressant or antipsychotic drugs, leading to lower scores.
QUALID has been found to have good internal consistency, with a Cronbach’s alpha of 0.77. It has good test-retest and interrater reliability (ICC = 0.8).
Link: https://pubmed.ncbi.nlm.nih.gov/12818023/ (accessed on 11 July 2022).

3.3.3. Quality of Life for People with Dementia (QUALIDEM)

QUALIDEM is a validated, questionnaire-based tool for assessing the quality of life of patients in residential care [32]. It is administered by professional caregivers.
Link: https://onlinelibrary.wiley.com/doi/10.1002/gps.1713 (accessed on 11 July 2022).

3.4. Neuropsychiatric Outcome Measures

3.4.1. Neuropsychiatric Inventory (NPI)

The NPI is a popular scale that assesses 12 behavioural and psychiatric issues that occur in persons with dementia [33]. It has been established as both valid and reliable [34]. It has been translated into more than 40 languages, enabling wide usage. Four major versions have been developed [34]. The original 10-item scale was expanded to include sleep and appetite changes, to form the 12-item NPI. Another version, NPI-12 with integrated caregiver distress, is used most frequently. The NPI, Nursing Home version (NPI-NH), is a modification for use in residential facilities. It is suitable for use in settings where non-family caregivers are the informants.
Link: https://download.lww.com/wolterskluwer_vitalstream_com/permalink/cont/a/cont_21_3_2015_02_26_kaufer_2015-10_sdc2.pdf (accessed on 11 July 2022).

3.4.2. Brief Psychiatric Rating Scale (BPRS)

BPRS was developed for patients with schizophrenia and was extrapolated for use with patients with dementia. However, the delusions and hallucinations observed in schizophrenia differ from those prevalent in neurodegenerative disorders.
Link: https://www.smchealth.org/sites/main/files/file-attachments/bprsform.pdf?1497977629 (accessed on 11 July 2022).

3.4.3. Hamilton Rating Scale for Depression

Certain aspects of the scale, such as weight and appetite changes and social withdrawal, may occur in persons with dementia in the absence of depression.
Several other scales used in clinical trials include the Cohen-Mansfield Agitation Inventory, the Geriatric Depression Scale and the Cornell Scale for Depression in Dementia.
Link: https://jnnp.bmj.com/content/23/1/56.long (accessed on 11 July 2022).

3.5. Global Outcomes Measures

3.5.1. Clinician’s Interview-Based Impression of Change Plus Caregiver Interview (CIBIC-Plus)

CIBIC-Plus is a global outcome measure widely used in trials among patients with advanced dementia as a co-primary outcome [35,36,37,38]. It is based on a seven-point rating scale based on clinician judgement and a semi-structured interview to assess cognition, behaviour and function. There has been a decline in its use with a shift towards using more objective measures. In a recent trial, CIBIC-Plus compared favorably to goal attainment scaling (GAS) for clinical meaningfulness [39].
Link: https://pubmed.ncbi.nlm.nih.gov/9236949/ (accessed on 11 July 2022).

3.5.2. Clinical Dementia Rating (CDR)

CDR is a global assessment tool which provides global and Sum of Boxes (SOB) scores [36,40]. Global scores are used for the assessment of dementia severity. However, it contains no measure of behavioural issues.
Link: https://www.cambridge.org/core/journals/the-british-journal-of-psychiatry/article/abs/new-clinical-scale-for-the-staging-of-dementia/D1AAE7A0836C1E36B450461613521D20 (accessed on 11 July 2022).

3.5.3. Clinical Dementia Rating-Sum of Boxes (CDR-SOB)

CDR-SOB is derived from the CDR. CDR-SOB has been assessed as a single primary outcome measure in mild to moderate AD. It showed good internal consistency (Cronbach’s alpha 0.88) and acceptable validity. It also has low variability, leading to smaller sample sizes compared with ADAS-Cog [41]. CDR-SOB is easier to calculate (Global CDR calculation is demanding and subject to errors unless one uses the online calculators) and better at detecting change across stages of dementia.
Link: https://pubmed.ncbi.nlm.nih.gov/3389756/ (accessed on 11 July 2022).

3.5.4. Clinical Global Impressions (CGI)

The CGI scales measure the severity of symptoms (CGI-S) or changes in several psychiatric conditions (CGI-C) [42].

3.6. Patient-Reported Outcome Measures

Growing significance is attached to patient-reported health-related quality of life (HRQoL) measures in chronic disorders. PROMs are a tool to assess HRQoL as reported by the patients. Ayton et al. identified seven dementia-specific PROMs in a scoping review [10]. These included: Alzheimer’s Disease-Related Quality of Life (ADRQOL), Bath Assessment of Subjective Quality of Life in Dementia (BASQID), Dementia Quality of Life measures (D-QoL), Quality of Life in Alzheimer’s Disease (QoL-AD), QUALID and QUALIDEM. Of these, QUALID has been used in advanced dementia. ADRQOL and QUALIDEM may be caregiver-administered so may also be useful in late-stage dementia.

4. Early-Stage AD

“Prodromal AD” refers to both MCI and pre-MCI, which is defined as a state of cognitive dysfunction which does not meet the criteria for MCI. These individuals are at greater risk for the development of AD [43]. Another ‘preclinical’ group may be identified based on an increased risk for AD by genetic or biomarker-based assessment. These populations are targets for AD prevention by disease modification. Prodromal AD assessment requires newer sensitive measures rather than traditional neuropsychological tests. The outcome measures in these groups should possess “bi-directional sensitivity, longitudinal tracking, and sensitivity to impairment” [44]. The widely used ADAS-Cog is not suited to the MCI population for several reasons discussed above. Tests of metacognition, social cognition, and prospective memory are steps in this direction [44]. A few examples include the Loewenstein-Acevedo Scales of Semantic Interference and Learning (LASSI-L) [45], and short-term visual memory binding (SVMB) [46].
Despite the well-known importance of quality of life, caregiver burden and other functional outcomes in MCI and early dementia trials, cognitive outcomes continue to be widely used [47], the reason being that detection of functional impairment in early stages also requires instruments sensitive to very subtle functional changes. Tests for financial capacity, performance-based skill assessments and computerized assessments based on virtual reality and video technology are emerging in this area [44]. However, till the time we have a single holistic outcome measure for MCI, the creation and validation of cognitive composite scores (e.g., a composite score including delayed word list recall, logical memory, category fluency, tests of processing speed, tests of performance IQ etc.) may be the best option [48].

5. Mild to Moderate AD

A systematic review identified 81 outcome measures used across trials on mild to moderate AD [12,49]. The most widely used were measures of cognition and global assessment. Others included ADLs, biological markers and neuropsychological outcomes. The authors recommended the use of either ADAS-Cog or MMSE for cognitive outcomes. Structural MRI was judged to be a core outcome. For neuropsychiatric symptoms, NPI was recommended. DEMQOL was recommended for the assessment of the quality of life. Fluid biomarkers and ADLs were not recommended in this consensus statement. Although the consensus panel did not recommend the global outcome as a core outcome, CDR was considered appropriate if it was to be used. Overall, both cognition and biomarkers were recommended as core outcomes in persons with mild to moderate AD.

6. Advanced AD

A different paradigm applies to the application of outcome measures in individuals with advanced dementia, with a shift to palliative care and ensuring the quality of life [50]. For assessing quality of life, QUALID may be appropriate, as it was designed for patients with advanced dementia in residential care. Comfort is another important target in this group. Discomfort Scale for Dementia of the Alzheimer’s type (DS-DAT), which is based on interviews of nurses caring for persons with dementia, has nine items [51]. It has good psychometric properties, with good internal consistency and interrater reliability. Similarly, other targets in this population include assessment of engagement, pain, behaviour, agitation, apathy, rejection of care, and respiratory issues. Scales are also available to assess end-of-life care outcomes in advanced dementia, such as the Mini-Suffering State Examination (MSSE) [52] and Comfort Assessment in Dying with Dementia (CAD-EOLD) [53].

7. Future Directions and Perspectives

There is wide heterogeneity in outcome measures used in dementia research. High variability in outcome measures usually results in a large sample size. Although multiple efforts have been made to achieve consensus on measurement scales and outcome instruments, these efforts have not produced consistent conclusions. MMSE still remains the cornerstone of outcome measures in dementia/MCI trials, despite not being designed as an outcome measure and lacking well described psychometric properties [7]. There have also been endeavors to understand how scales relate to each other and describe the correlation so that clinicians may be able to compare different scales [54], but such data remains limited. There is an urgent need for homogeneous and standardized measures in this field. It should also be emphasized that outcome measures likely need to be tailored to the type of dementia, and a one-size-fits-all approach may not be applicable, considering that different forms of dementia have varying impacts on patients.
From the Indian perspective, the ICMR-Neurocognitive toolbox (ICMR-NCTB) has been designed to diagnose dementia and MCI and can be applied across a wide linguistic and educational range in India. It has been validated for the diagnosis of MCI in India and has a sensitivity of 81.1% and specificity of 88.8% [55]. It has also been validated for the diagnosis of dementia in India, with a sensitivity and specificity of 70–100% [56] in five Indian languages (Hindi, Bengali, Telugu, Kannada, Malayalam). The ICMR-NCTB comprises multiple tests to evaluate multiple cognitive domains, including attention-executive function, memory, language, and visuospatial function, along with questionnaires for depression, functional activities and quality of life. Future studies should assess the utility of this tool as an outcome measure across the spectrum of MCI and dementia.
There is also growing interest in the clinical meaningfulness of outcome measures based on what persons living with dementia and their caregivers uphold as meaningful to their lived experience. A ‘Clinically meaningful’ effect is a large, statistically significant effect on the patient’s perception and living. Apart from memory and cognition, ADLs, mental and social health, quality of life, caregiver burden, and maintaining patient identity and independence are important to patients and caregivers [57,58], and outcome measures must include these perspectives. However, only 13% of dementia trials have included quality of life measures [7]. Hence, historically, dementia trials use the co-primary endpoint approach, that is, the use of at least two outcome measures for cognitive and functional or global impairment. However, this approach may not be valid for predementia trials due to a lack of significant functional impairment.

8. Conclusions

A large number of outcome measures have been described and evaluated in dementia and MCI trials, encompassing cognitive and non-cognitive outcomes. The large majority of current trials are still reliant on global measures such as MMSE and ADAS-Cog. Although there is a shift to enrich MCI cohorts using a biomarker-based approach, the clinical meaningfulness of such outcome measures remains uncertain. The success of future disease-modifying trials is reliant to a large extent on the selection of outcome measures which combine all outcomes of clinical relevance as well as clinical meaning. Outcome measures should be tied to the type and stage of dementia and to the specific interventions employed.

Author Contributions

V.Y.V. conceived the study. D.G. wrote the first draft. A.G., A.A., B.M., M.V.P.S., A.B. and V.Y.V. critically revised the manuscript and helped with scientific discussion. All the authors confirm they have full access to all the data in the study and accept responsibility of submission for publication. All authors have read and agreed to the published version of the manuscript.

Funding

This research and APC was funded by the Indian Council of Medical Research grant number 5/7/1704/CH/Adhoc/RBMCH2020.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

All data generated or analyzed during this study are included in this published article.

Acknowledgments

The research was supported by the Indian Council of Medical Research. The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; or the decision to submit the manuscript for publication.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Sosa-Ortiz, A.L.; Acosta-Castillo, I.; Prince, M.J. Epidemiology of dementias and Alzheimer’s disease. Arch Med. Res. 2012, 43, 600–608. [Google Scholar] [CrossRef] [PubMed]
  2. Biessels, G.J.; Staekenborg, S.; Brunner, E.; Brayne, C.; Scheltens, P. Risk of dementia in diabetes mellitus: A systematic review. Lancet Neurol. 2006, 5, 64–74. [Google Scholar] [CrossRef]
  3. Alster, P.; Dunalska, A.; Migda, B.; Madetko, N.; Królicki, L. The Rate of Decrease in Brain Perfusion in Progressive Supranuclear Palsy and Corticobasal Syndrome May Be Impacted by Glycemic Variability—A Pilot Study. Front. Neurol. 2021, 12, 767480. [Google Scholar] [CrossRef]
  4. Walker, K.A.; Power, M.C.; Gottesman, R.F. Defining the Relationship Between Hypertension, Cognitive Decline, and Dementia: A Review. Curr. Hypertens. Rep. 2017, 19, 24. [Google Scholar] [CrossRef] [PubMed]
  5. Arvanitakis, Z.; Shah, R.C.; Bennett, D.A. Diagnosis and Management of Dementia: A Review. JAMA 2019, 322, 1589–1599. [Google Scholar] [CrossRef]
  6. Posner, H.; Curiel, R.; Edgar, C.; Hendrix, S.; Liu, E.; Loewenstein, D.A.; Morrison, G.; Shinobu, L.; Wesnes, K.; Harvey, P.D. Outcomes Assessment in Clinical Trials of Alzheimer’s Disease and its Precursors: Readying for Short-term and Long-term Clinical Trial Needs. Innov. Clin. Neurosci. 2017, 14, 22–29. [Google Scholar] [PubMed]
  7. Harrison, J.K.; Noel-Storr, A.H.; Demeyere, N.; Reynish, E.L.; Quinn, T.J. Outcomes measures in a decade of dementia and mild cognitive impairment trials. Alzheimer’s Res. Ther. 2016, 8, 48. [Google Scholar] [CrossRef] [Green Version]
  8. Morant, A.V.; Vestergaard, H.T.; Lassen, A.B.; Navikas, V. US, EU, and Japanese Regulatory Guidelines for Development of Drugs for Treatment of Alzheimer’s Disease: Implications for Global Drug Development. Clin. Transl. Sci. 2020, 13, 652–664. [Google Scholar] [CrossRef]
  9. Katona, C.; Livingston, G.; Cooper, C.; Ames, D.; Brodaty, H.; Chiu, E. International Psychogeriatric Association consensus statement on defining and measuring treatment benefits in dementia. Int. Psychogeriatr. 2007, 19, 345–354. [Google Scholar] [CrossRef]
  10. Ayton, D.R.; Gardam, M.L.; Pritchard, E.K.; Ruseckaite, R.; Ryan, J.; Robinson, S.J.; Brodaty, H.; Ward, S.A.; Ahern, S. Patient-Reported Outcome Measures to Inform Care of People with Dementia—A Systematic Scoping Review. Gerontologist 2021, 61, e185–e194. [Google Scholar] [CrossRef]
  11. Bossers, W.J.R.; van der Woude, L.H.V.; Boersma, F.; Scherder, E.J.A.; van Heuvelen, M.J.G. Recommended Measures for the Assessment of Cognitive and Physical Performance in Older Patients with Dementia: A Systematic Review. Dement. Geriatr. Cogn. Dis. Extra 2012, 2, 589–609. [Google Scholar] [CrossRef] [PubMed]
  12. Webster, L.; Groskreutz, D.; Grinbergs-Saull, A.; Howard, R.; O’Brien, J.T.; Mountain, G.; Banerjee, S.; Woods, B.; Perneczky, R.; Lafortune, L.; et al. Core outcome measures for interventions to prevent or slow the progress of dementia for people living with mild to moderate dementia: Systematic review and consensus recommendations. PLoS ONE 2017, 12, e0179521. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  13. Tombaugh, T.N.; McIntyre, N.J. The mini-mental state examination: A comprehensive review. J. Am. Geriatr. Soc. 1992, 40, 922–935. [Google Scholar] [CrossRef]
  14. Shigemori, K.; Ohgi, S.; Okuyama, E.; Shimura, T.; Schneider, E. The factorial structure of the Mini-Mental State Examination (MMSE) in Japanese dementia patients. BMC Geriatr. 2010, 10, 36. [Google Scholar] [CrossRef] [Green Version]
  15. Arevalo-Rodriguez, I.; Smailagic, N.; i Figuls, M.R.; Ciapponi, A.; Sanchez-Perez, E.; Giannakou, A.; Pedraza, O.L.; Cosp, X.B.; Cullum, S. Mini-Mental State Examination (MMSE) for the detection of Alzheimer’s disease and other dementias in people with mild cognitive impairment (MCI). Cochrane Database Syst. Rev. 2015, 2015, CD010783. [Google Scholar] [CrossRef]
  16. O’Bryant, S.E.; Lacritz, L.H.; Hall, J.; Waring, S.C.; Chan, W.; Khodr, Z.G.; Massman, P.J.; Hobson, V.; Cullum, C.M. Validation of the new interpretive guidelines for the clinical dementia rating scale sum of boxes score in the national Alzheimer’s coordinating center database. Arch. Neurol. 2010, 67, 746–749. [Google Scholar] [CrossRef]
  17. Pangman, V.C.; Sloan, J.; Guse, L. An examination of psychometric properties of the mini-mental state examination and the standardized mini-mental state examination: Implications for clinical practice. Appl. Nurs. Res. 2000, 13, 209–213. [Google Scholar] [CrossRef]
  18. Kueper, J.K.; Speechley, M.; Montero-Odasso, M. The Alzheimer’s Disease Assessment Scale–Cognitive Subscale (ADAS-Cog): Modifications and Responsiveness in Pre-Dementia Populations. a Narrative Review. J. Alzheimers Dis. 2018, 63, 423–444. [Google Scholar] [CrossRef] [Green Version]
  19. Hobart, J.; Cano, S.; Posner, H.; Selnes, O.; Stern, Y.; Thomas, R.; Zajicek, J.; Alzheimer’s Disease Neuroimaging Initiative. Putting the Alzheimer’s cognitive test to the test II: Rasch Measurement Theory. Alzheimers Dement 2013, 9 (Suppl. S1), S10–S20. [Google Scholar] [CrossRef]
  20. Nasreddine, Z.S.; Phillips, N.A.; Bédirian, V.; Charbonneau, S.; Whitehead, V.; Collin, I.; Cummings, J.L.; Chertkow, H. The Montreal Cognitive Assessment, MoCA: A brief screening tool for mild cognitive impairment. J. Am. Geriatr. Soc. 2005, 53, 695–699. [Google Scholar] [CrossRef]
  21. Tariq, S.H.; Tumosa, N.; Chibnall, J.T.; Perry, M.H., 3rd; Morley, J.E. Comparison of the Saint Louis University mental status examination and the mini-mental state examination for detecting dementia and mild neurocognitive disorder—A pilot study. Am. J. Geriatr. Psychiatry 2006, 14, 900–910. [Google Scholar] [CrossRef] [PubMed]
  22. Yoelin, A.B.; Saunders, N.W. Score Disparity between the MMSE and the SLUMS. Am. J. Alzheimers Dis. Other Demen. 2017, 32, 282–288. [Google Scholar] [CrossRef] [PubMed]
  23. Galasko, D.; Bennett, D.; Sano, M.; Ernesto, C.; Thomas, R.; Grundman, M.; Ferris, S. An inventory to assess activities of daily living for clinical trials in Alzheimer’s disease. The Alzheimer’s Disease Cooperative Study. Alzheimer Dis. Assoc. Disord. 1997, 11 (Suppl. S2), S33–S39. [Google Scholar] [CrossRef]
  24. Galasko, D.; Schmitt, F.; Thomas, R.; Jin, S.; Bennett, D. Alzheimer’s Disease Cooperative Study. Detailed assessment of activities of daily living in moderate to severe Alzheimer’s disease. J. Int. Neuropsychol. Soc. 2005, 11, 446–453. [Google Scholar] [CrossRef]
  25. Hopman-Rock, M.; van Hirtum, H.; de Vreede, P.; Freiberger, E. Activities of daily living in older community-dwelling persons: A systematic review of psychometric properties of instruments. Aging Clin. Exp. Res. 2019, 31, 917–925. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  26. Yi, Y.; Ding, L.; Wen, H.; Wu, J.; Makimoto, K.; Liao, X. Is Barthel Index Suitable for Assessing Activities of Daily Living in Patients With Dementia? Front. Psychiatry 2020, 11, 282. [Google Scholar] [CrossRef]
  27. Gélinas, I.; Gauthier, L.; McIntyre, M.; Gauthier, S. Development of a functional measure for persons with Alzheimer’s disease: The disability assessment for dementia. Am. J. Occup. Ther. 1999, 53, 471–481. [Google Scholar] [CrossRef] [Green Version]
  28. Thorgrimsen, L.; Selwood, A.; Spector, A.; Royan, L.; de Madariaga Lopez, M.; Woods, R.T.; Orrell, M. Whose quality of life is it anyway? The validity and reliability of the Quality of Life-Alzheimer’s Disease (QoL-AD) scale. Alzheimer Dis Assoc. Disord. 2003, 17, 201–208. [Google Scholar] [CrossRef]
  29. Kahle-Wrobleski, K.; Ye, W.; Henley, D.; Hake, A.M.; Siemers, E.; Chen, Y.F.; Liu-Seifert, H. Assessing quality of life in Alzheimer’s disease: Implications for clinical trials. Alzheimers Dement. 2016, 6, 82–90. [Google Scholar] [CrossRef]
  30. Albert, S.M.; Del Castillo-Castaneda, C.; Sano, M.; Jacobs, D.M.; Marder, K.; Bell, K.; Bylsma, F.; Lafleche, G.; Brandt, J.; Albert, M.; et al. Quality of life in patients with Alzheimer’s disease as reported by patient proxies. J. Am. Geriatr. Soc. 1996, 44, 1342–1347. [Google Scholar] [CrossRef]
  31. Weiner, M.F.; Martin-Cook, K.; Svetlik, D.A.; Saine, K.; Foster, B.; Fontaine, C.S. The quality of life in late-stage dementia (QUALID) scale. J. Am. Med. Dir. Assoc. 2000, 1, 114–116. [Google Scholar] [PubMed]
  32. Ettema, T.P.; Dröes, R.M.; de Lange, J.; Mellenbergh, G.J.; Ribbe, M.W. QUALIDEM: Development and evaluation of a dementia specific quality of life instrument. Scalability, reliability and internal structure. Int. J. Geriatr. Psychiatry 2007, 22, 549–556. [Google Scholar] [CrossRef] [PubMed]
  33. Cummings, J.L.; Mega, M.; Gray, K.; Rosenberg-Thompson, S.; Carusi, D.A.; Gornbein, J. The Neuropsychiatric Inventory: Comprehensive assessment of psychopathology in dementia. Neurology 1994, 44, 2308–2314. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  34. Cummings, J. The Neuropsychiatric Inventory: Development and Applications. J. Geriatr. Psychiatry Neurol. 2020, 33, 73–84. [Google Scholar] [CrossRef]
  35. Cui, S.; Chen, N.; Yang, M.; Guo, J.; Zhou, M.; Zhu, C.; He, L. Cerebrolysin for vascular dementia. Cochrane Database Syst. Rev. 2019, 11, CD008900. [Google Scholar] [CrossRef]
  36. Birks, J.S.; Grimley Evans, J. Rivastigmine for Alzheimer’s disease. Cochrane Database Syst. Rev. 2015, 4, CD001191. [Google Scholar] [CrossRef]
  37. Birks, J.S.; Harvey, R.J. Donepezil for dementia due to Alzheimer’s disease. Cochrane Database Syst. Rev. 2018, 6, CD001190. [Google Scholar] [CrossRef]
  38. Craig, D.; Birks, J. Galantamine for vascular cognitive impairment. Cochrane Database Syst. Rev. 2006, 1, CD004746. [Google Scholar] [CrossRef]
  39. Stanley, J.; Howlett, S.E.; Dunn, T.; Rockwood, K. The Clinician’s Interview-Based Impression of Change (Plus caregiver input) and goal attainment in two dementia drug trials: Clinical meaningfulness and the initial treatment response. Alzheimers Dement. 2021, 17, 856–865. [Google Scholar] [CrossRef]
  40. Hughes, C.P.; Berg, L.; Danziger, W.L.; Coben, L.A.; Martin, R.L. A new clinical scale for the staging of dementia. Br. J. Psychiatry 1982, 140, 566–572. [Google Scholar] [CrossRef]
  41. Coley, N.; Andrieu, S.; Jaros, M.; Weiner, M.; Cedarbaum, J.; Vellas, B. Suitability of the Clinical Dementia Rating-Sum of Boxes as a single primary endpoint for Alzheimer’s disease trials. Alzheimers Dement. 2011, 7, 602–610.e2. [Google Scholar] [CrossRef]
  42. Guy, W. ECDEU Assessment Manual for Psychopharmacology. 1976. Available online: http://archive.org/details/ecdeuassessmentm1933guyw (accessed on 28 May 2022).
  43. Espinosa, A.; Alegret, M.; Valero, S.; Vinyes-Junqué, G.; Hernández, I.; Mauleón, A.; Rosende-Roca, M.; Ruiz, A.; López, O.; Tárraga, L.; et al. A longitudinal follow-up of 550 mild cognitive impairment patients: Evidence for large conversion to dementia rates and detection of major risk factors involved. J. Alzheimers Dis. 2013, 34, 769–780. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  44. Harvey, P.D.; Cosentino, S.; Curiel, R.; Goldberg, T.E.; Kaye, J.; Loewenstein, D.; Marson, D.; Salmon, D.; Wesnes, K.; Posner, H. Performance-based and Observational Assessments in Clinical Trials Across the Alzheimer’s Disease Spectrum. Innov. Clin. Neurosci. 2017, 14, 30–39. [Google Scholar] [PubMed]
  45. Albert, M.S.; DeKosky, S.T.; Dickson, D.; Dubois, B.; Feldman, H.H.; Fox, N.C.; Gamst, A.; Holtzman, D.M.; Jagust, W.J.; Petersen, R.C.; et al. The diagnosis of mild cognitive impairment due to Alzheimer’s disease: Recommendations from the National Institute on Aging-Alzheimer’s Association workgroups on diagnostic guidelines for Alzheimer’s disease. Alzheimers Dement. 2011, 7, 270–279. [Google Scholar] [CrossRef] [Green Version]
  46. Parra, M.A.; Abrahams, S.; Logie, R.H.; Méndez, L.G.; Lopera, F.; Della Sala, S. Visual short-term memory binding deficits in familial Alzheimer’s disease. Brain 2010, 133, 2702–2713. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  47. Couch, E.; Lawrence, V.; Co, M.; Prina, M. Outcomes tested in non-pharmacological interventions in mild cognitive impairment and mild dementia: A scoping review. BMJ Open 2020, 10, e035980. [Google Scholar] [CrossRef] [Green Version]
  48. Malek-Ahmadi, M.; Chen, K.; Perez, S.E.; He, A.; Mufson, E.J. Cognitive composite score association with Alzheimer’s disease plaque and tangle pathology. Alzheimer’s Res. Ther. 2018, 10, 90. [Google Scholar] [CrossRef]
  49. Webster, L.; Groskreutz, D.; Grinbergs-Saull, A.; Howard, R.; TO’Brien, J.; Mountain, G.; Banerjee, S.; Woods, B.; Perneczky, R.; Lafortune, L.; et al. Development of a core outcome set for disease modification trials in mild to moderate dementia: A systematic review, patient and public consultation and consensus recommendations. Health Technol. Assess. 2017, 21, 1–192. [Google Scholar] [CrossRef] [Green Version]
  50. Volicer, L.; van der Steen, J.T. Outcome Measures for Dementia in the Advanced Stage and at the End of Life. Adv. Geriatr. 2014, 2014, e346485. [Google Scholar] [CrossRef] [Green Version]
  51. Hurley, A.C.; Volicer, B.J.; Hanrahan, P.A.; Houde, S.; Volicer, L. Assessment of discomfort in advanced Alzheimer patients. Res. Nurs. Health 1992, 15, 369–377. [Google Scholar] [CrossRef]
  52. Aminoff, B.Z.; Purits, E.; Noy, S.; Adunsky, A. Measuring the suffering of end-stage dementia: Reliability and validity of the Mini-Suffering State Examination. Arch. Gerontol. Geriatr. 2004, 38, 123–130. [Google Scholar] [CrossRef] [PubMed]
  53. Volicer, L.; Hurley, A.C.; Blasi, Z.V. Scales for evaluation of End-of-Life Care in Dementia. Alzheimer Dis. Assoc. Disord. 2001, 15, 194–200. [Google Scholar] [CrossRef] [PubMed]
  54. Samara, M.; Levine, S.Z.; Yoshida, K.; Goldberg, Y.; Cipriani, A.; Efthimiou, O.; Iwatsubo, T.; Leucht, S.; Furakawa, T.A. Linking the Clinical Dementia Rating Scale-Sum of Boxes, the Clinician’s Interview-Based Impression plus Caregiver Input, and the Clinical Global Impression Scale: Evidence based on Individual Participant Data from Five Randomized Clinical Trials of Donepezil. J. Alzheimer’s Dis. 2021, 82, 1075–1084. [Google Scholar] [CrossRef]
  55. Menon, R.N.; Varghese, F.; Paplikar, A.; Mekala, S.; Alladi, S.; Sharma, M.; Onkarappa, S.A.; Gollahalli, D.; Dutt, A.; Ghosh, A.; et al. Validation of Indian Council of Medical Research Neurocognitive Tool Box in Diagnosis of Mild Cognitive Impairment in India: Lessons from a Harmonization Process in a Linguistically Diverse Society. Dement. Geriatr. Cogn. Disord. 2020, 49, 355–364. [Google Scholar] [CrossRef] [PubMed]
  56. Verma, M.; Tripathi, M.; Nehra, A.; Paplikar, A.; Varghese, F.; Alladi, S.; Narayanan, J.; Dhaliwal, R.S.; Sharma, M.; Saroja, A.O.; et al. Validation of ICMR Neurocognitive Toolbox for Dementia in the Linguistically Diverse Context of India. Front. Neurol. 2021, 12, 1828. Available online: https://www.frontiersin.org/article/10.3389/fneur.2021.661269 (accessed on 27 May 2022). [CrossRef] [PubMed]
  57. Tochel, C.; Smith, M.; Baldwin, H.; Gustavsson, A.; Ly, A.; Bexelius, C.; Nelson, M.; Bintener, C.; Fantoni, E.; Garre-Olmo, J.; et al. What outcomes are important to patients with mild cognitive impairment or Alzheimer’s disease, their caregivers, and health-care professionals? a systematic review. Alzheimer’s Dement. Diagn. Assess. Dis. Monit. 2019, 11, 231–247. [Google Scholar] [CrossRef] [PubMed]
  58. Reilly, S.T.; Harding, A.J.; Morbey, H.; Ahmed, F.; Williamson, P.R.; Swarbrick, C.; Leroi, I.; Davies, L.; Reeves, D.; Holland, F.; et al. What is important to people with dementia living at home? a set of core outcome items for use in the evaluation of non-pharmacological community-based health and social care interventions. Age Ageing 2020, 49, 664–671. [Google Scholar] [CrossRef] [Green Version]
Brainsci 12 00922 g001 550
Figure 1. Types of outcome measures used in dementia and MCI clinical trials.
Figure 1. Types of outcome measures used in dementia and MCI clinical trials.
Brainsci 12 00922 g001
Table
Table 1. Summary table of some outcome measures used in AD and MCI trials.
Table 1. Summary table of some outcome measures used in AD and MCI trials.
Outcome MeasureDescriptionStage of DementiaBenefitsDeficiencies
Mini-Mental State Examination (MMSE) Most commonly used cognitive measure in trials; scored on 30 pointsMild to moderate Easy to administer, rapid Floor and ceiling effects; Stand-alone MMSE may not be sufficient to predict MCI progression; limited data on psychometric properties; affected by level of education
Alzheimer’s Disease Assessment Scale-Cognitive subscale (ADAS-COG) More detailed than MMSEMild to moderateDetailed evaluation, may be more sensitive to subtle changesTedious, does not sufficiently assess attention, planning, working memory, executive function
Montreal Cognitive Assessment (MoCA) 30-point screening tool requiring around 10 minAll stagesExtends the cognitive evaluation offered by MMSE by including working memory, orientation, immediate and delayed memory, executive function and visuospatial abilitiesRequires more time to administer than MMSE
Saint Louis University Mental State (SLUMS)11-item scale, scores < 27 indicate cognitive impairmentAll stagesAssesses executive functionRequires more time to administer than MMSE
Alzheimer Disease Cooperative Study Activities of Daily Living scale (ADCS-ADL) Assesses basic and instrumental ADLs All stages For patients with MMSE 0–15, the Alzheimer Disease Cooperative Study ADL-sev scale was developed; Also adapted for patients with MCI
Disability Assessment for Dementia (DAD) 46 items via a questionnaire administered to the caregiver; Assesses basic and instrumental ADLs All stagesGood intra- and interrater reliability
Quality of Life in Alzheimer’s disease (QOL-AD) 13-item scale, with scores ranging from 13 to 52 Mild to moderate dementiaExcellent internal consistency and reliabilityCannot be applied for individuals with MMSE < 10
Quality of life in late-stage dementia (QUALID) 11-item scale, with total scores ranging from 11 to 55; scored over one week All stagesGood internal consistency, inter-rater reliability, and test-retest reliabilitySensitive to effects of medications, such as neuroleptics
Quality of Life for People with Dementia (QUALIDEM) Developed for patients in residential settingsAll stages21/40 items suitable for patients with very advanced dementia
Neuropsychiatric Inventory (NPI)Covers 12 neuropsychiatric symptoms over one month retrospectivelyMild to moderate dementiaGood validity and reliabilityMay be affected by recall bias
Clinician’s Interview-Based Impression of Change plus caregiver interview (CIBIC-Plus) Seven-point rating scale which measures global functioning All stages CIBIC-Plus may compare favourably to goal attainment scaling (GAS) for clinical meaningfulness May have subjective bias
Clinical Dementia Rating (CDR) CDR is a global assessment tool with a five-point scale, which provides global and Sum of Boxes (SOB) scores in six cognitive domains All stagesRecommended as a core outcome for global functionDoes not contain measures of behavioral issues
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Garg, D.; Gupta, A.; Agarwal, A.; Mishra, B.; Srivastava, M.V.P.; Basheer, A.; Vishnu, V.Y. Latest Trends in Outcome Measures in Dementia and Mild Cognitive Impairment Trials. Brain Sci. 2022, 12, 922. https://doi.org/10.3390/brainsci12070922

AMA Style

Garg D, Gupta A, Agarwal A, Mishra B, Srivastava MVP, Basheer A, Vishnu VY. Latest Trends in Outcome Measures in Dementia and Mild Cognitive Impairment Trials. Brain Sciences. 2022; 12(7):922. https://doi.org/10.3390/brainsci12070922

Chicago/Turabian Style

Garg, Divyani, Anu Gupta, Ayush Agarwal, Biswamohan Mishra, Madakasira Vasantha Padma Srivastava, Aneesh Basheer, and Venugopalan Y. Vishnu. 2022. "Latest Trends in Outcome Measures in Dementia and Mild Cognitive Impairment Trials" Brain Sciences 12, no. 7: 922. https://doi.org/10.3390/brainsci12070922

APA Style

Garg, D., Gupta, A., Agarwal, A., Mishra, B., Srivastava, M. V. P., Basheer, A., & Vishnu, V. Y. (2022). Latest Trends in Outcome Measures in Dementia and Mild Cognitive Impairment Trials. Brain Sciences, 12(7), 922. https://doi.org/10.3390/brainsci12070922

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop