Next Article in Journal
Reversed Mirror Therapy (REMIT) after Stroke—A Proof-of-Concept Study
Previous Article in Journal
Towards a Neurophenomenological Understanding of Self-Disorder in Schizophrenia Spectrum Disorders: A Systematic Review and Synthesis of Anatomical, Physiological, and Neurocognitive Findings
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Circulating Serum VEGF, IGF-1 and MMP-9 and Expression of Their Genes as Potential Prognostic Markers of Recovery in Post-Stroke Rehabilitation—A Prospective Observational Study

1
Department of Neurological Rehabilitation, Medical University of Lodz, Milionowa 14, 93-113 Lodz, Poland
2
Biohazard Prevention Centre, Faculty of Biology and Environmental Protection, University of Lodz, Pomorska 141/143, 90-236 Lodz, Poland
3
Department of Pharmacology and Toxicology, Medical University of Lodz, Żeligowskiego 7/9, 90-752 Lodz, Poland
4
Department of Physiology and Pharmacology “Vittorio Erspamer”, Sapienza University, P. le Aldo Moro 5, 00185 Rome, Italy
5
Department of General Biochemistry, Faculty of Biology and Environmental Protection, University of Lodz, Pomorska 141/143, 90-236 Lodz, Poland
*
Authors to whom correspondence should be addressed.
Brain Sci. 2023, 13(6), 846; https://doi.org/10.3390/brainsci13060846
Submission received: 3 April 2023 / Revised: 19 May 2023 / Accepted: 21 May 2023 / Published: 23 May 2023

Abstract

:
The key period in post-stroke recovery is the first three months due to the high activity of spontaneous and therapeutic-induced processes related to neuroplasticity, angiogenesis and reperfusion. Therefore, the present study examines the expression of VEGF, IGF-1 and MMP-9 proteins and their genes to identify biomarkers that can prognose brain repair ability and thus estimate the outcome of stroke. It also identifies possible associations with clinical scales, including cognitive assessment and depression scales. The study group comprised 32 patients with moderate ischemic stroke severity, three to four weeks after incident. The results obtained after three-week hospitalization indicate a statistically significant change in clinical parameter estimations, as well as in MMP9 and VEGF protein and mRNA expression, over the rehabilitation process. Our findings indicate that combined MMP9 protein and mRNA expression might be a useful biomarker for cognitive improvement in post-stroke patients, demonstrating 87% sensitivity and 71% specificity (p < 0.0001).

1. Introduction

Stroke still remains a principal cause of adult disability and mortality, and it is regarded as a priority for both the World Health Organization (WHO) and the United Nations to decrease the burden of non-communicable diseases [1,2]. The traditional definition of stroke was mainly clinical, comprising a sudden loss of focal neurological function due to infarction or hemorrhage lasting longer than 24 h; however, the updated definition integrates clinical and tissue criteria and is based on neuropathological, neuroimaging and/or clinical indications of permanent injury of brain, spinal cord or retinal cell [3]. Despite the use of increasingly effective early treatment strategies, such as intravenous tissue plasminogen activator (tPA) or endovascular thrombectomy (EVT) [4], post-stroke recovery is often incomplete and recovery rates of neurological function differ.
Stroke rehabilitation depends on both spontaneous and therapeutic-induced processes of recovery, with important roles played by, inter alia, the extent of a stroke, regeneration, angiogenesis, neuroplasticity, pharmacotherapy, and neurorehabilitation. The first three months are the most important period in post-stroke rehabilitation, during which time spontaneous and therapeutic-induced processes of recovery are most effective. From a clinical point of view, it is important to know which patient may demonstrate the greatest potential improvement in both cognitive and functional performance several months later.
Therefore, there is a need to identify biomarkers that can be used as prognostic tools to estimate the brain repair ability in a particular patient and also as predictive tools for monitoring the response to individually tailored rehabilitation interventions [5]. Consequently, the findings may support the development of new algorithms to estimate the outcome of stroke patients and subsequent treatment. For example, upon admission to hospital, patients could be tested against a panel of biochemical and genetic laboratory tests, which are aimed at genes and proteins involved in recovery processes such as neuroplasticity [6].
Post-stroke recovery mechanisms remain poorly understood; however, circulating molecules related to neuroplasticity, such as MMP-9, VEGF and IGF-1, can influence the outcome when accompanied by extensive rehabilitation [6]. Molecular pathways involved in brain repair after stroke are supposed to vary over time and those related to neuroplasticity become relevant later in the subacute phase [7]. Furthermore, we hypothesize that serum molecular differences between patients might provide a possible neurophysiological and biochemical basis for differences in treatment responsiveness. Studies on animal models indicate that changes in circulating proteins and expression of their genes correlate with post-stroke recovery, and this has been supported to some degree by clinical research studies [8,9,10,11,12].
It is generally believed that stroke recovery may be associated with the genes and proteins known to play significant roles in neuroplasticity. For example, matrix metalloproteinase 9 (MMP-9), a zinc-dependent endopeptidase, is associated with both physiological and pathological tissue reorganization or remodeling, including neovascularization. Its levels are upregulated in post-stroke patients, which correlates with a poorer outcome [13,14,15]. It has been shown that in post-stroke patients the concentration of MMP9 is significantly higher compared to healthy volunteers [16,17]. Furthermore, MMP-9 levels appear to have predictive value for hemorrhagic transformation, with the highest level of MMP-9 recorded six hours after stroke onset, and an increased level of MMP-9 being associated with infarct exacerbation and hemorrhage conversion [18]. Furthermore, in cases of acute stroke, the serum MMP-9 level was found to predict post-stroke cognitive impairment three months later [19].
Vascular endothelial growth factor (VEGF) is a well-known strong angiogenic factor that was first described as specific for endothelial cell mitogen [20]. VEGF has a well-documented role in the nervous system, where it promotes neuroplasticity, including nerve repair, neurogenesis, and glial growth [21]. During ischemic stroke, VEGF gene expression is upregulated as a result of hypoxia; the protein usually binds to the VEGF second receptor, which is a crucial mediator of angiogenesis and increased permeability. The second receptor may stimulate the production of matrix metalloproteinases (MMPs) and induce the release of endothelial growth factors [22]. Increases in circulating VEGF have been shown to aid neurological recovery in animal models of acute stroke [9,10]. An animal model study showed that treadmill exercise may promote post-stroke recovery by regulating the caveolin-1/VEGF pathway in the ischemic zone [23]. In addition, higher exercise intensity has been found to be related to increased VEGF levels in post-stroke rehabilitation [24]. VEGF expression was found to be significantly elevated for the 90-day period after stroke onset, and this may have a significant influence on the functional outcome for cardioembolic infarction origin stroke with higher VEGF values predicting poorer outcomes [25].
The insulin-like growth factor 1 (IGF-1) protein is structurally and functionally similar to insulin and as a neurotrophic hormone, it plays a crucial role in the development and maturation of the central nervous system. IGF-1 serum level has been found to be influenced by stroke, with a decrease in IGF-1 in acute stroke being related to a positive outcome [26]. However, IGF-1 only has a modest influence on long-term post-stroke recovery persisting after seven years, as reflected in the mRS score distributions at all time points and is not associated with mortality or recurrent stroke [27]. It has also been confirmed that IGF-1 serum level plays a key role in short-term outcomes (three months), whereas in the longer term, e.g., two years, the associations are weakened and attenuated by other factors, such as diabetes, smoking, hypertension and hyperlipidemia [28].
Hence, the aim of the present study was to evaluate the prognostic value of proteins VEGF, IGF-1 and MMP-9 and the expression of their genes as markers of recovery in stroke patients. The expression data was compared with patient demographics and various clinical scales, including cognitive assessment and depression scales, with the goal of identifying a recovery biomarker associated with individual functional domains. The study was performed in the early subacute phase of stroke, i.e., from seven days to three months, according to the Consensus Statements from the Stroke Recovery and Rehabilitation Roundtable [7].

2. Materials and Methods

2.1. Subject Presentation

A series of patients was prospectively collected for the study Figure 1; all had been admitted to the Department of Neurological Rehabilitation Medical University of Lodz, Poland, from January 2022 to January 2023.
The minimum number of patients needed for enrollment was 28, based on sample size estimation with the use of G*Power 3.1.9.2 software [29]. The one-way repeated-measure analysis of variance (ANOVA) was used to demonstrate the medium effect size (Cohen’s f = 0.25) with a statistical power of 0.8, statistical significance of 0.05, and correlation between repeated measures of 0.6 (as indicated in preliminary results for improvement in physical functioning and changes in MMP-9 levels).
A total of 32 patients were enrolled, all of whom with moderate severity ischemic stroke, three to four weeks after the incident. The mean age was 68.3 (±9.1), the other clinical and demographic characteristics of the patients are shown in Table 1.
The diagnosis of cerebrovascular event was based on clinical and neuroimaging criteria according to the updated definition of stroke [3]. Exclusion criteria included the following: intracerebral hemorrhage, chronic or significant acute inflammatory factors, neurological illness other than ischemic stroke, severe cardio-vascular disease such as myocardial infarction within less than 30 days or unstable angina pectoris, decompensated metabolic or endocrine diseases, or severe general condition with respiratory failure. The patients underwent a rehabilitation program provided by a physiotherapist, every day for a period of three weeks with weekend gaps. The post-stroke neurorehabilitation program consisted of a 60 min neurophysiological session in the morning (30 min of shaping techniques—based on daily living activities and 30 min of repetitive task practice or balance), 15 min of psychotherapy and 30 min aerobic training (two or three times a day for 10 min at 60 min intervals). However, training time was individually modified depending on physical condition and improvement in function, if necessary.

2.2. Clinical Parameter Determination

2.2.1. Cognitive Assessment

Cognitive assessment was performed using the Mini Mental State Examination (MMSE) and Montreal Cognitive Assessment (MoCA), depending on the patient’s age. The MMSE or Folstein test is a 30-point questionnaire most widely used in screening for cognitive impairment. It requires vocal responses, following verbal and written commands, writing a sentence spontaneously and coping with a complex figure, and provides information about orientation, memory, attention, learning, calculation, delayed recall, and construction [30]. The MMSE is regarded to have acceptable validity as a screening instrument for vascular dementia in post-stroke patients with a sensitivity of 0.96 and specificity of 0.83, assuming a cut-off score of 23/24 [31]. The MoCA is a one-page, 30-point test which evaluates the following domains: short-term memory, visuospatial abilities, executive functions, attention and working memory, language, and orientation to time and space. The threshold for normal cognitive function is ≥26. The tool is widely used as a brief cognitive tool in both the acute/subacute and chronic post-stroke terms [32], and a MoCA score of ≤23 allows prediction of post-stroke cognitive impairment and functional dependence after stroke [33,34].

2.2.2. Depressive Symptom Assessment

Depressive symptoms were assessed using The Geriatric Depression Scale (GDS) and Beck’s Depression Inventory (BDI), depending on the patient’s age. The GDS is a practical, self-reporting screening tool used in clinical settings to evaluate depression in older adults. The present study used the GDS short version; this version has adequate reliability and validity compared to the long version [35]. It consists of 15 “Yes/No” questions. Final scores of 0–4 are considered normal (depending on age, education and complaints), and scores of 5 and above indicate depression: 5–8 being mild, 9–11 being moderate and 12–15 indicating severe depressive symptoms [36].
The BDI is a widely used 21-question self-reporting inventory measuring the severity of depression in adolescents and adults. Each question has a set of at least four possible answer choices reflecting the intensity of symptoms with a value of 0 to 3 assigned for each answer. The total score determines the severity of depression as follows: 0–9 indicates minimal depression, 10–18 indicates mild depression, 19–29 indicates moderate depression, and 30–63 indicates severe depression [37,38].

2.2.3. Physical and Motor Condition Assessment

The National Institutes of Health Stroke Scale (NIHSS) is recommended as a valid tool to assess stroke severity; it is also considered to be a strong predictor of outcomes after stroke [39]. The NIHSS includes the following domains: level of consciousness, eye movements, integrity of visual fields, facial movements, arm and leg muscle strength, sensation, coordination, language, speech and neglect. Each domain is scored from 0 to 2, 0 to 3, or 0 to 4. The total score ranges from 0 to 42, and a higher score is associated with more severe stroke [40].
The Barthel Index (BI) is a measure of independence in activities of daily living (ADL) The 10 items included in the BI pay attention to self-care and mobility. Total possible scores range from 0–20, with lower scores indicating increased disability, and a score of 17 and above representing normal activity [41].
The Modified Rankin Scale is a global outcome rating scale that measures independence rather than the result of specific tasks; it also reflects both mental and physical adaptations to neurological deficit. The scale consists of six grades (from 0 to 5), with 0 indicating no symptoms and 5 corresponding to severe disability [42].

2.2.4. Blood Sample Collection

Whole blood samples were collected between 7 am and 9 am, under dietary fasting conditions. Study material was taken twice: before therapy and in the fourth week of hospitalization in the study group, and once prior to treatment in control. The research material was placed into CPDA1 containing tubes (Sarstedt, Nümbrecht, Germany). Immediately after collection, part of the sample was frozen at −80 °C to measure the mRNA expression. Plated pour plasma was obtained by differential centrifugation (15 min, 1500× g, 25 °C). All samples were collected and stored using protocol.

2.3. Determination of IGF1, VEGF and MMP-9 Level in Plasma

Measurement of MMP-9, VEGF, and IGF1 protein content was conducted using a Human IGF1 SimpleStep ELISA® Kit (Abcam, Cambridge, UK), Human VEGF SimpleStep ELISA® Kit (Abcam, Cambridge, UK), and Human MMP-9 ELISA Kit (Invitrogen, Waltham, MA, USA), respectively. All procedures were conducted in accordance with the manufacturer’s protocol. The endpoint reading (color intensity) of all analytes was measured at 450 nm and protein concentration was calculated from the standard curve.

2.4. Determination of MMP-9, VEGF-A and IGF1 Expression in Whole Blood Samples

RNA isolation and purification was performed using a DNA/RNA Extracol Kit (EURx, Gdansk, Poland). RNA purity and quantity was first estimated using a Synergy HTX Multi-Mode Microplate Reader, equipped with a Take3 Micro-Volume Plate (BioTek Instruments, Inc., Winooski, VT, USA). Following this, the diluted RNA samples were transcribed into cDNA using a High-Capacity cDNA Reverse Transcription Kit (Applied Biosystems, Waltham, MA). All steps were performed in accordance with the manufacturer’s protocol. Gene expression was determined by RealTime PCR using a TaqMan Universal Master Mix II (Life Technologies, Carlsbad, CA, USA) in a CFX96 real-time PCR system (BioRad Laboratories, Hercules, CA, USA). Expression levels were obtained using TaqMan probes (Hs01547656_m1 for human IGF-1, Hs00900055_m1 for human VEGF-A, Hs00957562_m1 for human MMP-9, and for endogenous control (GAPDH)—Hs99999905_m1) (Life Technologies, Carlsbad, CA, USA). All procedures followed the manufacturer’s protocol.

2.5. Data Analysis

The dataset presented 210 (12.6%) missing data points. These were imputed with the use of multiple imputation by chained equation procedure under the missing at random assumption before the data analyses were performed. Descriptive statistics for the continuous and ordinal variables included arithmetic mean with standard deviation (for the sample characterization) or standard error of mean (for the purpose of estimation), whereas dichotomous variables were expressed as numbers and frequencies. mRNA gene expression was presented as –ΔCt to make the distribution closer to normal. One-way repeated-measure ANOVA was used to compare post- and pre-rehabilitation outcomes. Additional covariates were optionally added linearly linked with the outcome in the ANOVA models. The other general linear models were used for selected purposes. As distribution normality was violated in a few instances, respective non-parametric tests were also performed; these are reported in Supplementary Materials. Multivariate logistic regression modeling was performed to estimate the predictive values of the selected pre-rehabilitation biochemical parameters in forecasting post-rehabilitation psychological outcomes. Internal validation of the proposed models was performed using the 10-fold cross-validation technique. The modeling results were additionally presented with the use of receiver operating characteristic (ROC) curves. p-values below 0.05 were considered statistically significant. The analysis was performed using STATISTICA 13.3 Software (StatSoft; Tulsa, OK, USA).

3. Results

After the three-week rehabilitation program, both the physical and psychological condition of the patients improved significantly as compared to pre-rehabilitation time. Details are presented in Table 2.
Among the tested proteins and mRNAs, significant changes in plasma MMP-9 and VEGF protein content were observed over the rehabilitation process. MMP-9 protein concentrations fell from 112.2 ± 9.7 pg/mL before rehabilitation to 85.4 ± 10.6 pg/mL afterwards, i.e., by 24%, whereas VEGF protein level increased from 44.8 ± 25.9 pg/mL to 66.3 ± 50.3 pg/mL, i.e., by 48% (Figure 2).
The reported changes in MMP-9 and VEGF protein plasma concentration observed over the rehabilitation process were correlated with improvements in cognitive function and amelioration of depressive symptoms. However, although MMP-9 decreased during rehabilitation, its rise predicted better cognitive outcome and although VEGF increased during the rehabilitation, its fall was associated with less depressive symptoms. Moreover, although a negligible change in VEGF-A mRNA expression was observed over the rehabilitation process, it significantly correlated with an amelioration of depressive symptoms (Table 3).
To gain a predictive insight into the improvement of psychological function over the rehabilitation process, the pre-rehabilitation parameters were compared with psychological improvement indices. Pre-rehabilitation mRNA expression of MMP-9 was found to be inversely linked to cognitive improvement, whereas baseline VEGF protein level predicted improvement in depression symptoms.
The MMP-9 and VEGF protein levels fluctuated during the rehabilitation process, and these proteins together with respective mRNAs could predict some improvements in cognitive function and depressive symptoms over the rehabilitation process, respectively (Table 3 and Table 4). Therefore, pre-rehabilitation MMP-9 protein and mRNA levels were used to model the odds for cognitive improvement after rehabilitation, whereas VEGF protein and mRNA levels were used to model the chance of amelioration of depressive symptoms. Both MMP-9 and VEGF were suitable for inclusion into the model at both the mRNA and protein levels, as the correlations between respective mRNA and protein were negligible: r = 0.20, p = 0.2661 for MMP-9 and r = −0.00, p = 0.9794 for VEGF.
A multivariate logistic regression analysis with internal validation was applied. An improvement in the psychometric indices by at least three points was the modeled outcome. Both models were adjusted with the additional inclusion of basic sociodemographic parameters (age and sex) as covariates to increase patient-specific relevance. It was found that pre-rehabilitation MMP-9 protein and mRNA levels could predict post-rehabilitation cognitive improvement. The model fitted well to the data and was not overfitted; sensitivity and specificity measures were promising, even in the validation set. In addition, the Hosmer–Lemeshow test results were insignificant, indicating that it could work well in all population subgroups (Table 5 and Figure 3A).
On the other hand, in multivariate logistic regression analysis, VEGF protein and mRNA levels failed to predict any amelioration in depressive symptoms, as indicated by insignificant predictors and unsatisfactory model indices (Table 6 and Figure 3B).

4. Discussion

One of the most difficult clinical challenges encountered during post-stroke rehabilitation is the unpredictable, multisymptomatic course of the condition. However, the results of treatment in various medical areas can be better predicted by including biomarkers. Our present findings suggest that the combined protein and mRNA expression of MMP-9 may be a useful biomarker for cognitive improvement in post-stroke patients, assuming an increase of three points on the MMSE scale, and that this test offers 87% sensitivity and 71% specificity (Figure 3A).
Post-stroke cognitive impairment and dementia (PSCID) is one of the most important problems influencing functional recovery and quality of life and can be a major source of morbidity and mortality after stroke [43]. Although a number of biomarkers related to neuroimaging are ready to be used in clinical trials, they are envisaged for implementation in motor and language functional domains; there are still no neuroimaging biomarkers intended for somatosensory problems and cognition [44]. Hence, there is a need for biomarkers of cognitive recovery in the biochemical and molecular fields.
In the classic sense, as defined by Naylor, a biomarker should be impartially measured and evaluated as a marker of both physiological and pathological processes, and also as an indicator of response to a therapeutic intervention [45]. Furthermore, a potentially useful biomarker must fulfil several important criteria, including sensitivity, specificity, repeatability of measurements, safety of collection, cost-effectiveness and quickness [46].
Our findings indicate that MMP-9 has predictive potential as a useful biomarker of stroke recovery in the field of cognition. Although MMP-9 involved in the degradation of the extracellular matrixes is considered to cause cerebral damage after stroke, studies confirm that MMP-9 also plays a role in synaptic plasticity and synapse remodeling [47]. In humans, the MMP family consists of 23 members, including secreted and transmembrane proteins. In the central nervous system, MMPs are synthesized and secreted by neurons and glia in an inactive pro-enzyme form (zymogen) and are later converted into active enzymes [48]. The levels of both the pro-form and active form of MMP-9 were found to increase after the induction of late-phase long-term potentiation (LTP), i.e., persistent strengthening of synapses in the CA1 area of the hippocampus, which is the physiological basis of synaptic plasticity [49]. Studies also indicate that MMP-9 is a crucial dendritic spine shape modulator [50]. Importantly, MMP-9 has been found to have a regulatory role in synaptogenesis, myelination and axonal outgrowth [51].
MMP-9 also plays a key role in the transformation of pro-BDNF into the mature form of BDNF in the hippocampus [52].
Interestingly, a study examined hypoxia response element (HRE)-regulated MMP-9 and its effect on glial scars and neurogenesis in delayed ischemic stroke, seven days after transient middle cerebral artery occlusion (tMCAO) based on a rat model. HRE-MMP-9 improved neurological outcomes at three and five weeks after tMCAO, reduced ischaemia-induced brain atrophy, and degraded glial scars (p < 0.05). Furthermore, HRE-MMP-9 increased the number of microvessels in the peri-infarct area (p < 0.001) [8]. These findings confirm that MMP-9 has a beneficial effect in the subacute phase of stroke, suggesting that its influence may be related to neuroplasticity and angiogenesis. This statement is consistent with our present findings suggesting that the combined protein and mRNA expression of MMP-9 may be a useful biomarker for cognitive improvement in the subacute phase.
Conversely, elevated MMP-9 serum concentrations compared to controls have been noted in the hyperacute period of stroke; however, MMP-9 also appears to play a role in damaging the blood-brain barrier and regulating inflammation [53,54]. This, in turn, is related to poor outcome [14,15]. Furthermore, those pathological processes probably explain recent results indicating that MMP-9 levels measured in the acute phase of stroke have predictive value for post-stroke cognitive impairment (PSCI) three months after stroke [19]. Furthermore, studies suggest that at high levels, there are reports that pro-inflammatory and atherosclerosis signaling factors, including MMP-9, were associated with a significantly higher risk of adverse clinical outcomes.
Guo et al. propose that MMP-9, S100A8/A9, high sensitivity C-reactive protein (hsCRP) and growth differentiation factor-15 (GDF-15) are effective predictors of prognosis at three months after ischemic stroke and may be useful for risk stratification among ischemic stroke patients [55]. Our findings indicate that MMP-9 serum levels decreased by 24% (p = 0.032) in stroke patients after a three-week course of rehabilitation, both as raw data and following adjustment for sex and age. The reported changes in MMP-9 occurring during rehabilitation were found to be correlated with an improvement in cognitive function; more specifically, its rise predicted better cognitive outcomes, which may indicate the presence of neuroplasticity promoted by MMP-9.
Zhong et al. demonstrated a linear association between MMP-9 levels and cognitive impairment (p < 0.001 for linearity). They indicate that an increased MMP-9 level in the acute phase of ischemic stroke was associated with three-month cognitive impairment, independent of adopted risk factors; this is consistent with our present data indicating that the pre-rehabilitation level of MMP-9 gene expression was negatively correlated with an improvement in cognition (Table 5) [56]. Interestingly, serum MMP-9 level has also been proposed as a significant factor for diagnosing cognitive impairment due to cerebral small vessel disease, which suggests that MMP-9 may be associated with vascular dementia [57].
Our present study indicates that pre-rehabilitation MMP-9 protein and mRNA levels could forecast a recovery in cognitive function with 87% sensitivity and 71% specificity (p < 0.0001) (Figure 3A), thus allowing further treatment to be planned in line with the concept of personalized medicine. By comparison, serum MMP-9 has also been found to predict hemorrhagic transformation after thrombolytic therapy in acute ischemic stroke with 92% sensitivity and 74% specificity [58]. Therefore, MMP-9 protein level appears to be a promising candidate for inclusion in future multicenter confirmatory studies as a possible marker of cognitive recovery. This conclusion was also confirmed by multivariate logistic regression analysis with internal validation, with the modeled outcome being assumed as an improvement in the psychometric indices by at least three points (Table 5).
However, any assessment of cognitive problems following stroke is complicated by the coexistence of stroke-related impairments, such as visual disturbance or paresis of the dominant hand and dysphasia; as such, it should be performed by qualified specialists. Despite these limitations, data indicates that MMSE and MoCA could be used to accurately screen for impairments at all levels of severity, and may represent the best option of screening for vascular dementia [59].
In our study, cognitive assessment was performed twice: once at the baseline and once again three weeks later, i.e., after rehabilitation MMSE and MoCA are considered to be test-retest reliable factors for monitoring cognitive function in dementia patients [60]. However, gains in neuropsychological test performance scores have been attributed to practice effects [61] and a short-term practice effect (even small) should be considered when interpreting changes in scores [62]. As reported by Feenay, in a study of 130 participants who underwent two cognitive assessments between two and four months apart, the standard error of measurement (SEM) of the MMSE was found to be one point, leading to an minimum detectable change (MDC) of three points [63]. In our study, the re-test was performed only three weeks later; therefore, the practice effect was estimated at two points for the MMSE, which was subtracted from the obtained result.
Our findings do not confirm previous reports indicating an association between serum MMP-9 levels and post-stroke depression (Table 4). A recent study suggests that elevated serum MMP-9 levels in the acute phase of ischemic stroke were related to increased risk of post-stroke depression within three months, suggesting that MMP-9 has a prognostic role [64]. However, as blood samples were collected in the subacute phase of stroke in the present study, further research is needed in this phase of stroke.
VEGF is another molecule involved with neuronal survival and angiogenesis. Its main role is associated with its pro-angiogenic activities, but it has also demonstrated neurotrophic and neuroprotective effects on both on the central and peripheral nervous systems [65,66]. Neurogenesis is achieved by stimulating the endothelium to release neurotrophic factors, as shown in Schwann cells after hypoxia in the peripheral nervous system [67]. It has been also demonstrated that VEGF generated by ependymal cells triggers and strengthens neuronal precursor proliferation in the central nervous system [68]. In addition, these neurogenic effects are believed to be related to an increase in the transdifferentiation of striatal astrocytes into new mature neurons [69]. Serum VEGF level increases immediately after stroke, peaks at three days post-stroke (p < 0.001) [70] and remains significantly higher than controls for 90 days after stroke onset [25]. In our study, a significant increase in VEGF serum level (by 48%) was observed after the three-week rehabilitation course (p = 0.011; Figure 2B).
VEGF level has been found to correlate with depression, and data suggests that polymorphisms in the VEGF pathway are associated with the severity of depressive symptoms and correspond with higher plasma concentrations of VEGF (p-value = 0.006) [71]. Interestingly, our present findings suggest that a decrease in VEGF protein plasma is correlated with an amelioration of depressive symptoms in post-stroke patients (p = 0.043; Table 3). Indirectly, this seems to be consistent with previous data. A series of studies indicates that increased VEGF serum level may be a strong predictor of unfavorable outcomes [72,73]. Notably, other studies have analyzed VEGF level in relation to various types of strokes. Zhang et al. [74] report that increased serum VEGF may be used as an independent predictor of a poor outcome in minor ischemic stroke. Matsuo et al. [25] divided stroke patients into four groups depending on subtype of ischemic stroke: atherothrombotic infarction (ATBI), lacunar infarction (LAC), cardioembolic infarction (CE) and other types (OT). After adjustment for possible confounding factors, VEGF plasma level was an independent predictor of unfavorable functional outcome only in CE patients. It can be concluded that many variables affect VEGF values, and further well-planned and detailed research is needed.
IGF-1 is a strong neuroendocrine regulator of CNS development, maturation and cellular neuroplasticity. Its potential mechanisms of action include modulation of glutamatergic receptor subunits, potentiation of glutamatergic transmission or alterations in calcium channel conductance [75]. It has been demonstrated that IGF-1 levels in circulation and the cerebrospinal fluid (CSF) decrease with age and a positive relationship exists between serum and CSF IGF-1 levels [76]. Furthermore, age-related reductions in IGF-1 correspond to increases in risk of hypertension, diabetes, and other cardiovascular diseases including ischemic stroke [77,78,79]. In turn, patients experiencing acute ischemic stroke demonstrate lower IGF-1 expression in comparison with healthy subjects with the amount decreasing with disease aggravation [80]. Although these literature data suggest that IGF-1 has prognostic value, our present findings indicate that IGF-1 plasma level was not affected by three-week rehabilitation following stroke (Figure 2C). IGF1 mRNA expression was only detectable in nine patients, and this parameter was excluded from further analyses.
Previous studies have also noted that changes in IGF-1 serum level are not statistically significant for assessing stroke recovery. Edinger et al. report that, based on a multivariate analysis including IGFBP-3 (insulin-like growth factor binding protein-3) IGF-1, age, thrombolysis and NIHSS, only low IGFBP-3 levels were associated with an unfavorable functional outcome (OR 7.2, 95%CI 1.8–29.0, p = 0.006). Additionally, after considering lesion volume, only IGFBP-3 levels were predictive of poor functional outcome (OR 7.2, 95%CI 1.5–35.5, p = 0.015) [81].
However, the scientific evidence regarding whether plasma levels of IGFBP-3 and IGF-1 are indicative of functional outcome remains inconsistent. Armbrust et al. report that low IGF-1 levels in the acute phase of stroke (day 8) were independently associated with a decreased risk of an unfavorable outcome [82]. Interestingly, higher levels of serum IGF-1 were observed in patients who demonstrated greater improvements in cognition after combined aerobic and cognition training, suggesting that IGF-1 may be involved in behaviorally induced plasticity [83]. In addition, higher exercise-induced levels of serum IGF-1 at the baseline were found to predict 40% of the variability in advance of fluid intelligence. It is noteworthy that only exercise-induced levels of IGF-1 were prognostic of this response and not resting levels [83]. Another study found that IGF-1 level was positively correlated with aerobic fitness in acute stroke patients: individuals with higher levels of IGF-1 demonstrated significant improvements in estimated aerobic fitness, which might be beneficial to stroke recovery [84].
The search for useful biomarkers for stroke recovery offers promise but requires further effort.

5. Limitations and Future Perspective

There are some limitations that should be included for data interpretation. The main limitation was the small number of stroke patients. However, it was still relatively large in comparison to previous studies analyzing a similar area of interest [14]. Moreover, another limitation was that this study included male and female participants. Our study focused on patients with moderate ischemic stroke severity; therefore, the results are characteristic to a defined level of disability, without long-term analyses.
Future research should aim to create a laboratory test panel as the use of multiple biomarkers covering several pathways has been found to improve prognostic ability. In addition, it would be worth analyzing the correlation between biomarkers and cognitive impairment not only regarding cognition as a whole, but in separation into the domains which constitute cognitive functioning. There is a clinical need for longitudinal studies of poststroke biomarkers, and very interesting research is in progress, the results of which we are waiting for [7,85,86,87,88].

6. Conclusions

There are no prognostic biomarkers to identify stroke recovery and to support decisions regarding stroke care and rehabilitation. In this prospective observational study, among the analyzed proteins and mRNAs, significant changes were observed in plasma MMP-9 and VEGF protein content over a 3-week follow-up time. Particularly, MMP-9 appears to be a promising molecule as a biomarker of cognitive recovery in the subacute phase, which should encourage a further multicenter confirmatory study.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/brainsci13060846/s1.

Author Contributions

Conceptualization, L.W., E.M. and N.C.; methodology, L.W. and M.S.K.; software, L.W.; validation, L.W., E.M. and J.S.; formal analysis, E.M., N.C. and M.S.K.; investigation, L.W.; resources, L.W.; data curation, L.W. and M.S.K.; writing—original draft preparation, L.W.; writing—review and editing, N.C. and M.S.K.; visualization, M.S.K.; supervision, E.M. and L.S.; project administration, L.W.; funding acquisition, E.M. All authors have read and agreed to the published version of the manuscript.

Funding

This study was supported by a grant from the Medical University of Lodz (Grant No. 503/6-127-05/503-51-001-19).

Institutional Review Board Statement

The study was conducted in accordance with the Declaration of Helsinki and was approved by the Institutional Review Board (or Ethics Committee) The present study was carried out according to the Declaration of Helsinki, and participation was voluntary and based on written informed consent from the participants. The study has been approved by the Research Ethics Committee of Medical University of Lodz (RNN/105/21/KE). ClinicalTrials.gov Identifier: NCT05750303.

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study. Written informed consent has been obtained from the patient(s) to publish this paper.

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Global Action Plan for the Prevention and Control of NCDs 2013–2020. Available online: https://www.who.int/publications/i/item/9789241506236 (accessed on 1 January 2020).
  2. Global, regional, and national burden of stroke, 1990-2016: A systematic analysis for the Global Burden of Disease Study 2016. Lancet Neurol. 2019, 18, 439–458. [CrossRef] [PubMed]
  3. Sacco, R.L.; Kasner, S.E.; Broderick, J.P.; Caplan, L.R.; Connors, J.J.; Culebras, A.; Elkind, M.S.; George, M.G.; Hamdan, A.D.; Higashida, R.T.; et al. An updated definition of stroke for the 21st century: A statement for healthcare professionals from the American Heart Association/American Stroke Association. Stroke 2013, 44, 2064–2089. [Google Scholar] [CrossRef]
  4. Powers, W.J.; Rabinstein, A.A.; Ackerson, T.; Adeoye, O.M.; Bambakidis, N.C.; Becker, K.; Biller, J.; Brown, M.; Demaerschalk, B.M.; Hoh, B.; et al. Guidelines for the Early Management of Patients With Acute Ischemic Stroke: 2019 Update to the 2018 Guidelines for the Early Management of Acute Ischemic Stroke: A Guideline for Healthcare Professionals From the American Heart Association/American Stroke Association. Stroke 2019, 50, e344–e418. [Google Scholar] [CrossRef] [PubMed]
  5. Cassidy, J.M.; Cramer, S.C. Spontaneous and Therapeutic-Induced Mechanisms of Functional Recovery After Stroke. Transl. Stroke Res. 2017, 8, 33–46. [Google Scholar] [CrossRef]
  6. Wlodarczyk, L.; Szelenberger, R.; Cichon, N.; Saluk-Bijak, J.; Bijak, M.; Miller, E. Biomarkers of Angiogenesis and Neuroplasticity as Promising Clinical Tools for Stroke Recovery Evaluation. Int. J. Mol. Sci. 2021, 22, 3949. [Google Scholar] [CrossRef]
  7. Bernhardt, J.; Hayward, K.S.; Kwakkel, G.; Ward, N.S.; Wolf, S.L.; Borschmann, K.; Krakauer, J.W.; Boyd, L.A.; Carmichael, S.T.; Corbett, D.; et al. Agreed definitions and a shared vision for new standards in stroke recovery research: The Stroke Recovery and Rehabilitation Roundtable taskforce. Int. J. Stroke 2017, 12, 444–450. [Google Scholar] [CrossRef]
  8. Cai, H.; Ma, Y.; Jiang, L.; Mu, Z.; Jiang, Z.; Chen, X.; Wang, Y.; Yang, G.-Y.; Zhang, Z. Hypoxia Response Element-Regulated MMP-9 Promotes Neurological Recovery via Glial Scar Degradation and Angiogenesis in Delayed Stroke. Mol. Ther. 2017, 25, 1448–1459. [Google Scholar] [CrossRef] [PubMed]
  9. Yang, J.P.; Liu, H.J.; Liu, X.F. VEGF promotes angiogenesis and functional recovery in stroke rats. J. Investig. Surg. 2010, 23, 149–155. [Google Scholar] [CrossRef] [PubMed]
  10. Li, C.; Zhang, B.; Zhu, Y.; Li, Y.; Liu, P.; Gao, B.; Tian, S.; Du, L.; Bai, Y. Post-stroke Constraint-induced Movement Therapy Increases Functional Recovery, Angiogenesis, and Neurogenesis with Enhanced Expression of HIF-1α and VEGF. Curr. Neurovasc. Res. 2017, 14, 368–377. [Google Scholar] [CrossRef]
  11. Zhu, W.; Fan, Y.; Frenzel, T.; Gasmi, M.; Bartus, R.T.; Young, W.L.; Yang, G.Y.; Chen, Y. Insulin growth factor-1 gene transfer enhances neurovascular remodeling and improves long-term stroke outcome in mice. Stroke 2008, 39, 1254–1261. [Google Scholar] [CrossRef]
  12. Okoreeh, A.K.; Bake, S.; Sohrabji, F. Astrocyte-specific insulin-like growth factor-1 gene transfer in aging female rats improves stroke outcomes. Glia 2017, 65, 1043–1058. [Google Scholar] [CrossRef]
  13. Turner, R.J.; Sharp, F.R. Implications of MMP9 for Blood Brain Barrier Disruption and Hemorrhagic Transformation Following Ischemic Stroke. Front. Cell Neurosci. 2016, 10, 56. [Google Scholar] [CrossRef] [PubMed]
  14. Abdelnaseer, M.M.; Elfauomy, N.M.; Esmail, E.H.; Kamal, M.M.; Elsawy, E.H. Matrix Metalloproteinase-9 and Recovery of Acute Ischemic Stroke. J. Stroke Cereb. Dis. 2017, 26, 733–740. [Google Scholar] [CrossRef] [PubMed]
  15. Zhong, C.; Yang, J.; Xu, T.; Xu, T.; Peng, Y.; Wang, A.; Wang, J.; Peng, H.; Li, Q.; Ju, Z.; et al. Serum matrix metalloproteinase-9 levels and prognosis of acute ischemic stroke. Neurology 2017, 89, 805–812. [Google Scholar] [CrossRef]
  16. Li, Y.; Han, X.; Luo, S.; Huang, H.; Huang, X.; Li, M.; Huang, Y.; Chen, Y.; Wu, Z. Predictive value of longitudinal changes of serum matrix metalloproteinase-9 and brain-derived neurotrophic factor in acute ischemic stroke. Front. Aging Neurosci. 2022, 14, 952038. [Google Scholar] [CrossRef]
  17. Kowalski, R.G.; Ledreux, A.; Violette, J.E.; Neumann, R.T.; Ornelas, D.; Yu, X.; Griffiths, S.G.; Lewis, S.; Nash, P.; Monte, A.A.; et al. Rapid Activation of Neuroinflammation in Stroke: Plasma and Extracellular Vesicles Obtained on a Mobile Stroke Unit. Stroke 2023, 54, e52–e57. [Google Scholar] [CrossRef]
  18. Mechtouff, L.; Bochaton, T.; Paccalet, A.; Crola Da Silva, C.; Buisson, M.; Amaz, C.; Bouin, M.; Derex, L.; Ong, E.; Berthezene, Y.; et al. Matrix Metalloproteinase-9 Relationship With Infarct Growth and Hemorrhagic Transformation in the Era of Thrombectomy. Front. Neurol. 2020, 11, 473. [Google Scholar] [CrossRef]
  19. Pu, M.; You, Y.; Wang, X. Predictive value of serum matrix metalloproteinase 9 combined with tissue inhibitor of metalloproteinase 1 for post-stroke cognitive impairment. J. Clin. Neurosci. 2022, 105, 103–108. [Google Scholar] [CrossRef]
  20. Ferrara, N.; Houck, K.; Jakeman, L.; Leung, D.W. Molecular and biological properties of the vascular endothelial growth factor family of proteins. Endocr. Rev. 1992, 13, 18–32. [Google Scholar] [CrossRef] [PubMed]
  21. Cichoń, N.; Bijak, M.; Czarny, P.; Miller, E.; Synowiec, E.; Sliwinski, T.; Saluk-Bijak, J. Increase in Blood Levels of Growth Factors Involved in the Neuroplasticity Process by Using an Extremely Low Frequency Electromagnetic Field in Post-stroke Patients. Front. Aging Neurosci. 2018, 10, 294. [Google Scholar] [CrossRef]
  22. Ferrara, N.; Kerbel, R.S. Angiogenesis as a therapeutic target. Nature 2005, 438, 967–974. [Google Scholar] [CrossRef]
  23. Pang, Q.; Zhang, H.; Chen, Z.; Wu, Y.; Bai, M.; Liu, Y.; Zhao, Y.; Tu, F.; Liu, C.; Chen, X. Role of caveolin-1/vascular endothelial growth factor pathway in basic fibroblast growth factor-induced angiogenesis and neurogenesis after treadmill training following focal cerebral ischemia in rats. Brain Res. 2017, 1663, 9–19. [Google Scholar] [CrossRef]
  24. Boyne, P.; Meyrose, C.; Westover, J.; Whitesel, D.; Hatter, K.; Reisman, D.S.; Carl, D.; Khoury, J.C.; Gerson, M.; Kissela, B.; et al. Effects of Exercise Intensity on Acute Circulating Molecular Responses Poststroke. Neurorehabil. Neural Repair 2020, 34, 222–234. [Google Scholar] [CrossRef] [PubMed]
  25. Matsuo, R.; Ago, T.; Kamouchi, M.; Kuroda, J.; Kuwashiro, T.; Hata, J.; Sugimori, H.; Fukuda, K.; Gotoh, S.; Makihara, N.; et al. Clinical significance of plasma VEGF value in ischemic stroke—Research for biomarkers in ischemic stroke (REBIOS) study. BMC Neurol. 2013, 13, 32. [Google Scholar] [CrossRef] [PubMed]
  26. Mattlage, A.E.; Rippee, M.A.; Sandt, J.; Billinger, S.A. Decrease in Insulin-Like Growth Factor-1 and Insulin-Like Growth Factor-1 Ratio in the First Week of Stroke Is Related to Positive Outcomes. J. Stroke Cereb. Dis. 2016, 25, 1800–1806. [Google Scholar] [CrossRef]
  27. Åberg, N.D.; Åberg, D.; Lagging, C.; Holmegaard, L.; Redfors, P.; Jood, K.; Nilsson, M.; Åberg, M.; Blomstrand, C.; Svensson, J.; et al. Association Between Levels of Serum Insulin-like Growth Factor I and Functional Recovery, Mortality, and Recurrent Stroke at a 7-year Follow-up. Exp. Clin. Endocrinol. Diabetes 2020, 128, 303–310. [Google Scholar] [CrossRef]
  28. Åberg, N.D.; Åberg, D.; Jood, K.; Nilsson, M.; Blomstrand, C.; Kuhn, H.G.; Svensson, J.; Jern, C.; Isgaard, J. Altered levels of circulating insulin-like growth factor I (IGF-I) following ischemic stroke are associated with outcome—A prospective observational study. BMC Neurol. 2018, 18, 106. [Google Scholar] [CrossRef]
  29. Faul, F.; Erdfelder, E.; Lang, A.G.; Buchner, A. G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav. Res. Methods 2007, 39, 175–191. [Google Scholar] [CrossRef]
  30. Folstein, M.F.; Folstein, S.E.; McHugh, P.R. “Mini-mental state”. A practical method for grading the cognitive state of patients for the clinician. J. Psychiatr. Res. 1975, 12, 189–198. [Google Scholar] [CrossRef] [PubMed]
  31. Bour, A.; Rasquin, S.; Boreas, A.; Limburg, M.; Verhey, F. How predictive is the MMSE for cognitive performance after stroke? J. Neurol. 2010, 257, 630–637. [Google Scholar] [CrossRef]
  32. Chiti, G.; Pantoni, L. Use of Montreal Cognitive Assessment in patients with stroke. Stroke 2014, 45, 3135–3140. [Google Scholar] [CrossRef] [PubMed]
  33. Abzhandadze, T.; Rafsten, L.; Lundgren Nilsson, Å.; Palstam, A.; Sunnerhagen, K.S. Very Early MoCA Can Predict Functional Dependence at 3 Months After Stroke: A Longitudinal, Cohort Study. Front. Neurol. 2019, 10, 1051. [Google Scholar] [CrossRef]
  34. Salvadori, E.; Cova, I.; Mele, F.; Pomati, S.; Pantoni, L. Prediction of post-stroke cognitive impairment by Montreal Cognitive Assessment (MoCA) performances in acute stroke: Comparison of three normative datasets. Aging Clin. Exp. Res. 2022, 34, 1855–1863. [Google Scholar] [CrossRef]
  35. Yesavage, J.A.; Sheikh, J.I. 9/Geriatric Depression Scale (GDS). Clin. Gerontol. 1986, 5, 165–173. [Google Scholar] [CrossRef]
  36. Marc, L.G.; Raue, P.J.; Bruce, M.L. Screening performance of the 15-item geriatric depression scale in a diverse elderly home care population. Am. J. Geriatr. Psychiatry 2008, 16, 914–921. [Google Scholar] [CrossRef] [PubMed]
  37. Steer, R.A.; Rissmiller, D.J.; Beck, A.T. Use of the Beck Depression Inventory-II with depressed geriatric inpatients. Behav. Res Ther. 2000, 38, 311–318. [Google Scholar] [CrossRef]
  38. Beck, A.T. Beck Depression Inventory–II. Available online: https://psycnet.apa.org/doiLanding?doi=10.1037%2Ft00742-000 (accessed on 1 January 2022).
  39. Scopus Preview. Available online: https://www.scopus.com/home.uri (accessed on 1 January 2022).
  40. Kwah, L.K.; Diong, J. National Institutes of Health Stroke Scale (NIHSS). J. Physiother. 2014, 60, 61. [Google Scholar] [CrossRef] [PubMed]
  41. Wade, D.T.; Collin, C. The Barthel ADL Index: A standard measure of physical disability? Int. Disabil. Stud. 1988, 10, 64–67. [Google Scholar] [CrossRef]
  42. Harrison, J.K.; McArthur, K.S.; Quinn, T.J. Assessment scales in stroke: Clinimetric and clinical considerations. Clin. Interv. Aging 2013, 8, 201–211. [Google Scholar] [CrossRef]
  43. Rost, N.S.; Brodtmann, A.; Pase, M.P.; van Veluw, S.J.; Biffi, A.; Duering, M.; Hinman, J.D.; Dichgans, M. Post-Stroke Cognitive Impairment and Dementia. Circ. Res. 2022, 130, 1252–1271. [Google Scholar] [CrossRef]
  44. Boyd, L.A.; Hayward, K.S.; Ward, N.S.; Stinear, C.M.; Rosso, C.; Fisher, R.J.; Carter, A.R.; Leff, A.P.; Copland, D.A.; Carey, L.M.; et al. Biomarkers of Stroke Recovery: Consensus-Based Core Recommendations from the Stroke Recovery and Rehabilitation Roundtable. Neurorehabil. Neural Repair 2017, 31, 864–876. [Google Scholar] [CrossRef] [PubMed]
  45. Naylor, S. Biomarkers: Current perspectives and future prospects. Expert Rev. Mol. Diagn. 2003, 3, 525–529. [Google Scholar] [CrossRef]
  46. Simpkins, A.N.; Janowski, M.; Oz, H.S.; Roberts, J.; Bix, G.; Doré, S.; Stowe, A.M. Biomarker Application for Precision Medicine in Stroke. Transl. Stroke Res. 2020, 11, 615–627. [Google Scholar] [CrossRef]
  47. Ferrer-Ferrer, M.; Dityatev, A. Shaping Synapses by the Neural Extracellular Matrix. Front. Neuroanat. 2018, 12, 40. [Google Scholar] [CrossRef] [PubMed]
  48. Stawarski, M.; Stefaniuk, M.; Wlodarczyk, J. Matrix metalloproteinase-9 involvement in the structural plasticity of dendritic spines. Front. Neuroanat. 2014, 8, 68. [Google Scholar] [CrossRef]
  49. Nagy, V.; Bozdagi, O.; Matynia, A.; Balcerzyk, M.; Okulski, P.; Dzwonek, J.; Costa, R.M.; Silva, A.J.; Kaczmarek, L.; Huntley, G.W. Matrix Metalloproteinase-9 Is Required for Hippocampal Late-Phase Long-Term Potentiation and Memory. J. Neurosci. 2006, 26, 1923–1934. [Google Scholar] [CrossRef]
  50. Michaluk, P.; Wawrzyniak, M.; Alot, P.; Szczot, M.; Wyrembek, P.; Mercik, K.; Medvedev, N.; Wilczek, E.; De Roo, M.; Zuschratter, W.; et al. Influence of matrix metalloproteinase MMP-9 on dendritic spine morphology. J. Cell Sci. 2011, 124, 3369–3380. [Google Scholar] [CrossRef]
  51. Reinhard, S.M.; Razak, K.; Ethell, I.M. A delicate balance: Role of MMP-9 in brain development and pathophysiology of neurodevelopmental disorders. Front. Cell. Neurosci. 2015, 9, 280. [Google Scholar] [CrossRef]
  52. Mizoguchi, H.; Nakade, J.; Tachibana, M.; Ibi, D.; Someya, E.; Koike, H.; Kamei, H.; Nabeshima, T.; Itohara, S.; Takuma, K.; et al. Matrix metalloproteinase-9 contributes to kindled seizure development in pentylenetetrazole-treated mice by converting pro-BDNF to mature BDNF in the hippocampus. J. Neurosci. 2011, 31, 12963–12971. [Google Scholar] [CrossRef]
  53. Cojocarui, I.M.; Cojocaru, M.; Sapira, V.; Socoliuc, G.; Hertea, C.; Paveliu, S. Changes in plasma matrix metalloproteinase-9 levels in patients with acute ischemic stroke. Rom. J. Intern. Med. 2012, 50, 155–158. [Google Scholar] [PubMed]
  54. Brouns, R.; Wauters, A.; De Surgeloose, D.; Mariën, P.; De Deyn, P.P. Biochemical Markers for Blood-Brain Barrier Dysfunction in Acute Ischemic Stroke Correlate with Evolution and Outcome. Eur. Neurol. 2011, 65, 23–31. [Google Scholar] [CrossRef]
  55. Guo, D.; Zhu, Z.; Zhong, C.; Wang, A.; Xie, X.; Xu, T.; Peng, Y.; Peng, H.; Li, Q.; Ju, Z.; et al. Prognostic Metrics Associated with Inflammation and Atherosclerosis Signaling Evaluate the Burden of Adverse Clinical Outcomes in Ischemic Stroke Patients. Clin. Chem. 2020, 66, 1434–1443. [Google Scholar] [CrossRef] [PubMed]
  56. Zhong, C.; Bu, X.; Xu, T.; Guo, L.; Wang, X.; Zhang, J.; Cui, Y.; Li, D.; Zhang, J.; Ju, Z.; et al. Serum Matrix Metalloproteinase-9 and Cognitive Impairment After Acute Ischemic Stroke. J. Am. Heart Assoc. 2018, 7, e007776. [Google Scholar] [CrossRef] [PubMed]
  57. Zhao, J.; Li, Q.; Meng, L.; Wang, F.; Li, Q.; Yang, F.; Wang, M.; Yu, M.; Zhang, J.; Li, S.; et al. Relationship between MMP-9 serum levels and tHcy levels and total imaging load and cognitive dysfunction. J. Stroke Cerebrovasc. Dis. 2022, 31, 106759. [Google Scholar] [CrossRef]
  58. Castellanos, M.; Sobrino, T.; Millán, M.; García, M.; Arenillas, J.; Nombela, F.; Brea, D.; Perez de la Ossa, N.; Serena, J.; Vivancos, J.; et al. Serum cellular fibronectin and matrix metalloproteinase-9 as screening biomarkers for the prediction of parenchymal hematoma after thrombolytic therapy in acute ischemic stroke: A multicenter confirmatory study. Stroke 2007, 38, 1855–1859. [Google Scholar] [CrossRef] [PubMed]
  59. Burton, L.; Tyson, S.F. Screening for cognitive impairment after stroke: A systematic review of psychometric properties and clinical utility. J. Rehabil. Med. 2015, 47, 193–203. [Google Scholar] [CrossRef]
  60. Lee, Y.C.; Lin, Y.T.; Chiu, E.C. A comparison of test-retest reliability of four cognitive screening tools in people with dementia. Disabil. Rehabil. 2022, 44, 4090–4095. [Google Scholar] [CrossRef]
  61. Goldberg, T.E.; Harvey, P.D.; Wesnes, K.A.; Snyder, P.J.; Schneider, L.S. Practice effects due to serial cognitive assessment: Implications for preclinical Alzheimer’s disease randomized controlled trials. Alzheimers Dement. 2015, 1, 103–111. [Google Scholar] [CrossRef]
  62. Galasko, D.; Abramson, I.; Corey-Bloom, J.; Thal, L.J. Repeated exposure to the Mini-Mental State Examination and the Information-Memory-Concentration Test results in a practice effect in Alzheimer’s disease. Neurology 1993, 43, 1559–1563. [Google Scholar] [CrossRef]
  63. Feeney, J.; Savva, G.M.; O’Regan, C.; King-Kallimanis, B.; Cronin, H.; Kenny, R.A. Measurement Error, Reliability, and Minimum Detectable Change in the Mini-Mental State Examination, Montreal Cognitive Assessment, and Color Trails Test among Community Living Middle-Aged and Older Adults. J. Alzheimers Dis. 2016, 53, 1107–1114. [Google Scholar] [CrossRef]
  64. Che, B.; Zhong, C.; Ge, J.; Li, R.; Zhu, Z.; Bu, X.; Xu, T.; Ju, Z.; Liu, J.; Zhang, J.; et al. Serum Matrix Metalloproteinase-9 Is Associated With Depression After Acute Ischemic Stroke. Circ. J. 2019, 83, 2303–2311. [Google Scholar] [CrossRef]
  65. Saban, M.R.; Davis, C.A.; Avelino, A.; Cruz, F.; Maier, J.; Bjorling, D.E.; Sferra, T.J.; Hurst, R.E.; Saban, R. VEGF signaling mediates bladder neuroplasticity and inflammation in response to BCG. BMC Physiol. 2011, 11, 16. [Google Scholar] [CrossRef] [PubMed]
  66. Yao, Y.; Zheng, X.R.; Zhang, S.S.; Wang, X.; Yu, X.H.; Tan, J.L.; Yang, Y.J. Transplantation of vascular endothelial growth factor-modified neural stem/progenitor cells promotes the recovery of neurological function following hypoxic-ischemic brain damage. Neural Regen. Res. 2016, 11, 1456–1463. [Google Scholar] [CrossRef]
  67. Cattin, A.L.; Burden, J.J.; Van Emmenis, L.; Mackenzie, F.E.; Hoving, J.J.; Garcia Calavia, N.; Guo, Y.; McLaughlin, M.; Rosenberg, L.H.; Quereda, V.; et al. Macrophage-Induced Blood Vessels Guide Schwann Cell-Mediated Regeneration of Peripheral Nerves. Cell 2015, 162, 1127–1139. [Google Scholar] [CrossRef]
  68. Chodobski, A.; Chung, I.; Koźniewska, E.; Ivanenko, T.; Chang, W.; Harrington, J.F.; Duncan, J.A.; Szmydynger-Chodobska, J. Early neutrophilic expression of vascular endothelial growth factor after traumatic brain injury. Neuroscience 2003, 122, 853–867. [Google Scholar] [CrossRef] [PubMed]
  69. Shen, S.-W.; Duan, C.-L.; Chen, X.-H.; Wang, Y.-Q.; Sun, X.; Zhang, Q.-W.; Cui, H.-R.; Sun, F.-Y. Neurogenic effect of VEGF is related to increase of astrocytes transdifferentiation into new mature neurons in rat brains after stroke. Neuropharmacology 2016, 108, 451–461. [Google Scholar] [CrossRef] [PubMed]
  70. Xue, L.; Chen, H.; Zhang, T.; Chen, J.; Geng, Z.; Zhao, Y. Changes in serum vascular endothelial growth factor and endostatin concentrations associated with circulating endothelial progenitor cells after acute ischemic stroke. Metab. Brain Dis. 2017, 32, 641–648. [Google Scholar] [CrossRef]
  71. Nunes, F.D.D.; Ferezin, L.P.; Pereira, S.C.; Figaro-Drumond, F.V.; Pinheiro, L.C.; Menezes, I.C.; Baes, C.V.W.; Coeli-Lacchini, F.B.; Tanus-Santos, J.E.; Juruena, M.F.; et al. The Association of Biochemical and Genetic Biomarkers in VEGF Pathway with Depression. Pharmaceutics 2022, 14, 2757. [Google Scholar] [CrossRef]
  72. Bhasin, A.; Srivastava, M.V.P.; Vivekanandhan, S.; Moganty, R.; Talwar, T.; Sharma, S.; Kuthiala, N.; Kumaran, S.; Bhatia, R. Vascular Endothelial Growth Factor as Predictive Biomarker for Stroke Severity and Outcome; An Evaluation of a New Clinical Module in Acute Ischemic Stroke. Neurol. India 2019, 67, 1280–1285. [Google Scholar] [CrossRef] [PubMed]
  73. Escudero, C.; Acurio, J.; López, E.; Rodríguez, A.; Benavente, A.; Lara, E.; Korzeniewski, S.J. Vascular endothelial growth factor and poor prognosis after ischaemic stroke. Eur. J. Neurol. 2021, 28, 1759–1764. [Google Scholar] [CrossRef]
  74. Zhang, Y.; Ma, T.; Hu, H.; Wang, J.; Zhou, S. Serum vascular endothelial growth factor as a biomarker for prognosis of minor ischemic stroke. Clin. Neurol. Neurosurg. 2020, 196, 106060. [Google Scholar] [CrossRef]
  75. Dyer, A.H.; Vahdatpour, C.; Sanfeliu, A.; Tropea, D. The role of Insulin-Like Growth Factor 1 (IGF-1) in brain development, maturation and neuroplasticity. Neuroscience 2016, 325, 89–99. [Google Scholar] [CrossRef] [PubMed]
  76. Ashpole, N.M.; Sanders, J.E.; Hodges, E.L.; Yan, H.; Sonntag, W.E. Growth hormone, insulin-like growth factor-1 and the aging brain. Exp. Gerontol. 2015, 68, 76–81. [Google Scholar] [CrossRef]
  77. Saber, H.; Himali, J.J.; Beiser, A.S.; Shoamanesh, A.; Pikula, A.; Roubenoff, R.; Romero, J.R.; Kase, C.S.; Vasan, R.S.; Seshadri, S. Serum Insulin-Like Growth Factor 1 and the Risk of Ischemic Stroke: The Framingham Study. Stroke 2017, 48, 1760–1765. [Google Scholar] [CrossRef]
  78. Hunt, K.J.; Lukanova, A.; Rinaldi, S.; Lundin, E.; Norat, T.; Palmqvist, R.; Stattin, P.; Riboli, E.; Hallmans, G.; Kaaks, R. A potential inverse association between insulin-like growth factor I and hypertension in a cross-sectional study. Ann. Epidemiol. 2006, 16, 563–571. [Google Scholar] [CrossRef]
  79. Teppala, S.; Shankar, A. Association between serum IGF-1 and diabetes among U.S. adults. Diabetes Care 2010, 33, 2257–2259. [Google Scholar] [CrossRef]
  80. Li, S.; Zhao, Y.; Zhao, J.; Mou, L. Expression and clinical value of miR-128 and IGF-1 in patients with acute ischemic stroke. Minerva Med. 2020, 111, 544–550. [Google Scholar] [CrossRef] [PubMed]
  81. Ebinger, M.; Ipsen, N.; Leonards, C.O.; Empl, L.; Hanne, L.; Liman, T.; Mai, K.; Strasburger, C.J.; Spranger, J.; Endres, M. Circulating insulin-like growth factor binding protein-3 predicts one-year outcome after ischemic stroke. Exp. Clin. Endocrinol. Diabetes 2015, 123, 461–465. [Google Scholar] [CrossRef] [PubMed]
  82. Armbrust, M.; Worthmann, H.; Dengler, R.; Schumacher, H.; Lichtinghagen, R.; Eschenfelder, C.C.; Endres, M.; Ebinger, M. Circulating Insulin-like Growth Factor-1 and Insulin-like Growth Factor Binding Protein-3 predict Three-months Outcome after Ischemic Stroke. Exp. Clin. Endocrinol. Diabetes 2017, 125, 485–491. [Google Scholar] [CrossRef]
  83. Ploughman, M.; Eskes, G.A.; Kelly, L.P.; Kirkland, M.C.; Devasahayam, A.J.; Wallack, E.M.; Abraha, B.; Hasan, S.M.M.; Downer, M.B.; Keeler, L.; et al. Synergistic Benefits of Combined Aerobic and Cognitive Training on Fluid Intelligence and the Role of IGF-1 in Chronic Stroke. Neurorehabil. Neural Repair 2019, 33, 199–212. [Google Scholar] [CrossRef]
  84. Mattlage, A.E.; Rippee, M.A.; Abraham, M.G.; Sandt, J.; Billinger, S.A. Estimated Prestroke Peak VO2 Is Related to Circulating IGF-1 Levels During Acute Stroke. Neurorehabil. Neural Repair 2017, 31, 65–71. [Google Scholar] [CrossRef] [PubMed]
  85. Rost, N.S.; Meschia, J.F.; Gottesman, R.; Wruck, L.; Helmer, K.; Greenberg, S.M. Cognitive Impairment and Dementia After Stroke: Design and Rationale for the DISCOVERY Study. Stroke 2021, 52, e499–e516. [Google Scholar] [CrossRef] [PubMed]
  86. Drag, L.L.; Mlynash, M.; Nassar, H.; Osborn, E.; Kim, D.E.; Angst, M.S.; Aghaeepour, N.; Buckwalter, M.; Lansberg, M.G. A longitudinal study of the post-stroke immune response and cognitive functioning: The StrokeCog study protocol. BMC Neurol. 2020, 20, 313. [Google Scholar] [CrossRef]
  87. Wardlaw, J.M.; Doubal, F.; Brown, R.; Backhouse, E.; Woodhouse, L.; Bath, P.; Quinn, T.J.; Robinson, T.; Markus, H.S.; McManus, R.; et al. Rates, risks and routes to reduce vascular dementia (R4vad), a UK-wide multicentre prospective observational cohort study of cognition after stroke: Protocol. Eur. Stroke J. 2021, 6, 89–101. [Google Scholar] [CrossRef] [PubMed]
  88. Brännmark, C.; Klasson, S.; Stanne, T.M.; Samuelsson, H.; Alt Murphy, M.; Sunnerhagen, K.S.; Åberg, N.D.; Jalnefjord, O.; Björkman-Burtscher, I.; Jood, K.; et al. FIND Stroke Recovery Study (FIND): Rationale and protocol for a longitudinal observational cohort study of trajectories of recovery and biomarkers poststroke. BMJ Open 2023, 13, e072493. [Google Scholar] [CrossRef] [PubMed]
Figure 1. The study project. MRS—Modified Rankin Scale, NIHSS—National Institutes of Health Stroke Scale, MMSE—Mini Mental State Examination, MoCA—Montreal Cognitive Assessment, GDS—Geriatric Depression Scale, BDI- Beck Depression Inventory, MMP-9—Matrix metalloproteinase 9, VEGF—Vascular endothelial growth factor, IGF-1—Insulin-like growth factor 1.
Figure 1. The study project. MRS—Modified Rankin Scale, NIHSS—National Institutes of Health Stroke Scale, MMSE—Mini Mental State Examination, MoCA—Montreal Cognitive Assessment, GDS—Geriatric Depression Scale, BDI- Beck Depression Inventory, MMP-9—Matrix metalloproteinase 9, VEGF—Vascular endothelial growth factor, IGF-1—Insulin-like growth factor 1.
Brainsci 13 00846 g001
Figure 2. The protein concentrations and mRNA expressions of the tested genes before and after rehabilitation. Data points represent the levels of measurands for each patient, whereas the horizontal lines indicate the mean values. One-way repeated-measure analysis of variance was used. The p-values for significant results are presented in bold. (A) MMP9 protein concentration in plasma; (B) VEGF protein concentration in plasma; (C) IGF1 protein concentration in plasma; (D) MMP9 mRNA expression in whole blood; (E) VEGF mRNA expression in whole blood.
Figure 2. The protein concentrations and mRNA expressions of the tested genes before and after rehabilitation. Data points represent the levels of measurands for each patient, whereas the horizontal lines indicate the mean values. One-way repeated-measure analysis of variance was used. The p-values for significant results are presented in bold. (A) MMP9 protein concentration in plasma; (B) VEGF protein concentration in plasma; (C) IGF1 protein concentration in plasma; (D) MMP9 mRNA expression in whole blood; (E) VEGF mRNA expression in whole blood.
Brainsci 13 00846 g002
Figure 3. Receiver operating characteristic (ROC) curves for the performance of the model to predict (A) cognitive improvement based on MMP9 protein and mRNA levels, and (B) amelioration of depressive symptoms based on VEGF protein and mRNA levels. In each model, age and sex are included as covariates. Sensitivity and specificity are calculated based on the maximization of Youden’s index; sensitivity and specificity were not provided for the model of VEGF and depressive symptoms as the performance on the validation set was below the random guess. N/A—not applicable.
Figure 3. Receiver operating characteristic (ROC) curves for the performance of the model to predict (A) cognitive improvement based on MMP9 protein and mRNA levels, and (B) amelioration of depressive symptoms based on VEGF protein and mRNA levels. In each model, age and sex are included as covariates. Sensitivity and specificity are calculated based on the maximization of Youden’s index; sensitivity and specificity were not provided for the model of VEGF and depressive symptoms as the performance on the validation set was below the random guess. N/A—not applicable.
Brainsci 13 00846 g003
Table 1. Demographic characteristics.
Table 1. Demographic characteristics.
ParameterMean (SD) or Number (Frequency)
Sociodemographic
Sex
Female14 (44%)
Male18 (56%)
Age [years]68.3 (9.1)
Comorbidity and treatment
Hypertension17 (53%)
Diabetes10 (31%)
Atherosclerosis4 (13%)
Trombolytic treatment4 (13%)
Blood parameters
Sodium139.2 (3.0)
Potassium4.2 (0.7)
WBC7.46 (2.16)
RBC4.30 (0.63)
Hb12.9 (1.7)
HCT40.2 (4.0)
Urea29.8 (13.8)
Creatinine0.7 (0.2)
CRP9.0 (10.1)
6.0 (1.3–11.2) a
a—where the distribution was highly skewed, the results are also presented as median with 1st–3rd quartiles.
Table 2. Change in physical and psychological functioning before and after rehabilitation.
Table 2. Change in physical and psychological functioning before and after rehabilitation.
ParameterMean Estimate ± SEMp-Value
BeforeAfter
Physical
ADL8.8 ± 0.813.0 ± 0.90.0008
Rankin3.9 ± 0.23.1 ± 0.20.0024
NIHSS9.0 ± 0.56.6 ± 0.70.0059
Psychological
Cognitive function 0.0004 a,b
MOCA20.0 ± 1.924.6 ± 1.9
MMSE23.4 ± 0.625.2 ± 0.8
Depressive symptoms 0.0162 a,c
BDI7.6 ± 1.57.3 ± 2.4
GDS11.8 ± 0.96.8 ± 1.0
a—as different psychometric tests were used in patients of different ages, the type of the test was used as a covariate in two-way repeated-measure analysis of variance (a between factor), b—there was an insignificant within–between interaction observed (p = 0.1008) suggesting similar improvement in cognitive function in both age groups, c—there was a significant within–between interaction observed (p = 0.0302) suggesting that senior participants demonstrated greater improvements in cognitive function than younger adults
Table 3. Correlation between the improvement in psychological functioning over the rehabilitation process and changes in the levels of the tested protein/mRNA.
Table 3. Correlation between the improvement in psychological functioning over the rehabilitation process and changes in the levels of the tested protein/mRNA.
Change in the Level of Measurand over the Rehabilitation ProcessCognitive ImprovementImprovement in Depression
Beta Coefficient ± SEMp-ValueBeta Coefficient ± SEMp-Value
MMP9 protein0.49 ± 0.150.0034−0.02 ± 0.170.8878
VEGF protein0.01 ± 0.180.9333−0.34 ± 0.160.0427
IGF1 protein−0.03 ± 0.180.8721−0.07 ± 0.170.7083
MMP9 mRNA−0.02 ± 0.190.89740.29 ± 0.180.1076
VEGF-A mRNA−0.22 ± 0.180.22570.42 ± 0.160.0117
Changes in the levels of measurands were calculated as the difference between post-rehabilitation and pre-rehabilitation value. Separate general linear models were built with the psychological improvement indices as outcomes, changes in measurand levels as continuous predictors, and the type of the psychometric test used as a dichotomous covariate. Significant results are presented in bold.
Table 4. Correlation between the improvement in psychological functioning over the rehabilitation process and the pre-rehabilitation levels of measurands.
Table 4. Correlation between the improvement in psychological functioning over the rehabilitation process and the pre-rehabilitation levels of measurands.
Pre-Rehabilitation Level of MeasurandCognitive ImprovementImprovement in Depression
Beta Coefficient ± SEMp-ValueBeta Coefficient ± SEMp-Value
MMP9 protein0.26 ± 0.170.1402−0.16 ± 0.170.3504
VEGF protein0.03 ± 0.190.8603−0.41 ± 0.170.0218
IGF1 protein0.08 ± 0.180.65750.11 ± 0.170.5225
MMP9 mRNA−0.45 ± 0.170.01270.14 ± 0.180.4319
VEGF-A mRNA−0.15 ± 0.180.4084−0.25 ± 0.170.1499
Separate general linear models were built with psychological improvement indices as outcomes, pre-rehabilitation measurand levels as continuous predictors, and the type of the psychometric test as a dichotomous covariate. The significant results are presented in bold.
Table 5. Multivariate logistic regression performed to predict post-rehabilitation improvement in cognitive function based on pre-rehabilitation MMP-9 protein and mRNA levels. Age and sex are included as covariates. Hosmer–Lemeshow χ2(8) = 4.63, p = 0.7963, Nagelkerke R2 = 0.64, Cox-Snell R2 = 0.48.
Table 5. Multivariate logistic regression performed to predict post-rehabilitation improvement in cognitive function based on pre-rehabilitation MMP-9 protein and mRNA levels. Age and sex are included as covariates. Hosmer–Lemeshow χ2(8) = 4.63, p = 0.7963, Nagelkerke R2 = 0.64, Cox-Snell R2 = 0.48.
PredictorOdds RatioWald Statisticsp-Value
Point Estimate95% CI
MMP9 protein a1.341.04–1.735.20.0225
MMP9 mRNA b0.250.08–0.766.00.0147
Age c0.900.78–1.042.00.1547
Sex d4.260.22–82.920.90.3391
a—odds for a growth in MMP9 protein concentration by 10 pg/mL, b—odds for doubling the MMP9 mRNA expression, c—odds for a growth in the age by a year, d—odds for being male participants as compared to female participants.
Table 6. Multivariate logistic regression predicting post-rehabilitation improvement in depressive symptoms based on pre-rehabilitation VEGF protein and mRNA levels. Age and sex are included as covariates. Hosmer–Lemeshow χ2(8) = 7.51, p = 0.4827, Nagelkerke R2 = 0.12, Cox-Snell R2 = 0.09.
Table 6. Multivariate logistic regression predicting post-rehabilitation improvement in depressive symptoms based on pre-rehabilitation VEGF protein and mRNA levels. Age and sex are included as covariates. Hosmer–Lemeshow χ2(8) = 7.51, p = 0.4827, Nagelkerke R2 = 0.12, Cox-Snell R2 = 0.09.
PredictorOdds RatioWald Statisticsp-Value
Point Estimate95% CI
VEGF protein a0.840.61–1.151.20.2693
VEGF-A mRNA b0.820.60–1.121.60.2109
Age c1.010.92–1.110.10.7922
Sex d1.170.24–5.76<0.10.8438
a—odds for a growth in VEGF protein concentration by 10 pg/mL, b—odds for doubling the VEGF-A mRNA expression, c—odds for a growth in the age by a year, d—odds for being male participants as compared to female participants.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Włodarczyk, L.; Cichoń, N.; Karbownik, M.S.; Saso, L.; Saluk, J.; Miller, E. Circulating Serum VEGF, IGF-1 and MMP-9 and Expression of Their Genes as Potential Prognostic Markers of Recovery in Post-Stroke Rehabilitation—A Prospective Observational Study. Brain Sci. 2023, 13, 846. https://doi.org/10.3390/brainsci13060846

AMA Style

Włodarczyk L, Cichoń N, Karbownik MS, Saso L, Saluk J, Miller E. Circulating Serum VEGF, IGF-1 and MMP-9 and Expression of Their Genes as Potential Prognostic Markers of Recovery in Post-Stroke Rehabilitation—A Prospective Observational Study. Brain Sciences. 2023; 13(6):846. https://doi.org/10.3390/brainsci13060846

Chicago/Turabian Style

Włodarczyk, Lidia, Natalia Cichoń, Michał Seweryn Karbownik, Luciano Saso, Joanna Saluk, and Elżbieta Miller. 2023. "Circulating Serum VEGF, IGF-1 and MMP-9 and Expression of Their Genes as Potential Prognostic Markers of Recovery in Post-Stroke Rehabilitation—A Prospective Observational Study" Brain Sciences 13, no. 6: 846. https://doi.org/10.3390/brainsci13060846

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop