Next Article in Journal
Aptamers and Aptasensors for Highly Specific Recognition and Sensitive Detection of Marine Biotoxins: Recent Advances and Perspectives
Next Article in Special Issue
The Effect of Botulinum Toxin Injections on Gross Motor Function for Lower Limb Spasticity in Children with Cerebral Palsy
Previous Article in Journal
Repellency, Toxicity, Gene Expression Profiling and In Silico Studies to Explore Insecticidal Potential of Melaleuca alternifolia Essential Oil against Myzus persicae
Previous Article in Special Issue
Transitioning from Unilateral to Bilateral Upper Limb Tremor Therapy for Parkinson’s Disease and Essential Tremor Using Botulinum Toxin: Case Series
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Toxins
Volume 10
Issue 11
10.3390/toxins10110426
toxins-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Botulinum Toxin Injections and Electrical Stimulation for Spastic Paresis Improve Active Hand Function Following Stroke

by
Jong-Min Lee
1,
Jean-Michel Gracies
2,
Si-Bog Park
3,
Kyu Hoon Lee
3,
Ji Yeong Lee
1 and
Joon-Ho Shin
1,*
1
Department of NeuroRehabilitation, National Rehabilitation Center, Ministry of Health and Welfare, 58 Samgaksan-ro, Gangbuk-gu, Seoul 01022, Korea
2
Service de Rééducation Neurolocomotrice, Albert Chenevier-Henri Mondor Hospital, 94000 Créteil, France
3
Department of Rehabilitation Medicine, Hanyang University College of Medicine, Seoul 04763, Korea
*
Author to whom correspondence should be addressed.
Toxins 2018, 10(11), 426; https://doi.org/10.3390/toxins10110426
Submission received: 22 September 2018 / Revised: 19 October 2018 / Accepted: 21 October 2018 / Published: 25 October 2018
(This article belongs to the Special Issue Botulinum Neurotoxins and Nervous System: Future Challenges for Novel Indications)
Browse Figure
Versions Notes

Abstract

:
Botulinum toxin type A (BTX-A) injections improve muscle tone and range of motion (ROM) among stroke patients with upper limb spasticity. However, the efficacy of BTX-A injections for improving active function is unclear. We aimed to determine whether BTX-A injections with electrical stimulation (ES) of hand muscles could improve active hand function (AHF) among chronic stroke patients. Our open-label, pilot study included 15 chronic stroke patients. Two weeks after BTX-A injections into the finger and/or wrist flexors, ES of finger extensors was performed while wearing a wrist brace for 4 weeks (5 days per week; 30-min sessions). Various outcomes were assessed at baseline, immediately before BTX-A injections, and 2 and 6 weeks after BTX-A injections. After the intervention, we noted significant improvements in Box and Block test results, Action Research Arm Test results, the number of repeated finger flexions/extensions, which reflect AHF, and flexor spasticity. Moreover, significant improvements in active ROM of wrist extension values were accompanied by marginally significant changes in Medical Research Council wrist extensor and active ROM of wrist flexion values. In conclusion, BTX-A injections into the finger and/or wrist flexors followed by ES of finger extensors improve AHF among chronic stroke patients.
Keywords:
spasticity; spastic paresis; botulinum toxin; electrical stimulation; stroke management; rehabilitation; hand
Key Contribution: Botulinum toxin type A injections into the finger and/or wrist flexor muscles followed by electrical stimulation of finger extensors targeting spastic paresis improve active hand function and impairment among patients with chronic stroke.

1. Introduction

Upper limb spasticity (ULS) is a common impairment after stroke, which can cause abnormal posture, pain, reduced movement, and functional impairment [1,2,3,4]. Various interventions, including occupational therapy, physical therapy, oral medications, orthoses, and focal neurolysis, have been used in the management of ULS.
Intramuscular injection of botulinum toxin type A (BTX-A) is the recommended first-line treatment for focal ULS [5,6,7,8,9]. Botulinum toxin type A, which is produced by anaerobic, gram-positive, spore-forming microorganisms from the genus Clostridium, is the most potent toxin among various serotypes, and its safety is well established [10,11]. Its neurospecific activity, reversibility, and limited diffusion on local injection make it a safe and successful agent for symptoms characterized by hyperfunction of peripheral nerve terminals [12,13]. Although BTX-A injections have been shown to improve muscle tone, range of motion, and physical global assessment findings, evidence regarding their efficacy for improving active upper limb function (e.g., moving/manipulating objects) is lacking [14,15]. Indeed, systematic reviews have demonstrated that BTX-A has greater effects on passive function than on active function, and some studies have failed to show any improvements in active function with BTX-A treatment [4,16]. These results might hinder active treatment with BTX-A, because the ultimate goal of rehabilitation is functional improvement.
Several explanations for the discrepancy between impairment and function have been proposed. First, most previous treatments have targeted spasticity per se, while factors other than hypertonicity are likely to be associated with active function, including stretch-sensitive paresis, overactivity of agonistic muscles, and learned non-use [2,3,15]. Second, optimal adjunct therapies following BTX-A injections have not been used, despite the need for adjuvant therapy to improve efficacy [17,18]. The combination of modified constraint-induced movement therapy or active task-specific training with BTX-A injections has been shown to improve upper limb function among patients with chronic spastic hemiparesis [19,20]. However, these approaches are not available for individuals with moderate-to-severe muscle weakness, prompting the need for commonly or easily applicable optimal adjunctive therapies to augment hand function. Third, although hand function is critical for upper limb function as an end-effector, few studies have targeted active function of the hand itself [21]. As hand spasticity is particularly disruptive, improvements in shoulder or elbow function may not be adequate to ensure overall functional improvement [22]. In addition, appropriate outcome measures related to both impairment and function, especially active hand function, have not been used [23,24,25]. Although previous studies have used various outcome measures for upper limb function (e.g., disability assessment scale, motor activity log, and global self-assessment), none have focused on active hand function [20,24,26]. Fourth, standardized techniques for administering BTX-A are yet to be established, and thus there are variations in muscle selection, injection methods, and medication doses.
Many patients having ULS exhibit difficulty with finger extension, which is critical for gripping, grasping, and releasing movements. Thus, we hypothesized that relief of finger flexor spasticity following BTX-A treatment would help patients regain finger extensor strength, which would in turn improve hand function. Based on the findings of previous studies, we further hypothesized that electrical stimulation (ES) is a simple yet effective form of adjuvant therapy, as it facilitates muscle reeducation, including antagonist strengthening and agonist lengthening [27,28]. Therefore, in the present preliminary study, we aimed to determine whether BTX-A injections targeting finger flexor spasticity with concomitant ES of finger extensors can improve active hand function and impairment among patients with chronic stroke.

2. Results

Among the 15 included patients, 7 experienced cerebral infarction and 8 experienced intracerebral hemorrhage. Thirteen patients experienced lesions in the right hemisphere. The mean time from stroke was 12.8 ± 3.7 months. All 15 patients completed the study, and none experienced adverse events.

2.1. Primary Outcomes

After the intervention, there were significant improvements in Box and Block (BB) (χ2 = 6.50, p = 0.039), Action Research Arm Test (ARAT)-total (χ2 = 6.29, p = 0.043), and ARAT-gross movement scores (χ2 = 8.64, p = 0.013), which reflect active hand function (Table 1). Post-hoc analysis confirmed that there were significant improvements in these outcomes in the T1–T3 phase. In addition, significant improvements in ARAT-total and ARAT-gross movement scores were observed in the T2–T3 and T1–T2 phases, respectively.

2.2. Secondary Outcomes

Repeat-finger extensor/extension (FE) values showed significant increases (χ2 = 10.00, p = 0.007) in the T1–T3 and T2–T3 phases. Active-FE values showed marginally significant increases during the T2–T3 phase (χ2 = 5.615, p = 0.060).
Flexor spasticity significantly decreased (Modified Ashworth Scale for wrist flexor/flexion (MAS-WF): χ2 = 8.40, p = 0.015; MAS for finger flexor/flexion (MAS-FF): χ2 = 20.18, p < 0.001), and post-hoc tests revealed significant differences in the T1–T2 and T1–T3 phases, but not in the T2–T3 phase (Table 2). Significant increases in active range of motion for wrist extensor/extension (AROM-WE) values (χ2 = 9.08, p = 0.011) were accompanied by marginally significant increases in Medical Research Council WE (MRC-WE) (χ2 = 6.00, p = 0.05) and AROM-WF (χ2 = 6.00, p = 0.05) values. Post-hoc analyses demonstrated that AROM-WE values were higher at T3 than at T1 or T2 and that MRC-WE values significantly increased from T1 to T3. No significant changes were noted for other secondary outcomes, such as the Quick Disabilities of Arm, Shoulder, and Hand (QDASH) score (χ2 = 3.636, p = 0.162).

3. Discussion

In the present study, we attempted to examine the efficacy of BTX-A treatment targeting finger flexor spasticity and concomitant ES for improving active hand function in the chronic phase of stroke. We found that BTX-A injections into the flexor muscles of the fingers and wrist followed by ES improved active hand function, as indicated by significant improvements in BB, ARAT (ARAT-total, ARAT-gross movement), and Repeat-FE scores. These findings are in contrast to those of previous studies, which failed to demonstrate improvements in active function after BTX-A treatment among patients with chronic stroke [29,30,31].
These discrepancies may be explained by our concept of intervention, as we considered treatment according to spastic paresis rather than flexor muscle spasticity only [2,3]. Overactivity of flexor muscles due to spasticity impedes finger and wrist extension, aggravating stretch-sensitive paresis and inhibiting voluntary recruitment of extensors. Similarly, a previous study demonstrated co-activation of spastic flexors and weak extensors [32]. Rehabilitation strategies often focus on finger/wrist flexors, as most hand functions are related to flexor movement (e.g., grasping, pinching, and manipulating objects). However, many patients with ULS following stroke exhibit difficulty with finger extension, which is needed to initiate hand grip and grasp. Thus, we attempted to restore the balance between the flexors and extensors of the wrist/fingers by targeting asymmetrical spastic paresis in two main steps (BTX-A injections and ES with a wrist brace).
Botulinum toxin type A injections to finger/wrist flexors reduced involuntary activation of the flexor muscles and further led to “therapeutic weakness” [33]. However, such weakness is only observed when the doses of BTX-A administered at the flexor digitorum superficialis and profundus are much larger than the doses recommended by the current consensus report [34,35]. Therapeutic weakening of the agonistic finger/wrist flexors allowed for restoration of antagonistic finger/wrist extensor function by reducing reciprocal inhibition from the flexors, stretch-induced paresis, agonistic flexor overactivity, and reflex tone [2,3,36].
Improved balance between agonist and antagonist muscles was accomplished with proprioceptive discharge reduction via group-IA afferents [37,38]. Botulinum toxin type A decreased afferent input of the muscle spindles and their stretch sensitivity. Presynaptic acetylcholine release block with BTX-A was more profound and earlier in gamma-motoneurons than in alpha-motoneurons. From the viewpoint of cortical activity, this BTX-A effect on proprioception is important because abnormal spindle activity generating irregular proprioceptive input may result in abnormal intracortical inhibition and cortical excitability [39]. Similarly, BTX-A-induced spasticity relief was associated with central sensorimotor activation change, which was likely mediated by an altered afferent input from spastic muscles [40]. Moreover, BTX-A reduced the activation of the contralateral putamen and thalamus via modulation of basal ganglia activation [41].
On the other hand, fascicle selection was tailored to each patient, and a flexible dosing regimen was utilized to ensure the appropriate level of weakness without inadvertent over-weakness. This may explain why we observed significant changes in ARAT-gross movement and MAS-WF/finger flexor/flexion (FF) values, but not MRC-WF/FF values or functional deterioration, between T1 and T2.
We subsequently utilized ES to facilitate target muscle (finger/wrist extensor) contraction, which has been difficult to achieve owing to increased flexor tone. ES enabled reeducation, stretching, and strengthening of the extensor muscles [28]. This selective facilitation of agonistic extensor muscles resulted in significant improvements in AROM-WE between T2 and T3, rather than improvements in AROM-WF. Concordantly, active function-related improvements were observed in ARAT-total, ARAT-gross movement, Repeat-FE, and Active-FE values during this phase, but not between T1 and T2.
Such differences between T1–T2 and T2–T3 may explain the discrepancies between active and passive functions following BTX-A treatment [42]. Our findings suggest that spasticity control is insufficient for restoring active function and that concomitant interventions selectively facilitating agonist muscles are necessary to improve outcomes. Notably, cyclic ES of both finger flexors and extensors following BTX-A injections has not been shown to be associated with functional improvements [25], suggesting that such interventions should target agonist function rather than antagonist spasticity per se.
In addition, we utilized a wrist brace to maximize finger extension during ES while reducing unwanted wrist or metacarpophalangeal joint hyperextension. Full finger extension is difficult during concomitant wrist extension owing to passive tension that results from the muscle insertion pattern of two-joint finger flexor muscles. The wrist brace maintained the wrist/metacarpophalangeal joints in a neutral position to allow sufficient extension of the distal or proximal interphalangeal joints, and it may have decreased stretch-sensitive paresis of the finger extensors.
In the present study, we attempted to utilize comprehensive hand-relevant outcome measures for both impairment (MRC, MAS, and AROM) and active function (ARAT and BB). We also utilized measures specific to hand function, particularly finger extension (Active FE, Distance-FP, Repeat-FE, and grip strength). These various measures assessed at different time points helped classify the effects of our intervention in terms of spastic paresis. Recently, goal attainment scales have been used to determine individualized, focal changes following BTX-A injections [43]. However, such scales do not measure objective outcomes but rather measure subjective achievement according to the opinions of the physicians and patients [44]. Therefore, we believe that the objective scales utilized in the present study are more appropriate than goal attainment scales for determining focal functional changes in clinical trials.
According to the QDASH results, no improvement in subjective upper limb function was observed, and this is likely due to the small yet significant objective change over the course of the intervention. Further studies should examine whether both subjective and objective improvements in upper limb function can be observed in select groups of patients, such as those in the subacute phase of stroke and those with some level of strength in the wrist/finger extensors.
The present study had several limitations. First, as this was not a randomized controlled trial, it is difficult to conclude whether our intervention would outperform other interventions or whether the combination of BTX-A injections and ES is more effective than either approach alone. Second, we did not measure outcomes at 12 weeks, when the effects of BTX-A can no longer be observed [45]. Finally, we did not take into account thumb function. Considering that thumb function accounts for 40% of hand performance, future studies should investigate the efficacy of our intervention with regard to thumb function [46].

4. Conclusions

Our findings indicate that BTX-A injections into the finger and/or wrist flexor muscles followed by ES of finger extensors improve active hand function and impairment among patients with chronic stroke. The present results support the notion that interventions targeting spastic paresis, rather than spasticity per se, may improve active hand function.

5. Materials and Methods

The present open-label, pilot study included 15 patients (14 men; 14 right-handed individuals; mean age, 44.7 ± 15.0 years; age range, 22–74 years; Table 3) from the stroke unit of a rehabilitation hospital between March 2016 and February 2017. Patients engaged in inpatient rehabilitation programs at the time of enrollment were allowed to continue therapy throughout the intervention, and anti-spasticity medications were continued at current or reduced doses or were discontinued. The study was approved by the ethics review board of our rehabilitation hospital (NRC-2016-02-011), and all participants provided written informed consent prior to enrollment. The study was registered with ClinicalTrials.gov (number NCT 03549975, 29 March 2016).

5.1. Inclusion and Exclusion Criteria

The inclusion criteria were as follows: (1) age older than 18 years; (2) hemiplegic ULS secondary to unilateral ischemic or hemorrhagic stroke; (3) finger and wrist flexor spasticity graded at least 1+ on the MAS; and (4) stroke occurrence at least 6 months prior to enrollment. The exclusion criteria were as follows: (1) fixed contracture of the wrist or hand; (2) previous ULS treatment with neurolytic therapy, surgery, or BTX-A injection; (3) presence of any active implanted device; (4) presence of any neurological disorder causing motor deficit or spasticity, other than stroke; (5) inability to cooperate for all outcome measure-related tasks secondary to cognitive impairment or aphasia; (6) pregnancy, planned pregnancy, or lactation; and (7) contraindication for BTX-A treatment.

5.2. Intervention

5.2.1. BTX-A Administration

Patients received BTX-A (onabotulinumtoxin A, Botox; Allergan Inc., Irvine, KY, USA) injections at week 0. The dilution of BTX-A was standardized, such that each vial of BTX-A (100 U) was diluted with 2 mL of normal saline (5 U/0.1 mL). A physician with experience in BTX-A treatment performed clinical evaluations of ULS and freely selected the target muscle and dose according to the pattern and severity of ULS in each patient. BTX-A injections were then administered under ultrasonographic guidance (ACCUVIX XG; Medison, Seoul, Korea). Injections at the flexor digitorum superficialis and profundus were administered by tailoring fascicle selection to each patient and by adopting a flexible dosing regimen. In each patient, the total BTX-A dose did not exceed 360 U. The dosages and injection sites are listed in Table 3.

5.2.2. ES with a Wrist Brace

After 2 weeks of BTX-A injections, patients received ES of finger extensors while wearing a wrist brace for 4 weeks. ES was administered 5 days per week during 30-min sessions by trained occupational therapists, using a Novastim CU-FS1 unit (CU, Medical, Seoul, Korea). During ES, each patient was seated in a chair with the affected arm resting on a table and with a small pillow supporting the pronated forearm. Cyclic ES was administered to induce muscle contractions, using biphasic pulses (frequency, 50 Hz; pulse duration, 200 μs) delivered for 6 s (ramp up, 1 s; ramp down, 2 s), with a 12-s burst-off. Two surface electrodes (5 × 5 cm) were placed over the extensor digitorum communis. The amplitude of stimulation was adjusted to elicit adequate finger extension for a sufficient range of motion against gravity, without pain.
The wrist brace (National Rehabilitation Center, Seoul, Korea) used during ES was made of soft fabric, and it covered the area from the wrist to the metacarpophalangeal joint. Two metal bars were attached to the volar and dorsal regions of the brace to immobilize the wrist in a neutral position. This immobilization ensured that ES could induce selective distal or proximal interphalangeal joint extension while overcoming unwanted simultaneous wrist or metacarpophalangeal joint extension. A Velcro strap was used to restrain the dorsal and volar sides of the brace.

5.3. Outcome Measures

We recorded baseline characteristics, including sex, age, stroke type, brain lesion side, time since stroke, and handedness. Primary and secondary outcomes were assessed at baseline (week −1, T0), immediately before BTX-A injections (week 0, T1), and 2 weeks (week 2, T2) and 6 weeks (week 6, T3) after BTX-A injections, by an experienced research occupational therapist. For further screening, we evaluated changes in outcomes between T0 and T1. Patients exhibiting changes between these two time points were excluded from the study. We examined the effects of BTX-A at 2 weeks, as previous studies have indicated that BTX-A shows maximum efficacy between 1 and 2 weeks after treatment [47]. Outcomes at T3 represented the combined effects of BTX-A and ES with a wrist brace (Figure 1).

5.3.1. Primary Outcomes

The primary outcomes were changes in patient results in the BB test and ARAT between T1 and T3, which represent changes in the active function of the upper limb following the present intervention. The BB test is a reliable and validated measure of unilateral gross manual dexterity, in which participants are required to pick up a 1-inch block, lift it over a partition, and then release it within a target area as many times as possible within 60 s [48]. The ARAT is a reliable and validated measure of arm function using observational methods with the following four subsections: grasp, grip, pinch, and gross movement. It comprises 19 items rated using a 4-point ordinal scale (0–3). Thus, the total scores on the ARAT range from 0 (failure to perform all movements; patients cannot perform any part of the test) to 57 (all movements are performed normally) [49].

5.3.2. Secondary Outcomes

Hand-specific active function was assessed according to active finger extension (Active-FE), the maximum distance from the fingertip to the palm (Distance-FP), the number of repeated finger flexion/extension movements within 20 s (Repeat-FE), the rating of active thumb opposition (Thumb-opposition), and grip strength as measured using the Jamar dynamometer (Sammons Preston, Bolingbrook, IL, USA). Active-FE was rated using a 5-point ordinal scale (0 = no movement, 4 = functional extension of all digits) by reviewing a video clip [50]. Distance-FP (cm) values were obtained by measuring the distance from the middle fingertip to the mid-palmar crease [51]. Thumb-opposition was rated using a 7-point scale (1 = to the distal phalanx of the index finger, 7 = to the medial phalanx of the small finger) [51].
Upper limb impairment was assessed according to MAS scores for the wrist and finger flexors/extensors (MAS-WF/WE/FF/FE), MRC grades for wrist and finger flexor/extensor strength (MRC-WF/WE/FF/FE), and active range of motion for wrist flexion/extension (AROM-WF/WE) and radial/ulnar deviation (AROM-RD/UD) as determined through goniometric measurement. Modified Ashworth Scale grades of spasticity were converted from 1+, 2, 3, and 4 to 2, 3, 4, and 5, respectively [52].
Subjective general upper limb function was evaluated using the self-reported QDASH scale. The QDASH scale consists of 11 items regarding physical function and symptoms during the previous week, and the total scores range from 0 (no disability) to 100 (most severe disability) [53].

5.4. Statistical Analysis

For all outcome measures, the Friedman test was used to determine the overall change from pre-intervention (T1) to post-intervention (T2)/follow-up (T3). When the Friedman test yielded significant results, various pairwise comparisons were performed using post-hoc Wilcoxon signed-rank tests in order to determine the effect of each intervention (BTX-A/ES). All statistical analyses were performed using SPSS 20.0 (IBM SPSS Statistics, IBM Corporation, Armonk, NY, USA). The level of statistical significance was set at p < 0.05.

Author Contributions

Conceptualization, J.-M.L., J.-M.G. and J.-H.S.; methodology, J.-H.S.; software, J.Y.L.; validation, S.-B.P. and K.H.L.; formal analysis, J.-M.L.; investigation, J.Y.L.; resources, S.-B.P.; data curation, K.H.L.; writing—original draft preparation, J.-M.L.; writing—review and editing, J.-H.S.; visualization, J.-M.L.; supervision, J.-H.S.; project administration, J.-H.S.; funding acquisition, J.-H.S.

Funding

This research was funded by the Translational Research Center for Rehabilitation Robots, National Rehabilitation Center, Ministry of Health and Welfare, Republic of Korea, grant number NRCTR-IN14002, NRCTR-IN15002.

Conflicts of Interest

Jean-Michel Gracies served as a consultant on advisory boards for and received research grant support from Allergan, Ipsen, and Merz. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.

References

  1. Sommerfeld, D.K.; Eek, E.U.-B.; Svensson, A.-K.; Holmqvist, L.W.; von Arbin, M.H. Spasticity after stroke: Its occurrence and association with motor impairments and activity limitations. Stroke 2004, 35, 134–139. [Google Scholar] [CrossRef] [PubMed]
  2. Gracies, J.M. Pathophysiology of spastic paresis. I: Paresis and soft tissue changes. Muscle Nerve 2005, 31, 535–551. [Google Scholar] [CrossRef] [PubMed]
  3. Gracies, J.M. Pathophysiology of spastic paresis. II: Emergence of muscle overactivity. Muscle Nerve 2005, 31, 552–571. [Google Scholar] [CrossRef] [PubMed]
  4. Foley, N.; Pereira, S.; Salter, K.; Fernandez, M.M.; Speechley, M.; Sequeira, K.; Miller, T.; Teasell, R. Treatment with botulinum toxin improves upper-extremity function post stroke: A systematic review and meta-analysis. Arch. Phys. Med. Rehabil. 2013, 94, 977–989. [Google Scholar] [CrossRef] [PubMed]
  5. Esquenazi, A.; Albanese, A.; Chancellor, M.B.; Elovic, E.; Segal, K.R.; Simpson, D.M.; Smith, C.P.; Ward, A.B. Evidence-based review and assessment of botulinum neurotoxin for the treatment of adult spasticity in the upper motor neuron syndrome. Toxicon 2013, 67, 115–128. [Google Scholar] [CrossRef] [PubMed]
  6. Wissel, J.; Ward, A.B.; Erztgaard, P.; Bensmail, D.; Hecht, M.J.; Lejeune, T.M.; Schnider, P. European consensus table on the use of botulinum toxin type A in adult spasticity. J. Rehabil. Med. 2009, 41, 13–25. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  7. Simpson, D.; Gracies, J.; Graham, H.; Miyasaki, J.; Naumann, M.; Russman, B.; Simpson, L.; So, Y. Assessment: Botulinum neurotoxin for the treatment of spasticity (an evidence-based review) Report of the Therapeutics and Technology Assessment Subcommittee of the American Academy of Neurology. Neurology 2008, 70, 1691–1698. [Google Scholar] [CrossRef] [PubMed]
  8. Demetrios, M.; Khan, F.; Turner-Stokes, L.; Brand, C.; Mc-Sweeney, S. Multidisciplinary rehabilitation following botulinum toxin and other focal intramuscular treatment for post-stroke spasticity. Cochrane Database Syst. Rev. 2012, 6, 12. [Google Scholar] [CrossRef]
  9. Simpson, D.M.; Hallett, M.; Ashman, E.J.; Comella, C.L.; Green, M.W.; Gronseth, G.S.; Armstrong, M.J.; Gloss, D.; Potrebic, S.; Jankovic, J. Practice guideline update summary: Botulinum neurotoxin for the treatment of blepharospasm, cervical dystonia, adult spasticity, and headache Report of the Guideline Development Subcommittee of the American Academy of Neurology. Neurology 2016, 86, 1818–1826. [Google Scholar] [CrossRef] [PubMed]
  10. Pirazzini, M.; Rossetto, O.; Eleopra, R.; Montecucco, C. Botulinum neurotoxins: Biology, pharmacology, and toxicology. Pharmacol. Rev. 2017, 69, 200–235. [Google Scholar] [CrossRef] [PubMed]
  11. Pirazzini, M. Novel Botulinum Neurotoxins: Exploring Underneath the Iceberg Tip. Toxins 2018, 10, 190. [Google Scholar] [Green Version]
  12. Hallett, M.; Albanese, A.; Dressler, D.; Segal, K.R.; Simpson, D.M.; Truong, D.; Jankovic, J. Evidence-based review and assessment of botulinum neurotoxin for the treatment of movement disorders. Toxicon 2013, 67, 94–114. [Google Scholar] [CrossRef] [PubMed]
  13. Naumann, M.; Dressler, D.; Hallett, M.; Jankovic, J.; Schiavo, G.; Segal, K.R.; Truong, D. Evidence-based review and assessment of botulinum neurotoxin for the treatment of secretory disorders. Toxicon 2013, 67, 141–152. [Google Scholar] [CrossRef] [PubMed]
  14. Shaw, L.; Rodgers, H.; Price, C.; van Wijck, F.; Shackley, P.; Steen, N.; Barnes, M.; Ford, G.; Graham, L. BoTULS: A multicentre randomised controlled trial to evaluate the clinical effectiveness and cost-effectiveness of treating upper limb spasticity due to stroke with botulinum toxin type A. Health Technol. Assess. 2010, 14, 1–113. [Google Scholar] [CrossRef] [PubMed]
  15. Gracies, J.M.; Brashear, A.; Jech, R.; McAllister, P.; Banach, M.; Valkovic, P.; Walker, H.; Marciniak, C.; Deltombe, T.; Skoromets, A.; et al. Safety and efficacy of abobotulinumtoxinA for hemiparesis in adults with upper limb spasticity after stroke or traumatic brain injury: A double-blind randomised controlled trial. Lancet Neurol. 2015, 14, 992–1001. [Google Scholar] [CrossRef]
  16. Dong, Y.; Wu, T.; Hu, X.; Wang, T. Efficacy and safety of Botulinum Toxin type A for upper limb spasticity after stroke or traumatic brain injury: A systematic review with meta-analysis and trial sequential analysis. Eur. J. Phys. Rehabil. Med. 2016, 53. [Google Scholar] [CrossRef]
  17. Rosales, R.L.; Efendy, F.; Teleg, E.S.; Santos, M.M.D.; Rosales, M.C.; Ostrea, M.; Tanglao, M.J.; Ng, A.R. Botulinum toxin as early intervention for spasticity after stroke or non-progressive brain lesion: A meta-analysis. J. Neurol. Sci. 2016, 371, 6–14. [Google Scholar] [CrossRef] [PubMed]
  18. Mills, P.B.; Finlayson, H.; Sudol, M.; O’Connor, R. Systematic review of adjunct therapies to improve outcomes following botulinum toxin injection for treatment of limb spasticity. Clin. Rehabil. 2016, 30, 537–548. [Google Scholar] [CrossRef] [PubMed]
  19. Sun, S.-F.; Hsu, C.-W.; Sun, H.-P.; Hwang, C.-W.; Yang, C.-L.; Wang, J.-L. Combined botulinum toxin type A with modified constraint-induced movement therapy for chronic stroke patients with upper extremity spasticity: A randomized controlled study. Neurorehabil. Neural Repair 2010, 24, 34–41. [Google Scholar] [CrossRef] [PubMed]
  20. Meythaler, J.M.; Vogtle, L.; Brunner, R.C. A preliminary assessment of the benefits of the addition of botulinum toxin a to a conventional therapy program on the function of people with longstanding stroke. Arch. Phys. Med. Rehabil. 2009, 90, 1453–1461. [Google Scholar] [CrossRef] [PubMed]
  21. Broeks, J.G.; Lankhorst, G.J.; Rumping, K.; Prevo, A.J. The long-term outcome of arm function after stroke: Results of a follow-up study. Disabil. Rehabil. 1999, 21, 357–364. [Google Scholar] [CrossRef] [PubMed]
  22. Mayer, N.H.; Esquenazi, A.; Childers, M.K. Common patterns of clinical motor dysfunction. Muscle Nerve 1997, 20, 21–35. [Google Scholar] [CrossRef]
  23. Picelli, A.; Lobba, D.; Midiri, A.; Prandi, P.; Melotti, C.; Baldessarelli, S.; Smania, N. Botulinum toxin injection into the forearm muscles for wrist and fingers spastic overactivity in adults with chronic stroke: A randomized controlled trial comparing three injection techniques. Clin. Rehabil. 2014, 28, 232–242. [Google Scholar] [CrossRef] [PubMed]
  24. Brashear, A.; Gordon, M.F.; Elovic, E.; Kassicieh, V.D.; Marciniak, C.; Do, M.; Lee, C.-H.; Jenkins, S.; Turkel, C. Intramuscular injection of botulinum toxin for the treatment of wrist and finger spasticity after a stroke. N. Engl. J. Med. 2002, 347, 395–400. [Google Scholar] [CrossRef] [PubMed]
  25. Weber, D.J.; Skidmore, E.R.; Niyonkuru, C.; Chang, C.-L.; Huber, L.M.; Munin, M.C. Cyclic functional electrical stimulation does not enhance gains in hand grasp function when used as an adjunct to onabotulinumtoxinA and task practice therapy: A single-blind, randomized controlled pilot study. Arch. Phys. Med. Rehabil. 2010, 91, 679–686. [Google Scholar] [CrossRef] [PubMed]
  26. Gracies, J.M.; Bayle, N.; Goldberg, S.; Simpson, D.M. Botulinum toxin type B in the spastic arm: A randomized, double-blind, placebo-controlled, preliminary study. Arch. Phys. Med. Rehabil. 2014, 95, 1303–1311. [Google Scholar] [CrossRef] [PubMed]
  27. Billian, C.; Gorman, P.H. Upper extremity applications of functional neuromuscular stimulation. Assist. Technol. 1992, 4, 31–39. [Google Scholar] [CrossRef] [PubMed]
  28. Chae, J.; Bethoux, F.; Bohinc, T.; Dobos, L.; Davis, T.; Friedl, A. Neuromuscular stimulation for upper extremity motor and functional recovery in acute hemiplegia. Stroke 1998, 29, 975–979. [Google Scholar] [CrossRef] [PubMed]
  29. Simpson, D.; Alexander, D.; O’brien, C.; Tagliati, M.; Aswad, A.; Leon, J.; Gibson, J.; Mordaunt, J.; Monaghan, E. Botulinum toxin type A in the treatment of upper extremity spasticity A randomized, double-blind, placebo-controlled trial. Neurology 1996, 46, 1306. [Google Scholar] [CrossRef] [PubMed]
  30. Smith, S.; Ellis, E.; White, S.; Moore, A. A double-blind placebo-controlled study of botulinum toxin in upper limb spasticity after stroke or head injury. Clin. Rehabil. 2000, 14, 5–13. [Google Scholar] [CrossRef] [PubMed]
  31. Sheean, G.L. Botulinum treatment of spasticity: Why is it so difficult to show a functional benefit? Curr. Opin. Neurol. 2001, 14, 771–776. [Google Scholar] [CrossRef] [PubMed]
  32. Kamper, D.; Rymer, W. Impairment of voluntary control of finger motion following stroke: Role of inappropriate muscle coactivation. Muscle Nerve 2001, 24, 673–681. [Google Scholar] [CrossRef] [PubMed]
  33. Li, S.; Francisco, G.E. New insights into the pathophysiology of post-stroke spasticity. Front. Hum. Neurosci. 2015, 9, 192. [Google Scholar] [CrossRef] [PubMed]
  34. Picelli, A.; Baricich, A.; Cisari, C.; Paolucci, S.; Smania, N.; Sandrini, G. The Italian real-life post-stroke spasticity survey: Unmet needs in the management of spasticity with botulinum toxin type A. Funct. Neurol. 2017, 32, 89–96. [Google Scholar] [CrossRef] [PubMed]
  35. Simpson, D.M.; Patel, A.T.; Alfaro, A.; Ayyoub, Z.; Charles, D.; Dashtipour, K.; Esquenazi, A.; Graham, G.D.; McGuire, J.R.; Odderson, I. OnabotulinumtoxinA Injection for Poststroke Upper-Limb Spasticity: Guidance for Early Injectors from a Delphi Panel Process. PM R 2017, 9, 136–148. [Google Scholar] [CrossRef] [PubMed]
  36. Filippi, G.M.; Errico, P.; Santarelli, R.; Bagolini, B.; Manni, E. Botulinum A toxin effects on rat jaw muscle spindles. Acta Oto-Laryngol. 1993, 113, 400–404. [Google Scholar] [CrossRef]
  37. Manni, E.; Bagolini, B.; Pettorossi, V.E.; Errico, P. Effect of botulinum toxin on extraocular muscle proprioception. Doc. Ophthalmol. 1989, 72, 189–198. [Google Scholar] [CrossRef] [PubMed]
  38. Wöber, C.; Schnider, P.; Steinhoff, N.; Trattnig, S.; Zebenholzer, K.; Auff, E. Posturographic findings in patients with idiopathic cervical dystonia before and after local injections with botulinum toxin. Eur. Neurol. 1999, 41, 194–200. [Google Scholar] [CrossRef] [PubMed]
  39. Kaňovský, P.; Rosales, R.L. Debunking the pathophysiological puzzle of dystonia–with special reference to botulinum toxin therapy. Parkinsonism Relat. Disord. 2011, 17, S11–S14. [Google Scholar] [CrossRef] [PubMed]
  40. Veverka, T.; Hluštík, P.; Hok, P.; Otruba, P.; Zapletalová, J.; Tüdös, Z.; Krobot, A.; Kaňovský, P. Sensorimotor modulation by botulinum toxin A in post-stroke arm spasticity: Passive hand movement. J. Neurol. Sci. 2016, 362, 14–20. [Google Scholar] [CrossRef] [PubMed]
  41. Dresel, C.; Bayer, F.; Castrop, F.; Rimpau, C.; Zimmer, C.; Haslinger, B. Botulinum toxin modulates basal ganglia but not deficient somatosensory activation in orofacial dystonia. Mov. Disord. 2011, 26, 1496–1502. [Google Scholar] [CrossRef] [PubMed]
  42. Zakin, E.; Simpson, D. Evidence on botulinum toxin in selected disorders. Toxicon 2018, 147, 134–140. [Google Scholar] [CrossRef] [PubMed]
  43. McCrory, P.; Turner-Stokes, L.; Baguley, I.J.; De Graaff, S.; Katrak, P.; Sandanam, J.; Davies, L.; Munns, M.; Hughes, A. Botulinum toxin A for treatment of upper limb spasticity following stroke: A multi-centre randomized placebo-controlled study of the effects on quality of life and other person-centred outcomes. J. Rehabil. Med. 2009, 41, 536–544. [Google Scholar] [CrossRef] [PubMed]
  44. Fheodoroff, K.; Ashford, S.; Jacinto, J.; Maisonobe, P.; Balcaitiene, J.; Turner-Stokes, L. Factors influencing goal attainment in patients with post-stroke upper limb spasticity following treatment with botulinum toxin A in real-life clinical practice: Sub-analyses from the Upper Limb International Spasticity (ULIS)-II Study. Toxins 2015, 7, 1192–1205. [Google Scholar] [CrossRef] [PubMed]
  45. Shaw, L.C.; Price, C.I.; van Wijck, F.M.; Shackley, P.; Steen, N.; Barnes, M.P.; Ford, G.A.; Graham, L.A.; Rodgers, H. Botulinum Toxin for the Upper Limb after Stroke (BoTULS) Trial: Effect on impairment, activity limitation, and pain. Stroke 2011, 42, 1371–1379. [Google Scholar] [CrossRef] [PubMed]
  46. Emerson, E.T.; Krizek, T.J.; Greenwald, D.P. Anatomy, physiology, and functional restoration of the thumb. Ann. Plast. Surg. 1996, 36, 180–191. [Google Scholar] [CrossRef] [PubMed]
  47. Asutay, F.; Atalay, Y.; Asutay, H.; Acar, A.H. The Evaluation of the Clinical Effects of Botulinum Toxin on Nocturnal Bruxism. Pain Res. Manag. 2017, 2017, 5. [Google Scholar] [CrossRef] [PubMed]
  48. Desrosiers, J.; Bravo, G.; Hébert, R.; Dutil, É.; Mercier, L. Validation of the Box and Block Test as a measure of dexterity of elderly people: Reliability, validity, and norms studies. Arch. Phys. Med. Rehabil. 1994, 75, 751–755. [Google Scholar] [PubMed]
  49. Van der Lee, J.H.; Beckerman, H.; Lankhorst, G.J.; Bouter, L.M. The responsiveness of the Action Research Arm test and the Fugl-Meyer Assessment scale in chronic stroke patients. J. Rebab. Med. 2001, 33, 110–113. [Google Scholar]
  50. Rodriquez, A.A.; McGinn, M.; Chappell, R. Botulinum toxin injection of spastic finger flexors in hemiplegic patients. Am. J. Phys. Med. Rehabil. 2000, 79, 44–47. [Google Scholar] [CrossRef] [PubMed]
  51. Ring, H.; Rosenthal, N. Controlled study of neuroprosthetic functional electrical stimulation in sub-acute post-stroke rehabilitation. J. Rehabil. Med. 2005, 37, 32–36. [Google Scholar] [CrossRef] [PubMed]
  52. Bohannon, R.W.; Smith, M.B. Interrater reliability of a modified Ashworth scale of muscle spasticity. Phys. Ther. 1987, 67, 206–207. [Google Scholar] [CrossRef] [PubMed]
  53. Gummesson, C.; Ward, M.M.; Atroshi, I. The shortened disabilities of the arm, shoulder and hand questionnaire (Quick DASH): Validity and reliability based on responses within the full-length DASH. BMC Musculoskelet. Disord. 2006, 7, 44. [Google Scholar] [CrossRef] [PubMed]
Toxins 10 00426 g001 550
Figure 1. Experimental design. Primary and secondary outcomes are assessed at baseline (week 1, T0), immediately before BTX-A injections (week 0, T1), and 2 weeks (week 2, T2) and 6 weeks (week 6, T3) after BTX-A injections, by an experienced research occupational therapist. Patients exhibiting changes in outcomes between T0 and T1 were excluded. BTX-A: botulinum toxin type A, ES: electrical stimulation.
Figure 1. Experimental design. Primary and secondary outcomes are assessed at baseline (week 1, T0), immediately before BTX-A injections (week 0, T1), and 2 weeks (week 2, T2) and 6 weeks (week 6, T3) after BTX-A injections, by an experienced research occupational therapist. Patients exhibiting changes in outcomes between T0 and T1 were excluded. BTX-A: botulinum toxin type A, ES: electrical stimulation.
Toxins 10 00426 g001
Table
Table 1. Changes in primary outcome measures over time.
Table 1. Changes in primary outcome measures over time.
Outcome p†
T1T2T3p *T1–T2T2–T3T1–T3
BB test3.07 ± 3.853.60 ± 4.914.67 ± 5.250.0390.4730.1200.028
ARAT-total11.33 ± 8.0311.27 ± 7.7112.73 ± 7.670.0431.0000.0360.044
ARAT-grasp2.87 ± 3.823.00 ± 3.983.27 ± 3.750.2760.8540.3340.276
ARAT-grip2.13 ± 2.071.67 ± 1.922.53 ± 1.920.1200.1410.0590.257
ARAT-gross movement5.67 ± 2.165.93 ± 2.156.27 ± 2.370.0130.0460.0960.021
ARAT-pinch0.67 ± 1.230.67 ± 0.980.67 ± 0.981.0001.0001.0001.000
Data are presented as mean ± standard deviation. Outcomes were assessed at immediately before (T1), 2 weeks (T2), and 6 weeks of BTX-A injections (T3). * Friedman test; † Post-hoc Wilcoxon signed-rank test between T1 and T2, between T2 and T3, and between T1 and T3. ARAT: Action Research Arm Test, BB: Box and Block.
Table
Table 2. Changes in secondary outcome measures over time.
Table 2. Changes in secondary outcome measures over time.
Outcome p†
T1T2T3p *T1–T2T2–T3T1–T3
Active-FE1.73 ± 0.882.00 ± 0.852.20 ± 0.940.0600.1020.1800.053
Distance-FP (cm)2.58 ± 3.123.80 ± 3.023.67 ± 2.580.2120.0230.6550.027
Repeat-FE2.07 ± 1.582.27 ± 1.163.13 ± 1.770.0070.5580.0170.008
Thumb opposition0.07 ± 0.260.13 ± 0.350.33 ± 0.620.2230.3170.1800.102
MAS-WF2.13 ± 0.351.80 ± 0.411.73 ± 0.460.0150.0250.3170.014
MAS-WE0.20 ± 0.410.13 ± 0.350.07 ± 0.260.2230.3170.3170.157
MAS-FF2.33 ± 0.491.67 ± 0.491.60 ± 0.51<0.0010.0020.3170.001
MAS-FE0.07 ± 0.260.00 ± 0.000.00 ± 0.000.3680.3171.0000.317
MRC-WF2.40 ± 0.912.40 ± 0.912.53 ± 0.830.1351.0000.1570.157
MRC-WE2.27 ± 0.882.40 ± 0.742.53 ± 0.830.0500.1570.1570.046
MRC-FF2.60 ± 0.512.60 ± 0.512.67 ± 0.490.7171.0000.3170.564
MRC-FE1.47 ± 0.641.60 ± 0.631.67 ± 0.620.0970.1570.3170.083
AROM-WF (°)46.67 ± 26.3746.67 ± 26.3749.33 ± 23.440.0501.0000.1020.102
AROM-WE (°)32.67 ± 26.0433.67 ± 25.5338.00 ± 24.550.0110.3960.0240.023
AROM-RD (°)4.67 ± 7.425.33 ± 6.406.67 ± 7.240.1740.5640.1570.083
AROM-UD (°)4.67 ± 6.405.33 ± 6.405.33 ± 6.400.3680.3171.0000.317
Grip strength (kg)0.27 ± 1.030.13 ± 0.520.33 ± 1.050.2230.3170.1800.317
QDASH56.88 ± 17.2254.65 ± 14.8853.87 ± 16.540.1620.2200.5290.058
Data are presented as mean ± standard deviation. Outcomes were assessed at immediately before (T1), 2 weeks (T2), and 6 weeks of BTX-A injections (T3). * Friedman test; † Post-hoc Wilcoxon signed-rank test between T1 and T2, between T2 and T3, and between T1 and T3. FE: finger extensor/extension, FP: fingertip to the palm, MAS: Modified Ashworth Scale, WF: wrist flexor/flexion, WE: wrist extensor/extension, FF: finger flexor/flexion, MRC: Medical Research Council, AROM: active range of motion, RD: radial deviation, UD: ulnar deviation, QDASH: Quick Disabilities of Arm, Shoulder, and Hand, °: degree.
Table
Table 3. Clinical characteristics of the study population.
Table 3. Clinical characteristics of the study population.
PatientStroke TypeLesion SideTime from Stroke (Months)Dominant HandInjection Site (Dosage in Units)
1IschemicRt.11.5Rt.FCU (25), FDP 2/3/4/5 (10/20/20/10), FDS 2/3/4/5 (20/30/25/15), FPL (25)
2IschemicRt.10.9Rt.FCU (25), FDP 2/3/4 (15/20/15), FDS 2/3/4/5 (25/25/30/15), FPL (30)
3HemorrhagicRt.13.2Rt.FCU (40), FDP 2/3/4 (10/15/15), FDS 2/3/4/5 (25/30/35/20), FPB (10), FPL (30), PT (50)
4HemorrhagicRt.17.4Rt.AP (10), BB (40), brachialis (50), FDP 2/3/4/5 (10/15/15/5), FDS 2/3/4/5 (25/35/40/15), FPL (40)
5IschemicRt.18.9Rt.AP (10), brachialis (55), FDP 2/3/4 (10/10/10), FDS 2/3/4/5 (30/40/35/20), FPL (40), PT (40)
6IschemicRt.10.2Rt.FCU (30), FDP 2/3/4/5 (10/20/10/10), FDS 2/3/4/5 (25/25/20/15), FPB (5), FPL (25), PT (20)
7HemorrhagicLt.12.6Rt.Brachialis (60), FCU (40), FDP 2/3/4 (10/10/15), FDS 2/3/4/5 (25/35/30/15), FPL (20), PT (40)
8IschemicLt.13Rt.Brachialis (60), deltoid (20), FDP 2/3/4/5 (15/15/10/10), FDS 2/3/4/5 (30/40/30/10), FPB (10), FPL (30), PT (20)
9HemorrhagicRt.10.3Lt.Brachialis (60), deltoid (20), FDP 2/3/4/5 (15/15/10/10), FDS 2/3/4/5 (30/40/30/10), FPB (10), FPL (30), PT (20)
10HemorrhagicRt.20.3Rt.AP (10), FCU (30), FDS 2/3/4 (10/15/10), FPB (10), FPL (20), lumbricals (10)
11IschemicRt.8.6Rt.AP (10), FCU (30), FDP 2/3/4/5 (5/10/10/5), FDS 2/3/4/5 (20/30/20/10), FPL (20)
12IschemicRt.7.2Rt.AP (10), BB (40), FDP 2/3/5 (10/10/5), FDS 2/3/4/5 (30/35/20/10), FPL (30)
13HemorrhagicRt.9.8Rt.AP (15), FDP 2/3/4 (5/10/5), FDS 2/3/4/5 (10/20/15/5), FPL (15)
14HemorrhagicRt.12.8Rt.FDP 2/3/4/5 (30/30/35/25), FDS 2/3/4 (10/15/15), FPL (30), opponens (5)
15HemorrhagicRt.14.9Rt.Brachialis (50), brachioradialis (20), FCU (30), FDP 2/3/4 (15/15/15), FDS 2/3/4/5 (35/40/30/20), FPL (30)
AP: adductor pollicis, BB: biceps brachii, FCU: flexor carpi ulnaris, PT: pronator teres, FDP: flexor digitorum profundus, FDS: flexor digitorum superficialis, FPB: flexor pollicis brevis, FPL: flexor pollicis longus, Rt.: right, Lt.: left.

Share and Cite

MDPI and ACS Style

Lee, J.-M.; Gracies, J.-M.; Park, S.-B.; Lee, K.H.; Lee, J.Y.; Shin, J.-H. Botulinum Toxin Injections and Electrical Stimulation for Spastic Paresis Improve Active Hand Function Following Stroke. Toxins 2018, 10, 426. https://doi.org/10.3390/toxins10110426

AMA Style

Lee J-M, Gracies J-M, Park S-B, Lee KH, Lee JY, Shin J-H. Botulinum Toxin Injections and Electrical Stimulation for Spastic Paresis Improve Active Hand Function Following Stroke. Toxins. 2018; 10(11):426. https://doi.org/10.3390/toxins10110426

Chicago/Turabian Style

Lee, Jong-Min, Jean-Michel Gracies, Si-Bog Park, Kyu Hoon Lee, Ji Yeong Lee, and Joon-Ho Shin. 2018. "Botulinum Toxin Injections and Electrical Stimulation for Spastic Paresis Improve Active Hand Function Following Stroke" Toxins 10, no. 11: 426. https://doi.org/10.3390/toxins10110426

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop