Preliminary Evaluation of an Adaptive Robotic Training Program of the Wrist for Persons with Multiple Sclerosis

Round 1

Reviewer 1 Report

The purpose of this study was to use a robotic device to develop an adaptive and individualized raining program of the distal upper extremity for individuals with multiple sclerosis (MS).

General comment:

First, I’d like to congratulate the authors to a nice and well-written manuscript. The topic is relevant and the potential of using robot-assisted training equipment for individuals with MS is relevant to explore further. The background is relevant and comprehensive, the theoretical framework introduces and describe the theory that explain the research problem and the results are presented in a clear, concise and nice way. Limitations are discussed and the conclusion draws out the implications for rehabilitation for the paper.

However, there is a little bit of mismatch between the study aim and conclusions drown. While in the abstract, the authors write that the purpose of the study is to use a robotic device to develop an adaptive and individualized training program of the distal upper extremity for individuals with multiple sclerosis (MS), the conclusion states that the results “provide proof-of-concept that motor accuracy and muscular strength can be improved by adaptive 22 robotic rehabilitation in an MS population”. Given the feasibility character of the study (proof of concept), I suggest to either widen the study aim to include both method development and preliminary evaluation, or to avoiding expressions such as the last sentence in the abstract (inserted above). An alternative could be to simply conclude that the robot-assisted training equipment is demonstrated to be a feasible method that has potential to improve distal upper extremity motor accuracy and muscular strength in patients with MS.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

The manuscript presents this was the first study to deliver an adaptive training program to the upper limb in an MS population. However, the adaptive training algorithm, study motivation, and results are limited, and do not clearly present their contribution beyond existing studies and literature on neurorehab. Clarifying the contribution of the manuscript, placing the work in context with the large existing body of literature on adaptive controllers for stroke rehab, and identifying the neurorehab mechanisms targeted with the study will improve the manuscript, but the manuscript will ultimately be limited by the intervention design, experimental methods, and results. This would make for a very strong conference submission (ICORR, BioROB, etc).


In the abstract: the sentence “Grip endurance increased by 60% post-intervention” is referring to a finding that was not statistically significant. It would be preferable to restrict the claims in the abstract to significant results. The Mann-Whitney U Test is used to compare the trained and control limbs at T1. It would be more appropriate to use a paired test, since the controls are the other limbs of the same individuals.

How does this approach of modulating assistance/resistance differ from the body of work in post-stroke and post-SCI rehabilitation, especially that for the wrist? Adaptive controllers have been studies for several years, with some early works reviewed by Blank et al: Blank, Amy A., et al. "Current trends in robot-assisted upper-limb stroke rehabilitation: promoting patient engagement in therapy." Current physical medicine and rehabilitation reports 2.3 (2014): 184-195.

Similarly, is there a neurorehab mechanism targeted by the study beyond those provided by Kleim and Jones? Kleim, Jeffrey A., and Theresa A. Jones. "Principles of experience-dependent neural plasticity: implications for rehabilitation after brain damage." (2008). The discussion section addresses many of the potential issues/benefits to the experiment design with trained and untrained arms. However, the study does not compare robotic rehabilitation to standard of care. As expected, some rehab is better than no rehab, but it is unclear if the proposed adaptive method is better than standard of care, especially since there were no benefits found in the transfer of learned skills. This is an issue facing the field, with studies such as Rodgers et al identifying limitations of robotic rehab. Rodgers, Helen, et al. "Robot assisted training for the upper limb after stroke (RATULS): a multicentre randomised controlled trial." The Lancet 394.10192 (2019): 51-62.


The introduction makes the case for robotic devices to be used for real time performance measurements and assessment of performance, but the main metrics used in the study are in the task position space (task error and figural error). Are there other types of robotic measures, such as movement smoothness, that are applicable to the MS population? Some smoothness measures have been shown to be relevant ten years ago: O. Celik et al., “Normalized movement quality measures for therapeutic robots strongly correlate with clinical motor impairment measures,” IEEE Trans. on Neural Systems and Rehabilitation Engineering, vol. 18, no. 4, pp. 433–444, 2010. How does the training protocol compare to standard of care (more, less, the same)? Similarly, how was 80% ROM selected? Is there a difference between training for increasing quality of motion within existing ROM and increasing ROM?

Did wrist ROM assessed along anatomical axes have an impact on the figure 8 shape? Some work has shown some complexity to wrist ROM beyond the anatomical axes: J. J. Crisco et al., “The mechanical axes of the wrist are oriented obliquely to the anatomical axes,” The J. of Bone & Joint Surgery, vol. 93, no. 2, pp. 169–177, 2011.   Is there a particular mechanism the resistive training was targeting? How might that compare to training that amplifies error (error augmentation)?
The limitations section of the paper identifies that greater reps at higher levels of resistance could be done to increase participant strength gains, but then suggests that other sensorimotor improvements could be had if the adaptive algorithm were changed. The manuscript could suggest particular modalities, goals, neurorehab mechanisms to be engaged, etc, to support this claim.

How do you know it is a high dose, high frequency training?

why there is no change in the size of the figure even after weeks of training? "The size of the tracking figure was scaled to a maximum of 80% of the individual’s active ROM collected in T0."

It is highly recommended to provide the specific control law (i.e., the equation) in addition to the vague description. "The training force was implemented as a virtual spring that pulled the handle of the robot towards the moving target"

How was the level of assistant/resistant determined for the first three laps exactly? " The level of assistance/resistance for the first 3 laps was based on the amount of FE assessed in absence of training forces (active tracking). "

What is the unit of modulation? what do a and b stand for?

What are the empirically tested safety conditions? How did you test those? It is not clear at all. "Because of empirically tested safety conditions (maximum ROM assessment), positive value of mod-ulationnew reached saturation when greater than 1, negative values when lower than -0.2."   Eq (5) needs a detailed explanation. In its current form, it is very difficult to understand what is happening especially if modulation <0. There is no explanation of anything. If, -d*theta_dot is applied to the arm, that will make the system unstable. It seems that the tau_dof when modulation <0 may make the system unstable, indicating it may be dangerous to use such torque for rehabilitation. It is an engineering journal basically. Just needed resistance training? What does that mean? They can reach any of the targets without any assistance almost from the beginning? " On average, training data revealed that participants performed in the resistive modality (modulation < 0) for 90% of the entire training protocol." It is highly recommended to delete FE from Fig. 2. Since there was no significant difference.

For Fig. 3, the individual data can be provided as the spread of data with the medians instead of providing all subjects' data on separate columns. It is highly recommended to compare the TE of the trained limb and that of the untrained (control) limb at T0. It is missing. Thus, it is not possible to see whether there is any effect of training or not. "Tracking error in the
trained limb improved significantly for each participant from T0 to T1 (Fig 3). There was
a significant reduction in tracking error, with an average decrease of 0.97 ± 0.13º (p = 0.03;
3.77 ± 2.12º at T0, 2.79 ± 1.99º at T1). No significant improvements were found in the con-
trol limb (p = 0.87; 2.91 ± 2.12º at T0, 3.45 ± 2.51º at T1). No significant differences were
found when comparing group averages of the trained vs control limb (p = 0.25) at T1."

For the same reason, it is highly recommended to compare FE of trained limb and that of untrained (control) limb at T0. "Although both limbs significantly improved, there were significant differences found between trained and control limbs (p < 0.001), as trained limbs had less figural error at T1. "

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 3 Report

The paper is well written and the message is clear but some issues should be addressed:

• Introduction still has room for improvement. The authors have stated that ‘there is a paucity of research on robotic neurorehabilitation for persons with MS, particularly for the upper limb’. However, there are many upper limb rehabilitation robots that are used with other types of patients, such as stroke patients. And some of these robots use adaptive training modes. Therefore, I think they should be added .
• The validation would have been stronger if the results had been compared with those obtained with a traditional therapy without robotic devices. Also, as the authors point out, it would have been nice to do test with more patients.
• It is mentioned that a and b are experimental parameters, but they should be better detailed. The process carried out to obtain these values is missing.
• In the analysis of Figure 3, it is said that ‘Tracking error in the trained limb improved significantly for each participant from T0 to T1’ and ‘No significant improvements were found in the control limb’. However, subjects S01 and S03 worsened considerably. I think the data should be better analyzed and justify this deterioration.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 4 Report

General Comments

This paper presents an interesting study on the use of upper extremity robotic training to improve wrist function in individuals with multiple sclerosis. The training paradigm of first providing real-time assistance until a minimum level of performance is reached and then switching to real-time resistance seems to be novel as well as effective. These findings should be of interest to the robotic rehabilitation research community.

There are two general areas where I believe the manuscript could use improvement:

1) Unclear descriptions of primary performance measures (TE and FE).

2) Seemingly conflicting descriptions or comments at various places in the Methods and Discussion.

Specific examples of these two general issues are provided below in the Specific Comments section.

Specific Comments

Introduction

“Despite challenges facing robotics that there are no further improvements than that of standard care [10]” This phrase does not make sense – please rewrite and clarify.

Methods

“TE was defined as the mean angular distance in degrees between end-effector and target position at each sample point.” However, Equation (2) defining TE appears to be a distance in units of length, not degrees or radians, consistent with x and y in Equation (1) being translational coordinates. Please clarify what TE means physically.

Equation (3) for the Figural Error FE is also confusing. Calculation of distAB(i) appears to involve fixing the value of i and then cycling through all the values of j to find the minimum of abs(Ai-Bj). If so, then there should be j rather than an i under the word “min.” The same issue exists for distBA(j). It is also unclear what A and B are in this equation. The text says that they are time series trajectories, but the time series trajectories defining the end point position should be x(t) and y(t).

Clarification of what TE and FE are, meaning both how they are computed and what they mean physically, is critical since these are the quantities used to evaluate improvement in performance.

Results

As shown in Figure 3, two of the seven subjects (or 29%) were non-responders to the intervention, exhibiting higher tracking errors in both the trained and control sides following training.

Also as shown in Figure 3, for three of the seven subjects (or 43%), improvements in the trained wrist also corresponded to improvements in the untrained control wrist, suggesting that some type of positive neurological carry-over effect between sides may have occurred.

Discussion

“In this work we presented a proof of concept of a robotic rehabilitation program for the improvement of hand and wrist motor control in individuals with MS.” It seems like only wrist, and not hand, motor control was studied. Please clarify.

“Training results showed that participants seemed to improve their performance across sessions in terms of spatial accuracy (FE), while TE tended to get worse.” This result does not make sense to me, since it’s not clear how Tracking Error can get worse while Figural Error (which I interpret as a measure shape error) got better. Please clarify. This paragraph goes on to state, “FE considered only the spatial component of tracking, conversely TE considered the tracking speed.” This explanation does not make sense to me, since tracking speed is not a quantity that appears in Equation (2) for TE. Again, please clarify.

“In absence of assistive/resistive torques, figural error reduced by 46%, while tracking error decreased by 26%. These significant reductions in both figural and tracking error demonstrate improvements in task performance.” These statements that both TE and FE went down with training seem to contradict the earlier statement in the same paragraph that “Training results showed that participants seemed to improve their performance across sessions in terms of spatial accuracy (FE), while TE tended to get worse.” Please clarify.

“However, our underestimation of MS individuals’ performance took the algorithm to converge quickly towards the resistive modality. Because of this limitation, the training resulted mainly resistive for all participants.” This comment suggests that the assistive phase of the adaptive training paradigm may not have been very helpful. Given the emphasis of the manuscript on the benefits of the adaptive training paradigm used here, please comment in the Discussion on whether or not the assistive portion of the protocol is really necessary.

“Only radial deviation showed significant improvements in the control limb. However, the other directions demonstrated an average increase of 22%, 19%, 42% and 56%, for flexion, extension, radial and ulnar deviation, respectively, which could have clinical or functional significance.” These two sentences seem to contradict each other. Only radial deviation showed a significant improvement, but then all four directions showed large improvements. Both cannot be true at the same time. Please clarify.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Round 2

Reviewer 3 Report

The manuscript has been revised according to the reviewers. And from my point of view, the paper should be accepted for publication.