Comparisons of Learning Effectiveness of Therapeutic Motion Techniques: Practicing with an Educational Hemiplegic Robot Arm versus Practicing with Other Students
Round 1
Reviewer 1 Report
Comments and Suggestions for AuthorsDear Authors,
I have carefully read your article entitled Comparisons of Learning Effectiveness of Manual Therapy for Different Symptoms: Practice with an Educational Hemiplegic Arm Robot Versus Practice with Students. It is essential and efficient work, as it is a practical application beyond the academic environment. This article compares the effects of practical learning of students using an educational robotic arm (SAMO and already patented) simulating a patient and applied to learning therapeutic movement techniques. Two study groups are formed: (i) one group using the robot arm and (ii) another group using conventional practices. A machine learning system using Support Vector Machine (SVM) and Random Forest (RF) autonomously measures each student's ability after completing this training.
Therefore, I favor publishing this article in the Applied Science Journal. However, I think I can help the authors in revising this article in the following aspects:
1. In the article's title, start the main words in capital letters instead of lowercase letters.
2. In the Keywords. The first word, "patient," should start in capital letters.
3. End the Introduction with a brief paragraph stating what will be done in the rest of the article. For example, " Section 2 it is described... In Section 3 it is presented...".
4. Make the originality of the proposed method more straightforward in the Introduction. For example, before the end of the Introduction, start a paragraph like this: "In this sense, the originality of this work consists of (i) originality I... (ii) originality II, among others."
5. (Page 5, lines 167-168) Change "support vector machine (SVM)" to "Support Vector Machine (SVM)." Do the same with the acronym RF. When an acronym appears together with its meaning, the meaning of the acronym should be highlighted by starting the words in capital letters.
6. A full stop is missing in the caption of Tables 1, 2, and 3.
7. Leave Figure 3 and its respective caption on the same page for organizational reasons.
8. A full stop is missing in the paragraph on line 209 (Page 6).
9. Make clear in the article the meaning of the variables df, F, p, and n^2 that appear in Table 2.
10. What does the red curve represent in the graphs of Figure 3? And what does the blue curve represent? A caption is missing for this.
11. (Page 8) The first paragraph of Section 4 is too long. Perhaps it would be better to divide it into two.
12. This is just a suggestion. Please include a list of Abbreviations (in alphabetical order) at the end of the article for the acronyms SVM, RF, SAMO, ANOVA, JASP, and JSPS.
Best regards,
The Reviewer.
Author Response
Thank you for your review and guidance, despite your busy schedule. Regarding the comments we received, we have summarized the answers as follows. Thank you for your consideration.
Reviewer #1
Comment 1
In the article's title, start the main words in capital letters instead of lowercase letters.
Response to Comment 1
Indeed, as you mentioned, we had originally written the title in lowercase. We have now capitalized the main words. Therefore, it has been changed to “Comparisons of Learning Effectiveness of Therapeutic Motion Techniques for Different Symptoms: Practice with an Educational Hemiplegic Arm Robot Versus Practice with Students”.(Page 1, Line 2-4)
Comment 2
In the Keywords. The first word, "patient," should start in capital letters.
Response to Comment 2
The keywords were originally written in lowercase; we have capitalized them according to your recommendation. Patient robot; Simulation robot; Spasticity; Rehabilitation; Therapeutic motion techniques; Exercise therapy education. (Page 1, Line 32-34)
Comment 3
End the Introduction with a brief paragraph stating what will be done in the rest of the article. For example, " Section 2 it is described... In Section 3 it is presented...".
Response to Comment 3
We have added the following explanation of the verification methods after stating the objectives of the study: With regards to the verification methods, the study assessed the changes in the features of motor therapy techniques before and after practice, and evaluated whether students' techniques approach those of therapists after practice. (Page 2, Line 67-69)
Comment 4
Make the originality of the proposed method more straightforward in the Introduction. For example, before the end of the Introduction, start a paragraph like this: "In this sense, the originality of this work consists of (i) originality I... (ii) originality II, among others."
Response to Comment 4
Thank you for the advice. We have added information regarding the uniqueness and originality of the study, as follows: "The uniqueness of this study lies in constructing an educational system that assesses clinical knowledge and skills from the early stages of students’ training by visualizing the acquisition of rehabilitation techniques. Simulation robots with such capabilities for motor therapy education are not yet used in educational settings worldwide. Furthermore, with the advancement of education utilizing simulation robots, students are expected to acquire proficient techniques at an earlier stage, even before obtaining their qualifications." (Page 2, Line 70-75)
Comment 5
(Page 5, lines 167-168) Change "support vector machine (SVM)" to "Support Vector Machine (SVM)." Do the same with the acronym RF. When an acronym appears together with its meaning, the meaning of the acronym should be highlighted by starting the words in capital letters.
Response to Comment 5
We have changed the notation for “support vector machine” and “random forest” by capitalizing them to “Support Vector Machine” and “Random Forest”, respectively. (Page 5, Line 188-189)
Comment 6
A full stop is missing in the caption of Tables 1, 2, and 3.
Response to Comment 6
We have added a period at the end of the sentence.
Comment 7
Leave Figure 3 and its respective caption on the same page for organizational reasons.
Response to Comment 7
We have placed Figure 3 and its caption on the same page.
Comment 8
A full stop is missing in the paragraph on line 209 (Page 6).
Response to Comment 8
Indeed, the period was missing; we have now added it.
Comment 9
Make clear in the article the meaning of the variables df, F, p, and n^2 that appear in Table 2.
Response to Comment 9
We have added the following explanations for the values in Table 2: df: degrees of freedom, F: F-statistic, p: p-value, η²: Eta-squared. (Page 6, Table 2)
Comment 10
What does the red curve represent in the graphs of Figure 3? And what does the blue curve represent? A caption is missing for this.
Response to Comment 10
We have added the following explanations for the terms in Figure 3: Blue-outlined circles: mean values of the Robo group, red-outlined circles: mean values of the Human group, vertical bars: standard deviation. Blue solid line: the slope between the pre- and post-intervention values in the Robo group, red solid line: the slope between the pre- and post-intervention values in the Human group. (Page 7, Figure 3)
Comment 11
(Page 8) The first paragraph of Section 4 is too long. Perhaps it would be better to divide it into two.
Response to Comment 11
Thank you for the advice. Following your suggestion, We have separated them into two parts.
Comment 12
This is just a suggestion. Please include a list of Abbreviations (in alphabetical order) at the end of the article for the acronyms SVM, RF, SAMO, ANOVA, JASP, and JSPS.
Response to Comment 12
Thank you for the suggestion. We have created a list of abbreviations for SVM, RF, SAMO, ANOVA, JASP, and JSPS, and arranged them in alphabetical order.
Abbreviations:
ANOVA: Analysis of Variance
JASP: Jeffreys’s Amazing Statistics Program
JSPS: Japan Society for the Promotion of Science
SAMO: Samothrace
SVM: Support Vector Machine
RF: Random Forest
(Page 10-11, Line 390-396)
Reviewer 2 Report
Comments and Suggestions for AuthorsRefer to attached file for comments.
Comments for author File: 
Comments.pdf
ok
Author Response
Reviewer #2
Comment 1
How did the researchers ensure that the random assignment of students to the Robo and Human groups did not introduce any biases that could affect the results?
Response to Comment 1
We have added the following to the text regarding randomization:
“Due to the random assignment of students to the Robo and Human groups, there was a possibility of introducing biases that could affect the results. However, since the students had studied the same content, the impact of these biases was expected to be minimal. Additionally, to reduce the impact on measurement results before and after practice, randomization was not only employed to divide the subjects, but also to create six patterns of muscle tone levels (A-F) as shown in Figure 1, with random assignment within these patterns.” (Page 3, Line 103-109)
Comment 2
Given that there were no significant changes in most kinematics data before and after practice, how did the researchers justify the use of machine learning techniques to evaluate the effectiveness of the robotic arm?
Response to Comment 2
We have added the following justification for using machine learning: “While simple statistical analysis of the data before and after practice can determine how each feature has changed, it cannot assess whether the students' techniques have comprehensively approached those of therapists. Therefore, this method was also implemented.” (Page 5, Line 191-194)
Comment 3
How reliable are the identifiers created by the Support Vector Machine (SVM) and Random Forest (RF) algorithms in distinguishing between students' and therapists' therapeutic motion techniques? What steps were taken to validate these models?
Response to Comment 3
We have added the following regarding the reliability of the student and therapist evaluations: For the SVM and RF classifiers, after supervised learning was conducted, the accuracy was assessed using test data. The results confirmed an accuracy of 1.000 for SVM and 0.889 for RF. (Page 8, Line 260-263)
Comment 4
Why was there no control group that received no practice or a different type of intervention, and how might the inclusion of such a group have strengthened the study’s conclusions?
Response to Comment 4
We agree that including control groups that do not undergo practice, or practice groups that only watch videos, would be interesting and might reveal effective methods beyond the robot-based approach. However, due to ethical concerns about not providing learning opportunities to students, setting up such groups is ethically challenging in educational settings and was therefore not performed in this study.
Comment 5
How did the researchers account for individual differences in students' prior knowledge and skill levels in therapeutic motion techniques, which might influence the outcomes of the study?
Response to Comment 5
We have noted that it is possible that the students' prior knowledge levels could have influenced the results of the skill tests. This point is addressed as a limitation in the study, as follows:
“The third limitation concerns the students' individual knowledge levels regarding the practice methods. Although students who had received the same educational content were targeted, there was no indication of differences in the amount of knowledge provided. However, it was not determined whether students have fully acquired this knowledge, and potential differences in individual knowledge levels were unclear. Future research should include evaluations of prior knowledge to enable more tailored assessments for students.” We also plan to evaluate and stratify prior knowledge in future studies to address this issue. (Page 9, Line 309,327-333)
Comment 6
The paper mentions that SAMO-A and SAMO-B have slightly different mechanical expressions of muscle tension. How might these differences affect the comparability of results between the Robo and Human groups?
Response to Comment 6
In the Robo group, SAMO-B was used during practice, providing more opportunities to interact with the robot than in the Human group. However, during practice, only a moderate pattern of muscle tone was used, while multiple patterns of muscle tone were included in the evaluations before and after practice, meaning that assessments also included settings not experienced during practice. Additionally, the instructions and criteria for evaluation were the same for both the Robo and Human groups. Therefore, we believe that this does not affect the comparison of results.
Comment 7
The instructions given to students during practice were abstract (e.g., "move slowly and gently"). How might more specific instructions have impacted the study's findings?
Response to Comment 7
You are right in pointing out that the practice instructions in this study were rather abstract. It is indeed interesting to consider the potential effects of more specific instructions on practice outcomes. However, overly specific instructions or an excess of information may also hinder the learning process. Therefore, in this study, the instructions were abstract but aligned with the recommendations from Japanese textbooks. That said, we would like to investigate the effects of more specific or information-rich instructions in future research.
Comment 8
What evidence, if any, did the researchers provide to suggest that the skills acquired through practice with the SAMO robot would be retained long-term or applied effectively in real-world clinical settings?
Response to Comment 8
This study did not investigate whether the skills acquired are retained long-term. However, this is an important aspect in student education, and I intend to address it as a topic for future research.
Comment 9
Considering the involvement of students and therapists, what ethical concerns might arise from the use of the SAMO robotic arm in educational settings, particularly regarding the potential for over-reliance on technology?
Response to Comment 9
We have added the following point to the text:
“Lastly, the use of simulation robots like SAMO allows for repeated, consistent training before working with actual patients, which is one of the advantages of robot-assisted practice. However, practicing solely with robots makes experiencing essential human interaction aspects, such as how to talk to patients, handle their arms, and move them gently, challenging. As a solution, features that enable dialogue can be included. In recent years, the development of communication robots has advanced, and many robots now have conversational capabilities, making this solution possible to apply [23]. In any case, both the benefits and drawbacks of robots should be considered, ensuring they are not over-relied upon. Education and explanations on proper usage, including mindful consideration of how the robot's actions translate to patient care, are necessary.” (Page 9-10, Line 334-343)
Comment 10
How generalizable are the study's findings to other forms of therapeutic motion techniques or to students in different educational contexts?
Response to Comment 10
We have added the following regarding the generalizability of the study:
“Although this study focuses on motor therapy for the elbow joint, its findings can be generalized to other joints, such as the fingers, wrist, hip, knee, and ankle joints. Additionally, from the perspective of using simulation robots, the study's approach could be applicable to other educational content as well. Therefore, this research can potentially be utilized and expanded in various directions in the future.” (Page 10, Line 345-349)
Comment 11
The study used specific settings for muscle tension during the tasks. How might different muscle tension settings (outside of mild, moderate, severe) influence the outcomes and conclusions of the study?
Response to Comment 11
During the practice in this study, only one type of muscle tone was used. Although practice with multiple types of muscle tone was considered, doing so would have resulted in the same practice conditions and pre/post measurements for the Robo group, making it easier for learning effects due to familiarity to occur. Therefore, practice was limited to one type of muscle tone. However, since each patient has different muscle tone, ideally, practice should involve various muscle tone conditions. In future studies, I would like to verify the effects of practice with multiple types of muscle tone. (Page 10, Line 343-345)
Comment 12
The paper suggests further research on the number of practices sets and instruction methods. What specific experimental designs could address these gaps, and what hypotheses should be tested in future studies?
Response to Comment 12
As noted in sections '7', '8', '9', and '11', there are numerous hypotheses that need to be tested, such as the learning effects when practicing with various muscle tone conditions and the differences in learning outcomes based on the amount of guidance or the specificity of instructions. I intend to continue investigating these areas.
Comment 13
Honestly, this paper needs to beef up on the content especially the literature review and introduction section. The reference list needs to have at least 40 recent references to make this manuscript more worthwhile for publication. I have proposed some recent publications which you could include in your literature review.
Response to Comment 13
Thank you for your suggestion. We have included the references you provided. (Page 9, Line 340)
We have summarized the answers as described above. Thank you for your consideration.
Round 2
Reviewer 2 Report
Comments and Suggestions for AuthorsNo more comments. Ready for acceptance and publication.
