Development of an Autonomous Driving Smart Wheelchair for the Physically Weak
, Je-Seong Kwon 1, Jeong-Hak Lim 1,*, A-Hyeon Kim 2, Su-Jin Baek 2 and Jong-Wook Kim 2,*Round 1
Reviewer 1 Report
The Authors presented the developed electric wheelchair with autonomous driving function in order to help the physically weak. There is a lack of novelty, but the topic is interesting and the entire manuscript presents the real application of autonomous driving functions, voice control, reinforcement learning, path planning, and environment mapping. The additional comments and questions to the authors:
- First of all, the presented graphics are not acceptable in the manuscript. please use high-quality graphics. Some of the labels are not readable due to poor quality.
- Captions do not describe what is presented in figures. (e.g. Fig. 13, 15)
- An elastic band technique has been used to provide smooth trajectory, but what about the acceleration and jerk limitation? in the wheelchair case, it is very important.
- There is a lack of description of the used global path planning algorithm.
- Due to this paper is about the development, the following part: "Furthermore, when the robot sends a value to cmd_vel, which is the ROS
topic that controls the velocity of the robot while driving to the entered
destination, it checks whether the position value on the local path is the
same as the current position value and minimizes the error by the PI
control. For this, the ticks per meter values of the two motors are set
differently in the Python code of the motor driver that controls the motors
to calculate the encoder values, so that the robot’s position can be
determined more accurately."
must be described in detail. - In my opinion, global path optimization is not safe enough. What if the obstacles were not present during the mapping procedure? Global path planning may give a path with potential collision. Did the author consider dynamic obstacles? Maybe there should be a connection between global and local path planning algorithms, i.e.g global one gives a checkpoint to reach by local path planning algorithm.
Author Response
First of all, thank you for reviewing this paper. Thanks to your review, we are sure that the paper has been elevated by making corrections such as improving the quality of the graphics and detailing the picture captions. We wrote the corrections to the contents of your review below as an answer. Please review the revised paper file along with the review response. We look forward to your positive review.
The Authors presented the developed electric wheelchair with autonomous driving function in order to help the physically weak. There is a lack of novelty, but the topic is interesting and the entire manuscript presents the real application of autonomous driving functions, voice control, reinforcement learning, path planning, and environment mapping. The additional comments and questions to the authors:
- First of all, the presented graphics are not acceptable in the manuscript. please use high-quality graphics. Some of the labels are not readable due to poor quality.
Figures 3, 4, 11, 14 are replaced with higher quality graphics. The labels in Figures 14 and 15 are enlarged to raise readability.
The entire graphic of the paper has been modified to high quality. You can check that in the edit file.
- Captions do not describe what is presented in figures. (e.g. Fig. 13, 15)
-> The description of each graph in fig.13 has been added to the caption.
Descriptions of each symbol (O,S,A,B,C,D,E) of fig. 14 and 15 have also been added to the caption.
In Figures 13 to 15, labels in the figures are explained and added in each caption.
- An elastic band technique has been used to provide smooth trajectory, but what about the acceleration and jerk limitation? in the wheelchair case, it is very important.
-> Acceleration settings are important for wheelchairs to drive safely. By modifying the acceleration parameters of the elastic band technique used in this paper, acceleration was applied so that when the robot stops, it stops slowly. However, the jerk restriction parameter was not set separately because it was not included in the elastic band technique parameter.
This content is added to Section 4.2.4 (Page 11).
- There is a lack of description of the used global path planning algorithm.
-> A planner is required to generate a driving path of the robot. The planner has a global path planner and a local path planner. We use the navfn algorithm as a global path planning algorithm, which finds the shortest path from one starting vertex to all other vertices based on the Dijkstra algorithm.
This description is added to Section 4.2.3 (Page 8).
- Due to this paper is about the development, the following part: "Furthermore, when the robot sends a value to cmd_vel, which is the ROS topic that controls the velocity of the robot while driving to the entered destination, it checks whether the position value on the local path is the same as the current position value and minimizes the error by the PI control. For this, the ticks per meter values of the two motors are set differently in the Python code of the motor driver that controls the motors to calculate the encoder values, so that the robot’s position can be determined more accurately." must be described in detail.
-> A detailed description of how to calculate the robot's location has been added to Section 4.2.4 with Eqs. (5) and (6) and Fig. 10. (Pages 10-11).
- In my opinion, global path optimization is not safe enough. What if the obstacles were not present during the mapping procedure? Global path planning may give a path with potential collision. Did the author consider dynamic obstacles? Maybe there should be a connection between global and local path planning algorithms, i.e.g global one gives a checkpoint to reach by local path planning algorithm.
-> Entering the destination creates a global path to the destination, creates a regional route according to the global path, and the robot moves, continuing to regenerate the local path according to the situation. Then, if unmapped obstacles suddenly appear, the global path will be recalculated and created in consideration of the obstacles and the local path will changed also. In addition, global and local path are connected by a navigation node called move base. This partial description and Figure 7 is added to Section 4.2.3 (Page 9).
Thank you for your review. We look forward to your review of this revision.
Author Response File: 
Author Response.docx
Reviewer 2 Report
The term robot in the title is not chosen correctly. The paper presents just another solution for a quasi-autonomous vehicle, which, among other things, could be tried to be used for people with disabilities.
Some solutions of automated wheelchairs are presented in the introduction. It is recommended to add solutions from the literature that are closer to what is presented in the paper. Namely smart wheelchairs, with driving techniques based on artificial intelligence.
The wheelchair described in the paper is not completely autonomous. He is led with human decision.
The technical solutions for moving the chair are known in the literature.
The paper presents experimental results of the practically implemented solution, highlighting certain facilities.
It is recommended to list the problems encountered in the movement of the chair and to present future perspectives for solving them.
Author Response
First of all, thank you for your review of our paper. I think the title has been made clearer by reflecting your opinion, and the part about background research and future prospects has also been materialized. We have responded to the corrections made to your review below. Please review the revised paper file along with the review response. We look forward to your positive review.
The term robot in the title is not chosen correctly. The paper presents just another solution for a quasi-autonomous vehicle, which, among other things, could be tried to be used for people with disabilities.
-> We have modified the title of the paper as follows.
Development of An Autonomous-Driving Smart Wheelchair Robot for the Physically Weak
Some solutions of automated wheelchairs are presented in the introduction. It is recommended to add solutions from the literature that are closer to what is presented in the paper. Namely smart wheelchairs, with driving techniques based on artificial intelligence.
-> Added research on self-driving wheelchairs controlled by artificial intelligence. References 12 and 13 are those.
The wheelchair described in the paper is not completely autonomous. He is led with human decision.
The technical solutions for moving the chair are known in the literature.
The paper presents experimental results of the practically implemented solution, highlighting certain facilities.
It is recommended to list the problems encountered in the movement of the chair and to present future perspectives for solving them.
-> Because the elastic band technique generates the bubble in the forward direction, it took a lot of time for the robot to reach the target position or did not arrive well. Among the parameters of the elastic band technique, xy_goal_tolerance is the parameter that sets a distance value to reach a target position. This parameter was adjusted so that the robot could arrive even if the target position is set a little far from actual one. Because of this limitation, the authors are going to develop and add to the robot a finishing algorithm near the target that slightly moves the robot back and aligns the target location to accurately reach the destination.
This sentence is added at the end of Section 5 (Page 18).
Thank you for your review. We look forward to your review of this revision.
Author Response File: Author Response.docx
Round 2
Reviewer 1 Report
The manuscript has been significantly improved and in my opinion, it can be accepted in the current form.
Reviewer 2 Report
The authors responded satisfactorily to the comments.
This manuscript is a resubmission of an earlier submission. The following is a list of the peer review reports and author responses from that submission.
Round 1
Reviewer 1 Report
The authors of the submitted article apply the reinforcement learning based chatbot and autonomous driving to the wheelchair. According to the article's name, it should impact the user convenience, but no experimental data support this.
The chatbot uses reinforcement learning to learn the preferred position of the wheelchair to avoid the necessity to set the position every time. However, it requires confirmation every time. I cannot see the improvement when only three options are available. Moreover, the article is not describing the context of the learning. Takes the chatbot also the time into account? I am missing an evaluation of the user experience. Is it more convenient to chat with the wheelchair than to command it with defined commands?
Autonomous driving uses gMapping and rubberband path planning ROS packages. The evaluation of autonomous driving is based on the effectiveness measured as "distance" to an optimal path. The authors stated that autonomous driving is better than manual because it goes nearer to obstacles. It is valid only if the goal is to minimize traversed trajectory. From a personal experience during the project that we solved once, I know that this optimization can scare the user a lot. Again, evaluation of the user convenience is missing entirely.
Moreover, there is an evaluation of the bump climbing ability in autonomous mode. The evaluation is missing basic information like how many trials were performed or a comparison to the manual mode. Is the wheelchair able to climb all the bumps in manual mode?
Formally, equations are not explained well. What is gamma in equation 2 or alpha in equation 4? How is the state S described? How is the reward computed?
The citation of gmapping is missing.
Conclusion:
I don't recommend publishing the article in its current form. The user should be involved in the evaluation of the system, description of the reinforcement learning needs improvement, experiments must be better described and compared to the variant without improvements (chatbot and/or autonomous driving).
Author Response
Reviewer 1:
The authors of the submitted article apply the reinforcement learning based chatbot and autonomous driving to the wheelchair. According to the article's name, it should impact the user convenience, but no experimental data support this.
Response) The authors are very grateful for your good comments on our paper. The user convenience survey is a study targeting humans, and in particular, the self-driving electric wheelchair developed in this study is in need of an IRB (Institutional Review Board) acquisition because the target audience is the weak. Normally, it takes more than 30 days to receive an IRB acquisition in Korea, where we are located. For this reason, it is not possible to receive an IRB acquisition within 10 days and proceed with a follow-up study. Therefore, we would like to finish this study by supplementing other contents without usability evaluation, and proceed with usability evaluation through follow-up studies.
The chatbot uses reinforcement learning to learn the preferred position of the wheelchair to avoid the necessity to set the position every time. However, it requires confirmation every time. I cannot see the improvement when only three options are available. Moreover, the article is not describing the context of the learning. Takes the chatbot also the time into account? I am missing an evaluation of the user experience. Is it more convenient to chat with the wheelchair than to command it with defined commands?
Response) As to the confirmation of the recommended position every time, confirmation can be programmed as an option or minimized if the user doesn’t want. That is, it can be an alternative to say “no” only when the posture is uncomfortable to the user, which means the recommended posture is default.
The three posture options such as upright, small incline, and large incline can be extended to more postures and also be expressed as postures defined by angle values such as 15 degree posture, 30 degree posture, etc. For more diverse postures, the same reinforcement learning methodology can be applied without problem.
If my understanding is correct, the ‘context of learning’ is set and Python and Soar agent is used for executing RL.
The chatbot can be programmed to take time into account if response time can be a measure of abnormality. For example, if the user does not answer within the specified time after the chatbot asks, it can ask again or contact the guardian of the user after several trials of asking.
Evaluation of the user experience will be performed later after careful preparation as is mentioned above.
Commanding the robot via chatbot is more convenient when the user has difficulty in manipulating joy stick or pushing buttons due to blurred eyes or shaking hands.
Autonomous driving uses gMapping and rubberband path planning ROS packages. The evaluation of autonomous driving is based on the effectiveness measured as "distance" to an optimal path. The authors stated that autonomous driving is better than manual because it goes nearer to obstacles. It is valid only if the goal is to minimize traversed trajectory. From a personal experience during the project that we solved once, I know that this optimization can scare the user a lot. Again, evaluation of the user convenience is missing entirely.
Response) The distance to obstacle can be adjusted by changing inflation radius value in the ROS navigation package. When a user feels scary in passing an obstacle closely, this can be reflected in the navigation package by adding another reinforcement learning function.
Moreover, there is an evaluation of the bump climbing ability in autonomous mode. The evaluation is missing basic information like how many trials were performed or a comparison to the manual mode. Is the wheelchair able to climb all the bumps in manual mode?
Response) The bump climbing ability evaluation was performed 3 times for each bump, and it was confirmed that the same results were obtained compared to the manual mode. You can find out more about this on page 12 of the revised paper.
Formally, equations are not explained well. What is gamma in equation 2 or alpha in equation 4? How is the state S described? How is the reward computed?
Response) Gamma and alpha are explained and added after Eqs. 2 and 4. The reward is computed as 1 for “yes” answer and -1 for “no” answer.
The citation of gmapping is missing.
Response) The citation of gmapping is added as [19].
Conclusion:
I don't recommend publishing the article in its current form. The user should be involved in the evaluation of the system, description of the reinforcement learning needs improvement, experiments must be better described and compared to the variant without improvements (chatbot and/or autonomous driving).
Response) As previously written in the reviewer's comments, each revision is also reflected in the content of the paper. Due to the limitation of the short revision period of 10 days, usability evaluation, which requires IRB acquisition, is inevitably going to be conducted through follow-up studies. We hope the reviewers have a broad understanding. Thank you very much for your review efforts and comments.
Reviewer 2 Report
Dear Authors,
Well done for conducting this interesting research and writing this paper.
Kindly find the following comment regarding your paper:
- Re-write Abstract to reflect research importance, describe research methodology and add results and conclusions.
- The opening paragraph in the Introduction Section contained facts that need to be backed up from literature.
- Explain how table 1 would contribute to the research presented in the paper.
- Highlight how your research would improve over other similar research available in literature.
- You mentioned “manufactured according to 292 the goal performances listed in Table 1”. Did you manufacture the product with the new approach presented in this paper installed to it? Is it available in market? Or just installed the approach presented in this paper to powered wheelchair available in the market?
- If Yes to 5: Did you conduct a cost/benefit analysis before manufacturing the product? Any feedback from consumers?
- Please explain the following:
- Why the front or front and middle wheels drove over 40mm bump?
- Did the system consider 51mm an obstacle that’s why it did not drive over it?
- Have you considered near-misses while testing your approach using manual and autonomous driving?
- Have you considered drivers’ ability and performance during testing?
- References have inconsistent format.
- Reference 13 is a website. Please use Journal guidelines to properly refence online sources.
Author Response
Reviewer 2:
Dear Authors,
Well done for conducting this interesting research and writing this paper.
Kindly find the following comment regarding your paper:
1. Re-write Abstract to reflect research importance, describe research methodology and add results and conclusions.
Response) Authors appreciate very much for your nice comments on our paper. We revised the abstract as you assessed it. You can see it in the page 1 abstract.
2. The opening paragraph in the Introduction Section contained facts that need to be backed up from literature.
Response) We have revised the introduction as you judged it. You can find it in the first paragraph of the introduction on page 1.
3. Explain how table 1 would contribute to the research presented in the paper.
Response) An electric wheelchair is an existing medical device, and its regulations are specified by each country's Medical Device Act. We set the development specifications for the self-driving electric wheelchair system by extracting the items related to the driving function by referring to the Korean standard for medical devices. I think this will ensure the safety of the user when driving in a wheelchair. This content is also described under Table 1 on page 5 of the paper.
4. Highlight how your research would improve over other similar research available in literature.
Response) In previous studies, there have been many studies in which autonomous driving of wheelchairs is driven along walls or driving lines. Recently, technologies capable of autonomous driving have been developed using lidar sensors, etc., but they are rarely applied to wheelchairs. In addition, research on self-driving wheelchairs using the system in hospitals has been conducted, but it is difficult for individuals to use them. In order to overcome this limitation, we tried to develop a mobility assistance system that can be operated freely by individuals. This is written at the end of 2.2.1 on page 3 of the paper.
5. You mentioned “manufactured according to 292 the goal performances listed in Table 1”. Did you manufacture the product with the new approach presented in this paper installed to it? Is it available in market? Or just installed the approach presented in this paper to powered wheelchair available in the market?
Response) The product according to Table 1 has been manufactured, but it is still in the prototype stage, and the product will be completed through further research in the future. Even after completion, it seems that it will take a little longer to get medical device certification and sell it. The contents of follow-up studies were added to the last part of 6. Conclusion.
6. If Yes to 5: Did you conduct a cost/benefit analysis before manufacturing the product? Any feedback from consumers?
Response) After the prototype is completed, cost/benefit analysis will be carried out. Usability evaluation will also be performed before obtaining medical device certification. In conclusion, this content will proceed as a follow-up project.
7. Please explain the following:
ï‚· Why the front or front and middle wheels drove over 40mm bump?
ï‚· Did the system consider 51mm an obstacle that’s why it did not drive over it?
Response) The contents of the 40mm and 51mm bumps were written in the second paragraph under Figure 12 on page 12. Both bumps were significantly lower than the sensing height of the 2D lidar sensor, so they were not recognized as obstacles. However, it is judged that the complete driving did not proceed due to reasons such as the size of the front wheel and motor performance. This consideration will be addressed through design changes or additional device attachments in subsequent studies.
8. Have you considered near-misses while testing your approach using manual and autonomous driving?
Response) Near-misses were not considered in the tests in this study. Since the real hazard occurs only when there is contact with an obstacle, only the case of contact with an obstacle is considered. We plan to conduct usability evaluation through follow-up studies, and at that time, I think that it can be considered by checking the degree of psychological threat of users according to near-misses. On page 14 of the content, it is stated that "If there is no direct contact with an obstacle, it is judged that an accident has not occurred."
9. Have you considered drivers’ ability and performance during testing?
Response) The user who participated in the manual driving test is a person who is proficient in research on wheelchairs by conducting several trial runs during the development stage. Therefore, it is considered that it is not unreasonable to compare it with the test results of autonomous driving, which are judged to have high driving skills. The driver's skill level and the number of trips were written in the second paragraph under Figure 14 on page 14.
10. References have inconsistent format.
Response) The format of the bibliography has been changed according to the guide. In addition, several additional references were used. You can find it in the bibliography at the end of the paper.
11. Reference 13 is a website. Please use Journal guidelines to properly reference online sources.
Response) Actually, the existing reference 13 is not a site but a Korean medical device standard. The information on Korean laws and regulations was checked and matched with other reference formats.
Thank you very much for your evaluation. We have tried to reflect your evaluation items as much as possible. Additional studies that are not possible within the 10-day revision period will be conducted again later to be introduced as a new paper. Please rate the revised manuscript as well.
Round 2
Reviewer 1 Report
Dear Authors,
now the title of the paper corresponds to the content of the paper much better. Also the abstract is more clear now.
The paper is more understandable than the previous version. What I still see as a drawback is the experimental part. The 3 runs of the experiments are not statistically significant. It should be at least 10 trials (if the variance is low). Also, there should be more scenarios for the autonomous drive (e.g. narrow passage (like going thru door), sharp turn etc.).
I think, it could be nice article at the end.
Author Response
now the title of the paper corresponds to the content of the paper much better. Also the abstract is more clear now.
The paper is more understandable than the previous version. What I still see as a drawback is the experimental part. The 3 runs of the experiments are not statistically significant. It should be at least 10 trials (if the variance is low).
Response) Thanks for the good evaluation. We performed an additional 10 experiments according to the reviewer's suggestion. As a result, the performance over bumps did not change in each experiment. The experimental results do not seem to be affected by the environment or the number of experiments. The contents including the number of experiments and the results were written on page 12 of the article. I'd like you to check that.
Also, there should be more scenarios for the autonomous drive (e.g. narrow passage (like going thru door), sharp turn etc.).
Response) To verify the indoor use of autonomous wheelchairs, we conducted experiments on additional scenarios as suggested. This is a scenario where you drive around a corner in a narrow corridor with a width of 1.2m. The 1.2m width is the narrowest corridor width that can be made in indoor environments such as homes and hospitals by Korean standards. In this experiment, the wheelchair drove without collision, and additional experiments and results were reflected by adding item 5.4 on page 15 of the article.
I think, it could be nice article at the end.
Response) Thank you for rating our article. We revised the article to reflect the reviewer's suggestions as much as possible, and I think it has made the article even better. In the future, we will continue to research so that we can write good articles. Thank you again.
Reviewer 2 Report
Dear Authors,
Well done. Minor English language improvement is required.
Author Response
Well done. Minor English language improvement is required.
Response) Thanks for the good evaluation. In order to improve the level of English proficiency, we request a paid English proofreading service and receive English proofreading. However, this correction service took more than 7 days, so it was not reflected in this revision. If the article is accepted, it is expected that it will be published as an article with improved English proficiency by reflecting the English correction. We ask for your understanding for the part that did not reflect the correction service in this revised version due to the deviation between the revision service schedule and the revision upload schedule. (Modifications in the uploaded revision are the contents that other reviewers have asked to correct.)
Round 3
Reviewer 1 Report
The authors include all the recommendations and now the paper is ready for publication.
