Driving Torque Distribution Strategy of Skid-Steering Vehicles with Knowledge-Assisted Reinforcement Learning
Round 1
Reviewer 1 Report
(i) The signals v_delta and w_delta may be contaminated with noise in a real environment. Similar noise can appear in acceleration signals v’ and w’. Such noisy signals usually present a challenge in real systems. How do you deal with this?
(ii) The proposed algorithm is quite complex. Please provide more comment on possible implementation in a practical system, especially regarding controller and real-time execution etc.
(iii) I advise to continue your research towards a real experiment, at least in a lab. environment.
Author Response
Please see the attachment.
Author Response File: 
Author Response.pdf
Reviewer 2 Report
This contribution presents original ideas in the study and advances the previous research in this area. The paper deserves publication. The level of the originality of contribution to the existing knowledge with an emphasis on the paper’s innovativeness in both theory development and methodology used in the study is very high.
This work makes a significant practical contribution and it makes impact on the research work on the research community.
The quality of arguments, the critical analysis of concepts, theories and findings, and consistency and coherency of debate are well addressed in this paper.
The paper has a good writing style in term of accuracy, clarity, readability, organization, and formatting.
Nevertheless, the following points should be clarified to imporve the Quality of the paper.
The cost function defined in (6) states an optimizion problem, please discuss more in depth the non convexity of this Problem and some issues in this context.
The scheme of Figure 3 states the framework of the proposed DDPG algorithm, please address the problem of the stability of this control scheme.
Similar considerations should be done for the scheme of Figure 6 in which the Knowledge-assisted learning method—guiding reward method is described.
Concerning the cited literature you can consider the following papers to improve the tutorial aspects of the paper. Some of These papers are alreadypublished in MDPI Journals.
Kruse, O.; Mukhamejanova, A.; Mercorelli, P. Super-Twisting Sliding Mode Control for Differential Steering Systems in Vehicular Yaw Tracking Motion. Electronics 2022, 11, 1330. https://doi.org/10.3390/electronics11091330
P. Mercorelli "Using Fuzzy PD Controllers for Soft Motions in a Car-like Robot", Advances in Science, Technology and Engineering Systems Journal, vol. 3, no. 6, pp. 380-390 (2018).
P. Mercorelli, "Fuzzy based control of a nonholonomic car-like robot for drive assistant systems," 2018 19th International Carpathian Control Conference (ICCC), 2018, pp. 434-439, doi: 10.1109/CarpathianCC.2018.8399669.
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Reviewer 3 Report
1- The research contributions and the significant features of the proposed algorithm compared to the previous ones have not been highlighted in the Abstract section.
2- What are the advantages of RL to address the studied problem? I mean that the authors should discuss the reasons behind the selection of RL for proposing their approach.
3- To clarify the research gap in the previous studies, I recommend that the authors separate the Literature review section from the Introduction one.
4- The further explanations regarding Figures 3 and 4 should be added to this manuscript.
5- The authors should discuss the practical implications of this study.
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Reviewer 4 Report
REVIEW
on article
Driving Torque Distribution Strategy of Skid Steering Vehicles with Knowledge-Assisted Reinforcement Learning
Huatong Dai, Pengzhan Chen, and Hui Yang
SUMMARY
The article is devoted to the topical issue of reasonable torque distribution for coordinating the control of independent driving wheels of cars.
The authors conducted a detailed study, both from the point of view of theory and from the point of view of modeling. The paper presents two parts: a model of the dynamics of vehicles controlled by skidding and the DDPG algorithm, a method of torque distribution based on KA-DDPG for cars controlled by skidding is developed. The settings of the simulation environment are presented. In addition, the effectiveness of the torque distribution strategy based on KA-DDPG and the contribution of auxiliary training methods to KA-DDPG are shown.
The results of modeling the tracking efficiency showed that the proposed torque distribution strategy has excellent control characteristics in various scenarios. In addition, the results of modeling the contribution of the auxiliary training method showed that both proposed methods of auxiliary training can speed up the agent's learning process. The criterion action method sets the direction of the agent's training and speeds up the learning rate by reducing random search actions, and the guiding reward method achieves the same by strengthening the reward function.
Thus, the study is original, has novelty and practical significance. However, for the article to be published, it is necessary to correct some of the comments that are presented below:
COMMENTS
- In the Abstract in line 13, the word "learning" is repeated three times and one more time in the next line. It is necessary to rewrite the last sentence of the Abstract since the essence of this proposal is not entirely clear.
- At the end of the "Introduction" section, the contribution of this study and the structure of the manuscript are listed. To complete this section, there are not enough formulations of the purpose and objectives of the study. It is necessary to add.
- It would also be more visual to present the research plan in the form of a flowchart. I recommend presenting a step-by-step research plan in graphical form. This would make it possible to structure the research plan more clearly and strengthen the manuscript.
- At the end of Section 4, after Figure 7 and Table 2, it would be methodologically correct to add a brief analysis and (or) an intermediate conclusion. It would be necessary to make the transition from methods to results smoother, without ending the section with a drawing and a table.
- Figure 13 should be placed immediately after the first mention of it. You need to move it.
- In the "References" section, for the convenience of searching for literature, it would be necessary to add links to it by type https://doi.org /or doi number.
- In line 252, the sentence begins with a lowercase letter "we introduced". Needs to be fixed.
- There is no section or subsection "Discussion" in the article. It is necessary to add at least a few paragraphs in which to discuss the results obtained with the results of other authors whose references are given in the "References" section.
- The section "Conclusions" in lines 394-407 should be divided into smaller paragraphs or presented point by point. In its current form, this section is difficult to understand and summarize the work done by the authors.
- In general, there are some stylistic inaccuracies and errors in the article. You should check the text of the article again.
- The English language of the manuscript requires some edits.
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Round 2
Reviewer 3 Report
I appreciate the authors' efforts to consider the reviewers' comments to improve the quality of this manuscript. I think the authors should add further explanations about the theoretical and practical implications of this research.
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
Reviewer 4 Report
The article is written on the topical task of reasonable torque distribution for coordinating the control of independent driving wheels of cars. This theme is relevant and will attract the attention of readers.
All my comments were considered and corrections were made in the text.
I recommend the article for publishing.
Author Response
Please see the attachment.
Author Response File: Author Response.pdf
