The Real-Time Optimal Attitude Control of Tunnel Boring Machine Based on Reinforcement Learning

Round 1

Reviewer 1 Report

In the paper, a real-time optimal control framework of TBM attitude based on reinforcement learning is proposed, which may be used to solve the problems of the snakelike motion around the designed tunnel axis and exceeding the deviation limit. The proposed control framework may predict the current geological information, provide the real-time optimal attitude control, and can be directly deployed to TBM without increasing costs. To verify the effectiveness of this control framework, the Xinjiang Yiner Water Supply Phase II Project is adopted as a case. The issue studied in the paper is meaningful in practical tunnelling engineering. The content is clear and has a certain degree of innovation. It is recommended to be accepted in this journal with some problems that need to be modified.

1. In the paper the GEC prediction model based on current excavation parameters is established in Section 3.3. So, the input excavation parameters are important determinants of prediction accuracy and generalization of the model. 20 excavation parameters were selected as inputs from 228 excavation parameters. How were they selected? What methods were used in the selection process?

2. In Section 4.3, OACP models for 4 GECs using the reinforcement learning is respectively established. We know that hyperparameters have a significant impact on the performance of machine learning. How to set the hyperparameters of reinforcement learning used in Section 4.3? More detailed analysis should be provided.

3. After training, four attitude optimal control policies are obtained in Section 4.3. By comparing these policies with manual control, the results indicate these policies can achieve better attitude control than manual control. The author should provide a more detailed introduction to the comparison index.

English writing is good and it may be improved in the revised version. In the context, some clauses and adverbials may be put after the main clause to emphasize the important content. 

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Reviewer 2 Report

Specific review comments are shown in the pdf document below.

Comments for author File: Comments.pdf

 The writing of this manuscript needs to be improved. Some grammar errors should be checked and modified in this manuscript.

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Reviewer 3 Report

First of all, I appreciate the invitation to review this interesting work. The work is a technical breakthrough in tunnel construction. However, some aspects deserve to be commented on. For instance:

1. The abstract surgically presents all the research work performed. Also including the results. It would also be interesting to mention the constructive method of the work used in the validation of the method.

2. The introduction is very well founded but could also present the construction methods of tunnels such as NATM and others. This would bring a didactic effect to the work.

3. Does the real-time optimal attitude control of tunnel boring machine based on reinforcement learning apply to any tunnel construction method? Could the authors comment on this?

4. A section 4, that applies the proposed framework to the Xinjiang Yiner Water 145 Supply Phase II Project to verify its effectiveness , could be called validation of the method developed in this work?

5. It would be interesting to include a section on tunnel construction methods.

6. In line 181 it is quoted that: The preprocessing of excavation parameters is as follows. Abnormal sampling of sensor will lead to the missing value data, which will be deleted directly; Routine maintenance and cutter change during excavation will result in non-working state data, which can be judged by zero value of the products of thrust, penetration, torque and rotational speed, and deleted directly. Question: what is the criterion for establishing that a parameter is abnormal?

7. In line 191 the authors cite that: The paper only considers the attitude control under the situation of the designed tunnel axis of straight line, the xcavation data under this situation are selected as the training data. In this paper, the origin excavation data from Xinjiang Yiner Water Supply Phase II Project are used. Question: why did you make this consideration?

8. As a suggestion, Table 1 can be reformatted to improve the presentation of the parameters it contains.

9. Figure 3 (Statistical distributions of the excavation parameters) shows normal distribution curves of specific excavation parameters. Question: How does the variation in the number of data affect these curves? Is there a criterion for this consideration?

10. It is suggested that Figure 4 be reformatted to improve the understanding of its contents.

11. Equations 2, 3, 4, 5 and 6 are presented inadequately. It is suggested that they be repositioned in the text of the paper.

12. The same comment from the previous recommendation applies to equation 7.

13. Do all the considerations presented in the methodology of the work apply to any size and type of tunnels? Could the authors comment on this dimensional aspect of the work?

14. Does the framework developed in this work apply to any type of soil or are there restrictions such as rock excavations?

 

The quality of the English language of the work can be considered good due to the huge amount of technical terms presented in the text.

Author Response

Please see the attachment

Author Response File: Author Response.pdf