Predicting the Liquid Steel End-Point Temperature during the Vacuum Tank Degassing Process Using Machine Learning Modeling

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

This article mainly studies the application of machine learning in VTD temperature prediction. Using inputation strategies combined with Grid Search method to supplement missing data, and input parameters are grouped and divided into batches. By evaluating the prediction results of each model, the best prediction model is selected from the aspects of prediction accuracy, stability, and simplification. Finally, the SHAP method was used to explain the influence of each input variable on the output temperature. The article is rich in content. However, there are still some issues that need to be corrected in this article, and specific suggestions are as follows:

 1.The third sentence in Abstract : “The produced ML model aims to achieve a 90% hit ratio for a prediction error within ±5◦C”. We suggest deleting it as the accuracy of this article has not been achieved, which may cause misunderstandings among readers.

2.The fourth sentence in Abstract :“Another objective is to assess the model’s alignment with metallurgical domain expertise”. The description may not be accurate. Although SHAP can explain machine learning from the degree of data dependence, it is not closely related to professional knowledge in the metallurgical field. It can be described as explaining machine learning models by analyzing the nonlinear relationship between input and output variables.

3.There is an issue with the citation format after "DIR" in the second line of the first paragraph of 2.1.

4.In section 3.1, it is recommended to include quantitative data such as input value range and mean for easy reference by readers.

5.When predicting the endpoint, only the input conditions of a single process are usually considered. In this study, the input variables not only contain initial and process information of VTD, but also partial information of the previous LF process (such as BOF to Ladle Addition, Tapping Temperature). What are the advantages of selecting variables in this way?

6.It is recommended to add horizontal and vertical coordinate values in Figure 10 to facilitate readers' clear understanding of the prediction effect.

7.The detailed indicators of Model of Choice may not have been specified in detail in section 4.1.

 

Comments on the Quality of English Language

good

Author Response

Dear Reviewer,

Thank you very much for your valuable insights and feedback. We have carefully considered your suggestions and made several revisions to the paper accordingly. Please find attached our response to your comments.

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

This paper should be published but as a study and use case on the development of ML models. Importantly, it includes a data/domain analysis that begins to address why an ML model does not perform well. The paper is more appropriately titled, something along the lines of, ‘An Extensive Development and Validation Study of an ML model for Predicting the Liquid Steel End-point Temperature.’ When viewed through this lens, the paper would more logically open with an overview of the development and validation process and then go into the steel temperature modeling as a use case. The paper also interspersed sections that are more like tutorials. These might come together in a more concise way as a brief tutorial section explaining the analysis process. 

There were several key decisions about how the model was conceptualized that need to be made clearer and upfront. I had to read through the paper several times to discern how the model was conceptualized (with respect to the secondary metallurgy process). Table 4 was particularly important when combined with Figures 1 and 2 and section 3.3.2 to come to grips with the basic model approach, i.e., the only way I could explain the selection of variables in Table 4 was to view the secondary metallurgy process as repeated “batches” that are affected by multiple internal and some upstream effects that occur in each batch process. These batches are called “heats” in the paper.  It is important this be made clear early in the paper. It is important to the review that I have this correct.

Also critical to the outcomes of the analysis was the decision to form variable groups based on ‘type’, e.g. temperature, mix, gas, etc. as shown in Table 5. This forces the study of the model to be conceptually based on types of effects regardless of when and where they occurred instead of taking them into account as process steps within the overall process. As noted, this throws causation and correlation together to be studied in variable groups. Some are more associated with process steps allowing for the impacts of individual and different combinations of process steps to have been studied. This was an interesting aspect of the study which I believe, with further analysis of the different variable batches in light of particular steps, could have shed greater insight on the modeling outcomes. This was somewhat alluded to on section 3.2.2. Nevertheless, it is interesting that the study analyzed 16 different variable group models.

Another foundational aspect to the analysis that was left somewhat unanswered was section 3.3.2 which treated the splitting of training and test data. There is discussion about the importance of a temporal aspect, but that aspect is not made clear. Presumably, that aspect has to do with changes in the secondary metallurgical process over long periods of time. By doing the cross validation as shown in Figure 7, the effects of long-haul changes can be observed. We too always train with older data, but test and validate with more recent data. This study took it step further by cross validating over time.

The above points ultimately set up for a study on data imputation, model performance and model selection (where model selection has to do with data models). This study ranks among the most extensive studies on the treatment of data models that I have seen. I think this analysis and the results discussion (how the analyses are done and interpreted) are exceptionally valuable.

This brings us an analysis of the outcomes which do not meet specifications.  Given the foundational decisions above, the analysis brings out the point that this is a process involving steps that when together lead to considerable complexity from many energy effects. The domain analysis starting on line 864 is quite informative. In general the analysis was a good example of thinking through domain explanations of observed data effects.However, the conclusions left the analysis somewhat unresolved by saying the results do not meet process engineer’s performance requirements because of the data quality.  For example, the results could point instead to the decision to model with the variable groups or missing key needed capture certain phenomena.

 

Author Response

Dear Reviewer,

Thank you very much for your valuable insights and feedback. We have carefully considered your suggestions and made several revisions to the paper accordingly. Please find attached our response to your comments.

Author Response File: Author Response.pdf