Kinetic Parameter Determination for Depolymerization of Biomass by Inverse Modeling and Metaheuristics

Round 1

Reviewer 1 Report

the R square or MSE is just in-sample stuff? 

How about out-of-sample power for the algorithm?

 

Author Response

We thank you for your valuable comments.

 

1. Comments and Suggestions for Authors the R square or MSE is just in-sample stuff? 
2. How about out-of-sample power for the algorithm?

 

Answer to both questions: Thank you for the observations. Originally both indexes were used just for in-sample data. We have now conducted a 4-fold cross validation scheme in order to measure the out-of-sample prediction power of the considered kinetic models. Please see the new Table 3, where you can also find that the AIC criterion was incorporated to consider the complexity of the models.

 

Reviewer 2 Report

This paper presents some analyses of drying regular of extruded food and the effects of interaction among operation parameters on target parameters. The results are new as far as I see. The topic is worth investigation. However, the presentation should be improved. There are some technical issues that need to be clarified. I will give my recommendation based on the quality of the revision. The detailed comments are as follows.

(1) Line 32, although RA has been mentioned in the abstract, the full name should be given when it first appears in the body of the article.
(2) The contribution and difficulty should be further highlighted in the introduction. It is not sufficient to just say there are few people have done this. Why is that? What is the novelty of the approach you adopted?
(3) What is the reason for putting potato flour in a constant temperature and humidity sealed container to regulate humidity and make the final moisture content 14%? Please shed some light on that.
(4) Line 89, I doubt the multiplication of 100 should be in the numerator. Please double check.
(5) Line 96, "t min/ min" seems not right.
(6) What is the delta v in line 160?
(7) How the PCA done for Figure 7 should be explained. It is not very clear to me.
(8) Figure 7 is too small to be seen. I at first printed the paper on A4 papers but have to go back to the pdf file to see the labels and text.
(9) The conclusion is weak. Ideally, some future directions and open problems should be discussed. This would be very helpful for interested readers.
(10) Finally I have a general comment about data collection of RA samples. It is known that some seemingly random sampling methods (e.g. Subgraph robustness of complex networks under attacks) may inadvertently yield prominent bias. It would be beneficial to discuss and ensure the readers that the data sample methodology in this work is acceptable.

 

 

Author Response

We thank you for your valuable comments.

Reviewer 3 Report

The manuscript by Aztatzi-Pluma et al. focuses on the inference of four Michaelis-Menten based mathematical models to formalise the depolymerisation reaction required to produce biofuels from vegetal biomass. The ability of the models to recapitulate time-series data both at the micro-scale and bench-scale is considered. As it stands, the study requires non-negligible clarifications, detailed in the following.

 

%% Major comments %%

1. Comparison between competing models for a biological system cannot rely simply on their ability to recapitulate experimental data in the training set. As a minimum, the performance of the models at predicting alternative behaviours should have been included (e.g. the experimental dataset should have been splitted into 2/3 training and 1/3 test set). In addition, a metric adjusting for the number of parameters in the models should be included (e.g. AIC) to prevent model complexity from determining the outcome of the comparison;
2. How is the uncertainty in experimental data accounted for in the optimisation? Why do not select a weighted least square cost function for the inference?
3. The authors refer to convergence of some optimisers to solutions that are not compatible with prior knowledge of the system at hand. However, by definition, those are not solutions for the considered problem and should not be presented.
4. Parameter inference requires an accuracy assessment: extremely large confidence intervals would indicate, for example, that the data are not informative enough to support model calibration. Please, include an estimate of parameters accuracy in Table 3;
5. Parameter estimates should be robust to the initialisation of the optimisers; if these are known to be prone to convergence to local minima, a multi-start optimisation should be performed;
6. As stated the inhibitor concentration in the ‘competitive inhibition’ model should be time varying. Why was it considered a constant parameter? This requires clarification.
7. The presented results suggest that IPA is the best optimiser to solve the inference problem. Why was it not selected in the comparison between models at different scales?
8. Am I correct in concluding that competitive inhibition best recapitulates micro-scale experiments, but the non-competitive inhibition model should be preferred when micro- and bench- scales are considered? As this is justified by the properties of the experiment selected for the comparison, the same analysis should have been extended to more than 1 experiment.

 

%% Minor %%

1. New results should not be presented in the discussions, which at the moment looks like a mere extension of the previous section. Rather, discussions should focus on the authors’ interpretation of the obtained results. As a minimum, the comparison between scales should be moved to the results section.
2. A quantitative comparison of the computational time required by the optimisers requires to specify the properties of the computer/cloud on which the optimisations were run;
3. Models do not ‘adjust’ to the experimental data, they are calibrated/fitted on data and eventually recapitulate it;
4. Note that according to conventional notation ‘x’ is used to identify state variables, while \theta is used for parameters;
5. In Equation (10) the index ‘i’ is not specified (I suppose it identifies the time-points);
6. Lines 291-292 require clarification: what do we mean by ‘describes the trajectory of the data’?
7. Please carefully proofread the manuscript, as it contains multiple errors and sentences that are long and convoluted. Please try and streamline them. A few examples are reported below:

Line 45: ‘analyzes’ -> ‘analysis’

Line 45: ‘The IM’ -> ‘IM’

Line 66: ‘based on’ -> ‘based on the’

Line 85: ‘shown’ -> ‘showed’

Line 99: ‘inhibition concentration’ -> ‘inhibitor concentration’

Line 103: ‘according the’ -> ‘according to the’

Line 128: ‘is as high’ -> ‘is so high’

Line 130: ‘to minimize’ -> ‘at minimizing’

Line 138: ‘differ on’ -> ‘differ in’

Line 172: ‘initialize’ -> ‘initialized’

Line 195: ‘estimate to’ -> ‘estimate of’

Line 208: ‘calculating determination’-> ‘calculating the determination’

Line 282: ‘bet-fit’ -> ‘best-fit’

Line 299: ‘affinity of the enzyme with the substrate’ ->‘affinity of the enzyme for the substrate’

Line 311: ’11 and 12 has’ -> ’11 and 12 have’

Author Response

We thank you for your valuable comments. We make an effort to answer most of the concerns; however, the time assigned by the editorial team was not enough to finish the experiments at different scales.

 

Major comments

 

1. Comparison between competing models for a biological system cannot rely simply on their ability to recapitulate experimental data in the training set. As a minimum, the performance of the models at predicting alternative behaviors should have been included (e.g. the experimental dataset should have been splitted into 2/3 training and 1/3 test set).  

 

Answer: Thank you for the comment. We conducted a 4-fold cross-validation scheme to measure models’ performance on prediction. Table #3 now reports the mean and standard deviation of the MSE on training set and MSE on test set.

 

1. In addition, a metric adjusting for the number of parameters in the models should be included (e.g. AIC) to prevent model complexity from determining the outcome of the comparison.

 

Answer: Thank you for the comment, we agree with the relevance to measure the complexity of the model. The Akaike Information Criterion (AIC) was included to prevent model complexity as suggested. Please see the new Table 3, where the best model found by each optimizer was selected according to the AIC. We found that models with less parameters were favored, but the R2 index remains similar.

 

1. How is the uncertainty in experimental data accounted for in the optimization? Why do not select a weighted least square cost function for the inference?

 

Answer: We implemented a weighted least square (WLS) function to lead optimizers’ search processes as suggested.  Each weight was computed as 1/variance of the corresponding experimental data point, except for those points where variance was 0. In those cases, weights were assigned to a value of 1. We found a similar behavior with both metrics (Euclidian norm and WLS), thus we reported the new results based on WLS.

 

1. The authors refer to convergence of some optimizers to solutions that are not compatible with prior knowledge of the system at hand. However, by definition, those are not solutions for the considered problem and should not be presented.

 

Answer: The reviewer is right. The results that do not provide a solution to the problem were removed from the manuscript.

 

1. Parameter inference requires an accuracy assessment: extremely large confidence intervals would indicate, for example, that the data are not informative enough to support model calibration. Please, include an estimate of parameters accuracy in Table 3;

 

Answer: For the sake of reproducibility, we think it is more convenient to report the optimal parameters in Table 3 instead of its confidence intervals, since this table presents just the best-fitted models. However, these intervals are provided as supplementary material in the Excel file entitled Table 3.xlsx and the optimizers.xlsx file provide the full set of results for further analysis.

 

1. Parameter estimates should be robust to the initialization of the optimizers; if these are known to be prone to convergence to local minima, a multi-start optimisation should be performed;

 

Answer: UMDA is the only method that we know can get easily trapped into local minima. Through the cross-validation scheme, we took care that multi-start points were assigned to the method to increase the reliability of its results.

 

1. As stated the inhibitor concentration in the ‘competitive inhibition’ model should be time varying. Why was it considered a constant parameter? This requires clarification.

 

Answer:. The value of inhibitor concentration was not considered constant. The value was calculated between 0 to 8.5 mg/mL. This is now clarified in the text. Please see page 12 line 331-332.:

 

 “The parameter I (inhibitor) was calculated, placing a range between 0 and 8.5. This range was used because it is well known that there is an intermediate product called cellobiose during the production of glucose which presents an inhibitory effect over biomass depolymerization reaction [2]. The maximum amount of glucose-cellobiose that can be ideally produced is 8.5 mg.

 

1. The presented results suggest that IPA is the best optimizer to solve the inference problem. Why was it not selected in the comparison between models at different scales?

 

Thank for the comment: The results shown in the original manuscript were for PSO optimizer. This was an arbitrarily decision since both PSO and IPA had similar performance. Nevertheless, in the actual manuscript the IPA optimizer was implemented.

 

1. Am I correct in concluding that competitive inhibition best recapitulates micro-scale experiments, but the non-competitive inhibition model should be preferred when micro- and bench- scales are considered? As this is justified by the properties of the experiment selected for the comparison, the same analysis should have been extended to more than 1 experiment.

 

Thank for the comment: The experiments tested in micro-reaction were carried out under several conditions. The conditions tested do not maximize the depolymerization process, but they did allow obtaining the conditions that maximize the depolymerization. For this reason, the competitive inhibition model recapitulates all these experiments better because exist inhibition associated with the operating parameters. Once the parameters that maximize the depolymerization were obtained, an experiment was performed in micro-reaction. The use of these parameters allowed obtaining a maximum depolymerization, in which the inhibition associated with the operating parameters is practically null. This is the reason why the traditional Michaelis-Menten model better recapitulates the last micro-reaction experiment. On the other hand, the bench-scale experiment reproduced the operating conditions of the last micro-reaction experiment, so the traditional Michaelis-Menten model is also the one that most closely approximates the trajectory of the bench scale experiment. For these final experiments, the IPA method was used, and more experiments were included.

 Minor corrections

1. New results should not be presented in the discussions, which at the moment looks like a mere extension of the previous section. Rather, discussions should focus on the authors’ interpretation of the obtained results. As a minimum, the comparison between scales should be moved to the results section.

 

Thank for the comment: The section was corrected. The results presented in the discussion section were moved to the results section. The changes can be seen over the document and are highlighted using the edition tool of Microsoft word.

 

1. A quantitative comparison of the computational time required by the optimizers requires to specify the properties of the computer/cloud on which the optimizations were run;

 

Thank for the comment. The characteristics of the computer are: Intel ® Core™ i7 CPU @ 2.60 GHz processor, 16GB RAM and 256 GB hard disk. The characteristics can be read in page 10, line 277-278.

 

1. Models do not ‘adjust’ to the experimental data, they are calibrated/fitted on data and eventually recapitulate it;

 

Thank for the comment: The redaction was changed. The word “adjust” was replaced. The document now includes the word calibrated/fitted based on the context.

 

1. Note that according to conventional notation ‘x’ is used to identify state variables, while \theta is used for parameters.

 

Thank for the comment. The reviewer is right. The notation was changed.

 

1. In Equation (10) the index ‘i’ is not specified (I suppose it identifies the time-points); The index ‘i’ was specified as suggested.

 

1. Lines 291-292 require clarification: what do we mean by ‘describes the trajectory of the data’?

 

Thank for the comment. The phrase “describes the trajectory of the data” was replaced by "recapitulates the trajectory shown by the experimental data.” Please see page 11, line 319.

 

1. Please carefully proofread the manuscript, as it contains multiple errors and sentences that are long and convoluted. Please try and streamline them. A few examples are reported below:

 

Answer: The authors reviewed and corrected all the misspellings.

 

Line 45: ‘analyzes’ -> ‘analysis’

Line 45: ‘The IM’ -> ‘IM’

Line 66: ‘based on’ -> ‘based on the’

Line 85: ‘shown’ -> ‘showed’

Line 99: ‘inhibition concentration’ -> ‘inhibitor concentration’

Line 103: ‘according the’ -> ‘according to the’

Line 128: ‘is as high’ -> ‘is so high’

Line 130: ‘to minimize’ -> ‘at minimizing’

Line 138: ‘differ on’ -> ‘differ in’

Line 172: ‘initialize’ -> ‘initialized’

Line 195: ‘estimate to’ -> ‘estimate of’

Line 208: ‘calculating determination’-> ‘calculating the determination’

Line 282: ‘bet-fit’ -> ‘best-fit’

Line 299: ‘affinity of the enzyme with the substrate’ ->‘affinity of the enzyme for the substrate’

Line 311: ’11 and 12 has’ -> ’11 and 12 have’

Answer: The authors reviewed and corrected all the misspellings.

Round 2

Reviewer 2 Report

The revised paper has been improved. However, there are still much work to do, in particular, the presentation can be better organized and the figures and charts need some further work. My comments for this version are as follows.

(1) Line 41, 'S' should be in italic font. It is a variable in Equation (1).
(2) Line 91, what are the considerations when an attempt on spatial homogeneity is made? More details are needed.
(3) Line 146, a fraction of the best solutions are selected to construct an estimate of their distribution. Is this sample a random sample or could it be something similar to the localised sample mentioned in 'Subgraph robustness of complex networks under attacks'? The potential bias of this sampling methodology should be made clear. A discussion is preferred.
(4) There are still many grammatical errors and typographical errors. For example, line 298, 'Figure 4 depict the...' should be 'Figure 4 depicted...'
(5) In the flow chart of Algorithm 1, some key words like 'while' and 'end' should be in bold. The same comment applies to other similar places.
(6) In equation (10), the meaning and rationale of R^2 should be remarked.
(7) In Figures 1,2, etc., what are the error bars?
(8) In Figure 5, the upper and lower bounds are displayed with the same type of lines. It might be acceptable as you can see which one is higher, and it must be the upper bound. However, rigorously speaking the methodology here is confusing. It would be better to use different line types or color schemes.

Author Response

Thank you for all your valuable comments. They have helped us to improve our work.

Author Response File: Author Response.docx

Reviewer 3 Report

The revision seems rushed in some points. I would encourage the authors to add to the introduction a clear statement on how this work fills a gap in scientific literature and what the reader should infer from it. This would allow to remark novelty and relevance of the results (currently unclear). 

%% Minor comments %%

1. [Line 84, page 2]: 'as are showed' --> 'as shown'
2. [Line 85, page 2]: 'model that shown' --> 'model that showed'
3. [Lines 96-97, page 3]: Rephrase. Example: 'the ability of the models [...] to fit experimental data was evaluated'
4. [Line 110, page 3]: 'that best fitted'
5. [Line 111, page 4]: 'the quality of fit achieved by each model'
6. [Line 113, page 4 and equation 6]: S should be expressed as a function of theta.
7. [Line 114, page 4]: 'standard deviation'?
8. [Line 207]: 'measure the accuracy (or quality) of fit'
9. [Line 215]: '(prm) of models, allowing to compare models of varying complexity'
10. [Line 243]: A model is calibrated. So here 'unsatisfactory fit' is better.
11. [Line 261] 'best fit to experimental data'
12. [Line 281]: 'computation time to calibrate the first three models to the available experimental data..'
13. [Line 292]: 'results presented in []'
14. [Line 295]: 'the two reported experiments is'
15. [Lines 328-330]: Saying that the value of a parameter was calculated in a range does not mean that it is time varying. It only implies that there is a range of admissible values. This variation does not answer the original comment on I. 
16. [Lines 361-362]: This sentence is out of context. 
17. [Line 364]: 'worse fit'
18. [Lines 388-391]: This sentence needs rephrasing. Example: 'a better calibration for mathematical models of kinetic polymerisation process [..] to inform the best decision for the process it is necessary ... 

Author Response

Thank you for all your valuable comments. They have helped us to improve our work.

Author Response File: Author Response.docx