Collaborative Decision-Making Method of Emergency Response for Highway Incidents
Round 1
Reviewer 1 Report
The paper is well organized, and methodologies and data are well explained. I only have some minor comments for consideration:
Abstract, Line 15: in this statement, please be very specific on “other decision-making algorithms”.
For third component name-“Road” in the Table 1, it would be more appropriate to use “Traffic”. Roads should be already included in “Road Facilities”.
Figure 4, please try to use larger fonts if possible. There are two Chinese characters in the figure.
Line 380, Table number is missing.
Thorough proofread is needed. Please double check the space between words/sentences. Also please consider rephrasing some long and convoluted sentences to avoid low readability.
Author Response
The paper is well organized, and methodologies and data are well explained. I only have some minor comments for consideration:
Response: We would like to thank you for your valuable suggestions and comments in the revisions, it is grateful to see a much better manuscript had been revised according to your suggestions.
- Abstract, Line 15: in this statement, please be very specific on “other decision-making algorithms”.
Response: Thank you for your suggestion.According to your suggestions, We have added descriptions of other decision-making algorithms, including genetic algorithm(EA),evolutionary strategy(ES) and deep Q network(DQN) , and rewritten the abstract, which can be found in the revised version.
- For third component name-“Road” in the Table 1, it would be more appropriate to use “Traffic”. Roads should be already included in “Road Facilities”.
Response: Thank you for your advice, we have changed third component name from “Road” to “Traffic”, and other relevant parts of the text have also been modified accordingly, all the rectifies have been labelled in red, and you may find them in the revised manuscript.
- Figure 4, please try to use larger fonts if possible. There are two Chinese characters in the figure.
Response: Thank you for your suggestion. We have reorganized Figure 4 is not clear enough and the font is not big enough, so we have revised the font of Figure 4, and also revised the two Chinese characters in Figure 4. The description of Fig. 4 (from line 244 to line 270) was improved for a better understanding of the principle of MADDPG. We hope that Figure 4 is easier to read after the revision, all the revisions were marked in red and you may find them in section 3.1 of the revised manuscript.
- Line 380, Table number is missing.
Response: Thank you for your advice, we are sorry for the table serial number missing and table missing problem in the manuscript, we have carefully checked the manuscript and corrected these problem in the revised one. We have filled the missing table number.
- Thorough proofread is needed. Please double check the space between words/sentences. Also please consider rephrasing some long and convoluted sentences to avoid low readability.
Response: We would like to thank you for your valuable suggestions and comments in the revisions, it is grateful to see a much better manuscript had been revised according to your suggestions.All the revisions have been labelled in red and we hope the quality of the manuscript could be improved.
Reviewer 2 Report
After carefully reading the proposed paper, this paper contains an interesting proposal; my overall impression is that the manuscript presents some results that could be useful in practice. I have a good opinion about this work and recommend its acceptance after addressing the following aspects:
1. The abstract need to rewritten to show the motivation of the work
2. The authors need to show the usefulness of their results.
3. The authors must add limitations to the study and more ideas for further research.
4. the authors should to update the reference list with resent articles.
5. In this work, why you used Petri net to simulate the emergency on-site disposal procedures? Is there other methods?
6. Table 1 shows the Mapping between Places and Emergency Tasks. Is it your work ? please cite it if it is from the previous literatures
7. You can add more information about figures a and 4
8. Please, cite all the equations (2-8) which need to be cited.
9. How you validate your results? Explain in more details the comparison which you did.
Author Response
After carefully reading the proposed paper, this paper contains an interesting proposal; my overall impression is that the manuscript presents some results that could be useful in practice. I have a good opinion about this work and recommend its acceptance after addressing the following aspects:
Response: We would like to thank you for your valuable suggestions and comments in the revisions, it is grateful to see a much better manuscript had been revised according to your suggestions.
- The abstract need to rewritten to show the motivation of the work
Response: Thank you for your suggestion. We have rewritten the abstract and highlighted the importance of emergency disposal, revealing that efficient emergency response can protect the life safety of person, and you may find them in the revised manuscript.
- The authors need to show the usefulness of their results.
Response: Thank you for your advice, we have rewritten the results and conclusions parts .By comparing the disposal time of MADDPG emergency disposal algorithm and actual emergency disposal, the effectiveness of the MADDPG emergency disposal algorithm proposed in this paper can be better highlighted.
- The authors must add limitations to the study and more ideas for further research.
Response: Thank you for your suggestion. According to your suggestions, We have illustrated the limitations of this research from both models and algorithms, and propose corresponding improvement methods and research directions based on these two aspects. We have revised the conclusion part from line 659 to line 688, and you can find them in the revised manuscript.
- the authors should to update the reference list with resent articles.
Response: Thank you for your suggestion. We are sorry that the references were old and have not show the latest research in emergency disposal fields, so we have updated our references to better illustrate the innovation of the article. We have change the ref2,ref4,ref6 and have added some new reference such as ref30, and you can find them in the revised manuscript.
- In this work, why you used Petri net to simulate the emergency on-site disposal procedures? Is there other methods?
Response: Thank you for your suggestion. The place, transition and flow in petri net can well describe the discrete emergency disposal process, and the token in petri net can well describe the relationship between the emergency disposal task and the emergency state in the emergency disposal process. So we have choosde petri net after comparing other methods.
- Table 1 shows the Mapping between Places and Emergency Tasks. Is it your work ? please cite it if it is from the previous literatures
Response: Thank you for your suggestion. We have abstracted the basic emergency dispoasl situation according to three cases, and established the Mapping releationship between Places and Emergency Tasks.
- You can add more information about figures a and 4
Response: Thank you for your suggestion. We have reorganized Figure a and 4 is not clear enough , so we have revised the size of Figure a the font of Figure 4. The description of Fig. a (from line 222 to line 225) have been added for a better description of the traffic emergency fire drill in Qin-ling No.1 Tunnel in order to understand emergency disposal site better, and the description of Fig. 4 (from line 244 to line 270) was improved for a better understanding of the principle of MADDPG. We hope that Figure 4 is easier to read after the revision, all the revisions were marked in red and you may find them in the revised manuscript.
- Please, cite all the equations (2-8) which need to be cited.
Response: Thank you for your advice. We have reorganized that formulas are not cited in the article, and we have also carefully checked the manuscript and corrected the citation of pictures and tables problem in the revised manuscript.
- How you validate your results? Explain in more details the comparison which you did.
Response: Thank you for your advice, we have rewritten the results parts .We have added MADDPG to compare with other algorithms and actual disposal, to better illustrate its excellent emergency disposal performance, and show that the algorithm can emergency response departments trained by MADDPG can effectively cooperate with each other to improve the efficiency of emergency disposal. You may find them from line 542 to line 563 in the revised manuscript.
Reviewer 3 Report
How does data collection from on-site happen no information or any points sensors utilized why? - no clarity data generation from on-site case studies -(represented by cases 1-3 in this paper) line 342 - must work on this to give proper information on how its works.
Figure 2. Road surveillance video of traffic emergency fire drill in Qin-ling No.1 Tunnel - not clarity high pixel with larger size picture must.
Figure 6. State in Markov game generated from road incidents onsite scenarios - Accident site marked in the red colour box - explain how data collected from this on-site with these particular location related sensors ..etc. any specific distance calculation for accident site?
line 292 Specifically,Optimizing the emergency - no character space between specifically and optimizing
Figure 9. Diagram of Cooperative Emergency Decision-Making Method - need more clarity and precise figure with good text colour pattern to highlight all marking of object identification.
line 326 Algorithm 1 gives the pseudo-code of the emergency disposal algorithm for highway accidents - Algorithm 1 where its algorithm one not understandable?
line 351 explains what you mean by disposal time in your research work and why it's important in MADDPG to optimize it.
Table 4. Hyperparameters of Each Emergency Decision-making Algorithm - need more clarity and explanation for listed algorithms and Meaning in the given table by describing why MADDPG is related to noise intensity which showcases the difference with others algorithms meaning value why Hyperparameters are different for others algorithms. what is the benefit and advantages of MADDPG with noise intensity?
line 362, and 363 which are not the understandable way, After table 4 mentions the initial emergency disposal tasks involved in the three cases, and their initial statuses- making it a proper understanding for readers' perspective in terms of tabulation and explaining it more elaborate.
line 380 The training 379 super parameters of the above algorithms are shown in Table ??. - table number?
line 385 - The number of neurons in the three 385 hidden layers is 96128 and 64, respectively - why can't authors give a neural network diagram for good understanding? if it added well and will be good for new learners' and readers' perspectives.
line 400 -A variety of emergency decision-making algorithms, including GA, 400 ES, DQN and MADDPG, have been trained by 1000 epides respectively - same as line 385 why can't authors give a neural network diagram for good understanding?
line 466 - As shown in Table 7, the performance of each algorithm in different cases is consistent - where is table 7?
Author Response
- How does data collection from on-site happen no information or any points sensors utilized why? - no clarity data generation from on-site case studies -(represented by cases 1-3 in this paper) line 342 - must work on this to give proper information on how its works.
Response: Thank you for your suggestion.We were aware of the problem of unclear interpretation in the process of introducing the cases, and we have added descriptions in line 413-419 to describe how to transmit emergency site information to the emergency department through cameras and communication equipment, and you may find them in the revised manuscript.
- Figure 2. Road surveillance video of traffic emergency fire drill in Qin-ling No.1 Tunnel - not clarity high pixel with larger size picture must.
Response: Thank you for your suggestion. We have reorganized Figure 2 is not clear enough, so we have adjusted the clarity and size of the picture. We hope that Figure 2 is easier to read after the revision.
- Figure 6. State in Markov game generated from road incidents onsite scenarios - Accident site marked in the red colour box - explain how data collected from this on-site with these particular location related sensors ..etc. any specific distance calculation for accident site?
Response: Thank you for your suggestion. We were aware of the problem of unclear expression of Figure 6, so we have added the description of the picture 6 from line 336 to line 342, which have introduced emergency department how to obtain information from the emergency scene and set the corresponding state information in the model.
- line 292 Specifically,Optimizing the emergency - no character space between specifically and optimizing
Response: Thank you for your suggestion. We have added character space between specifically and optimizing in line 360, we have also carefully checked the manuscript and corrected the textual problem in the revised edition.
- Figure 9. Diagram of Cooperative Emergency Decision-Making Method - need more clarity and precise figure with good text colour pattern to highlight all marking of object identification.
Response: Thank you for your advice. We were aware of the problem of unclear text expression of object identification in Figure 9 ,so we have revised Figure 9 to make the relevance between identification and objects more clear, reduce the redundancy of images, to facilitate reading. You can see the modified picture in the revised manuscript.
- line 326 Algorithm 1 gives the pseudo-code of the emergency disposal algorithm for highway accidents - Algorithm 1 where its algorithm one not understandable?
Response: Thank you for your advice. According to your suggestions, we have carefully arranged this algorithm, and added the corresponding algorithm pseudo code between line 395 and line 396, which could be found in the revised manuscript.
- line 351 explains what you mean by disposal time in your research work and why it's important in MADDPG to optimize it.
Response: Thank you for your suggestion. The disposal time shows the actual on-site emergency disposal process , so if the algorithm can improve the disposal time, it can realize the optimization of emergency disposal. We have added the the disposal time comparison between MADDPG algorithm and actual disposal, which reflects the advantages of MADDPG algorithm .You can see it from line 550 to line 563 in the revised manuscript.
- Table 4. Hyperparameters of Each Emergency Decision-making Algorithm - need more clarity and explanation for listed algorithms and Meaning in the given table by describing why MADDPG is related to noise intensity which showcases the difference with others algorithms meaning value why Hyperparameters are different for others algorithms. what is the benefit and advantages of MADDPG with noise intensity?
Response: Thank you for your advice. The algorithms' hyperparameters were adapted from classic parameter settings, tuning within the suggested range(Mitchell 1998, Beyer and Schwefel 2002, Mnih, Kavukcuoglu et al. 2013, Fan and Yan 2015, Mnih, Kavukcuoglu et al. 2015, Katoch, Chauhan et al. 2021). The following have introduced the setting criteria of hyperparameters in different algorithms. Meanwhile, we have added the basis for setting the hyperparameters of each algorithm from line 454 to line 478 in the revised manuscript.
1.Genetic algorithm(GA)
- Crossover rate
Suggested range: 0.4-0.99
Too high: the structure of individuals with high fitness in the population will be destroyed quickly, lead to the strong oscillation in strategy optimization.
Too small: the genes of population changes too slowly, and it is difficult to find an optimized strategy
- Variability
Suggested range: 0.0001-0.1
Too big: the process of optimization may turn out to be a random searching
Too small: fails to produce new genes
- Evolutionary strategy(ES)
For great similarity exist between GA and ES, the suggested hyper parameters are similar. However, if the variation intensity of ES is too large, the optimization will fluctuate violently, and if it is too small, the proceeding of optimization would be too slow.
- Deep Q network(DQN)
- Explore decay rate: the explore decay rate is the decay factor of -greedy strategy, which is an exploratory strategy designed to enhance the exploration of the environment by agents. At each time step, the agent with probability choose action randomly, otherwise choose action according to current strategy. The explore decay rate is exactly the decay factor of , where . The could be relatively large at the beginning of the training, and gradually get smaller as the training progresses. The explore decay rate is generally set to range from 0.9 to 0.995, we chose to set it to 0.995 to ensure that the agent has sufficient exploration of the environment.
- MADDPG
Being a member of DRL family, the MADDPG has a large number of sensitive hyperparameters, while most of them is similar with DQN hyperparameters. However, compared with DQN, noise intensity is a specific hyper parameter in MADDPG, and is specifically tuned in our proposed method. The noise intensity refers to the noise that needs to be added to the action during exploration. The explore noise is added to action when agent interacts with the environment and collects training data, such setting could improve the agent’s exploration of the environment.
- line 362, and 363 which are not the understandable way, After table 4 mentions the initial emergency disposal tasks involved in the three cases, and their initial statuses- making it a proper understanding for readers' perspective in terms of tabulation and explaining it more elaborate.
Response: Thank you for your suggestion. We were aware of the problem of unclear expression of the initial emergency disposal tasks involved in the three cases, and their initial statuses- making, we have added description of these process from line 425 to line 434, which have introduced how to transform the emergency scene into the initial state matrix, and you may find them in the revised manuscript.
- line 380 The training 379 super parameters of the above algorithms are shown in Table ??. - table number?
Response: Thank you for your advice, we were sorry for the table serial number missing in the manuscript, we have carefully checked the manuscript and corrected these problem in the revised one. We have filled the missing table number in line 453
- line 385 - The number of neurons in the three 385 hidden layers is 96128 and 64, respectively - why can't authors give a neural network diagram for good understanding? if it added well and will be good for new learners' and readers' perspectives.
Response: Thank you for your suggestion.According to your suggestions, we have added Figure 10 to describe the generation of emergency site state matrix and the application and network of MADDPG algorithm, meanwhile, we have also added a description to introduced Figure 10 for the convenience of readers, you may find them in the revised manuscript from line 479 to line 487.
- line 400 -A variety of emergency decision-making algorithms, including GA, 400 ES, DQN and MADDPG, have been trained by 1000 epides respectively - same as line 385 why can't authors give a neural network diagram for good understanding?
Response: Thank you for your suggestion.According to your suggestions, we have added Figure 10 to describe the generation of emergency site state matrix and the application and network of MADDPG algorithm, which can help readers understand the process of generating the emergency site state matrix, and understand how each agent in MADDPG processes these emergency site state data.
- line 466 - As shown in Table 7, the performance of each algorithm in different cases is consistent - where is table 7?
Response: Thank you for your advice, we were sorry for the table missing problem in the manuscript. We have filled the missing table number and established Table 6 which show the time step of different decision-making method in last training episode.
Reference
Mitchell, M. (1998). An introduction to genetic algorithms, MIT press.
Beyer, H.-G. and H.-P. Schwefel (2002). "Evolution strategies–a comprehensive introduction." Natural computing 1(1): 3-52.
Mnih, V., et al. (2013). "Playing atari with deep reinforcement learning." arXiv preprint arXiv:1312.5602.
Mnih, V., et al. (2015). "Human-level control through deep reinforcement learning." nature 518(7540): 529-533.
Fan, Q. and X. Yan (2015). "Self-Adaptive Differential Evolution Algorithm With Zoning Evolution of Control Parameters and Adaptive Mutation Strategies." IEEE transactions on cybernetics 46(1): 219-232.
Katoch, S., et al. (2021). "A review on genetic algorithm: past, present, and future." Multimedia Tools and Applications 80(5): 8091-8126.
Reviewer 4 Report
The paper suggests an interesting novel approach for a collaborative decision-making method in the field of highway emergency response. Therefore, it deserves to be published. However, text, tables figures, references, and equations must be checked for correctness and clarity.
Line Hints / Typos
24 iIncreasing -> increasing
26 The sentence in lines 26 -31 should be rewritten.
44 P é rez Gonz á lez -> Pérez González
124 diaposal -> disposal
270 net, At -> net. At
328 Please, give algorithm 1 in lines 328 – 337 more neatly arranged.
363 further illustrates -> further illustrate
380 Table ?? -> give appropriate Table No.
433 epides -> episodes
465 Table 7. > Table 7. Tables 6 and 7 are missing.
471 Optimize -> optimize
515 [? ] -> Please, give appropriate reference.
Author Response
- The paper suggests an interesting novel approach for a collaborative decision-making method in the field of highway emergency response. Therefore, it deserves to be published. However, text, tables figures, references, and equations must be checked for correctness and clarity.
Response: We would like to thank you for your valuable suggestions and comments in the revisions, it is grateful to see a much better manuscript had been revised according to your suggestions.
2.Line Hints / Typos
Line 24 iIncreasing -> increasing
Line 44 P é rez Gonz á lez -> Pérez González
Line 124 diaposal -> disposal
Line 270 net, At -> net. At
Line 363 further illustrates -> further illustrate
Line 433 epides -> episodes
Line 471 Optimize -> optimize
Response: Thank you for your advice, we were sorry for the text punctuation and capital letter problem in the manuscript, we have carefully checked the manuscript and corrected the textual problem in the revised one, capital letter rectification in line 48 , line 78, line 155, line 331 , line 411, line 540 and line 600 could be found in the revised manuscript. All the revisions have been labelled in red and we hope the quality of the manuscript could be improved.
3.Line Hints / Typos
Line 26 The sentence in lines 26 -31 should be rewritten.
Response: Thank you for your suggestion.According to your suggestions, we have carefully rewrote this sentence, reorganized the logic of this sentence, and updated the corresponding references, which could be found in the revised manuscript from line 50 toline 57.
4.Line Hints / Typos
Line 328 Please, give algorithm 1 in lines 328 – 337 more neatly arranged.
Response: Thank you for your suggestion.According to your suggestions, we have carefully arranged this algorithm, and added the corresponding algorithm pseudo code between line 395 and line 396, which could be found in the revised manuscript.
5.Line Hints / Typos
Line 380 Table ?? -> give appropriate Table No.
Line 465 Table 7. > Table 7. Tables 6 and 7 are missing.
Response: Thank you for your advice, we were sorry for the table serial number missing and table missing problem in the manuscript, we have carefully checked the manuscript and corrected these problem in the revised one. We have filled the missing table number and established Table 6 which show the time step of different decision-making method in last training episode.
6.Line Hints / Typos
Line 515 [? ] -> Please, give appropriate reference.
Response: Thank you for your suggestion.we were sorry for the missing serial number of references in the manuscript, we have deleted the question mark in line 712, because there is no reference in this part. Meanwhile, we have carefully checked the citation of full text references and we hope the quality of the manuscript could be improved.
