Human Factor Analysis of the Railway Traffic Operators
Round 1
Reviewer 1 Report
This is a well-authored manuscript so congratulations to the authors. I have a few comments/suggestions.
1) Line 77-79: Is this referring to only the Control rooms for rail operations or all types of control rooms? If it's specific in the reference, please be clear about it in the manuscript.
2) Line104-105: How are high-speed railway tasks different from the tasks treated in this paper?
3) Line 108: "Situation Awareness is a precondition of our analysis". How has this been fulfilled?
4) Line 326: The correct form is "Human Error Probability"
5) Line 334-336: How was consistency of estimations ensured?
6) This column ("HEP") on tables 5,6,&7 should be labeled as 'HEP Multipliers'.
7) It seems like the tasks described in section 2 (A-H ...J ... G) have been grouped into only 3 in section 3 i.e. Communication, ETD, and Interlocking. Why is this done? An explanation should be provided in the manuscript.
8) Line 408-409: Is there a specific table in the reference [17] where the point estimates of communication error probabilities are taken from? Placing them here will give better credibility to the results.
9) Table 9: Does "HEPn" mean Nominal HEP? Hence, it should be stated in the manuscript.
10) Table 12: The short form of 'Number' is 'No.' NOT 'No" Otherwise you can use '#' sign to represent 'Number'. Also 06-12, 12-18, 18-24, should be properly labeled with units i.e. 'Hours' or 'Hrs'.
11) Section 3.4: Kindly explain why 0.25, 1.56, and 0.54 was used in workplace, A, B, & C respectively. You can do that by making reference to tables 5,6,& 7.
12) Lines 515, 517, etc.: What does "ca" mean? Typo error? please correct.
13) PSF. Check this in Abbreviations. It is first used on line 146 of this manuscript. I think it should be 'Performance Shaping Factor'. Otherwise, the authors should give a reference where it (PSF) is "Power Shaping Factor" and how it is different from the "Performance Shaping Factor" in the HRA context.
Author Response
RESPONSES TO REVIEWER 1 (IN THE ENCLOSED DIRTY MANUSCRIPT DONE IN RED).
1) Line 77-79: Is this referring to only the Control rooms for rail operations or all types of control rooms? If it's specific in the reference, please be clear about it in the manuscript.
The source [29] is really referencing the specific control room (Railway Traffic Control Room in Nis, Serbia). To make it clear, this fact was mentioned in detail in lines 77-79:
Grozdanovic in [29] investigates specific operator-control desk interaction at the Railway Traffic Control Room (RTCR) in Nis, Serbia, using methods of anthropometric measurement of operators; determining the maximum strength of the operator's arm movements; workload analysis of operator's arms, head and trunk movements and error analysis of operators' movements in response to visual cues.
2) Line104-105: How are high-speed railway tasks different from the tasks treated in this paper?
There are some specificities.
Lines 110-115:
Centralized control systems in high-speed railways are more automatic and complex than in general speed railways; therefore, human error is the main factor in recent high-speed railway accidents. However, the conventional human error rate technique may have some weaknesses which are overcome by introducing hybrid methods for human error probability evaluation in high-speed railway dispatching tasks (sometimes mentioned as 3rd generation methods).
3) Line 108: "Situation Awareness is a precondition of our analysis". How has this been fulfilled?
The text about Situation Awareness has been reformulated and extended:
Rail signaling requires an accurate understanding of the system’s state that the operator controls to make correct, timely decisions and take effective action [33], Operators must do more than simply perceive the state of their environment. They must understand the integrated meaning of what they are perceiving in light of their goals. In dynamic environments, many decisions are required across a fairly narrow space of time, and tasks are dependent on an ongoing, up-to-date analysis of the environment. Because the state of the environment is constantly changing, often in complex ways, a major portion of the operator's job becomes that of obtaining and maintaining good Situation Awareness. Situation awareness is presented as a predominant concern in system operation, based on a descriptive view of decision-making. The decision-makers will act first to classify and understand a situation, immediately proceeding to action selection. Endsley [44] presents the model used to generate design implications for enhancing operator situation awareness.
[44] Endsley, Mica R. Toward a Theory of Situation Awareness in Dynamic Systems. Human Factors, 1995, 37(1), 32-64.
4) Line 326: The correct form is "Human Error Probability"
Our mistake – thank you for this remark, corrected.
5) Line 334-336: How was consistency of estimations ensured?
An explaining text added:
To ensure consistency of estimations, we tried to ensure the same or as similar observation conditions as possible: workplaces located in the neighborhood and on the same railway line, the same observer, the same operator within a specific workplace, and the same methodology.
6) This column ("HEP") on tables 5,6,&7 should be labeled as 'HEP Multipliers'.
Thanks, corrected.
7) It seems like the tasks described in section 2 (A-H ...J ... G) have been grouped into only 3 in section 3 i.e. Communication, ETD, and Interlocking. Why is this done? An explanation should be provided in the manuscript.
New text included:
The variety of work tasks resulting from different technological levels of control systems at Workplaces A, B, and C complicates their mutual comparison. To facilitate the comparison process, we decided to group the tasks into 3 groups. It is first necessary for the operators to recognize that something unusual has happened and to distinguish the relevant signals (functions of perception and discrimination) [17]. We treated this as primarily a display and communication problem. Having discerned that something unusual is happening, the operating personnel must diagnose the problem, decide what action to take, and carry out (function of response). Therefore we established comparable groups: Communication, ETD, and Interlocking.
8) Line 408-409: Is there a specific table in the reference [17] where the point estimates of communication error probabilities are taken from? Placing them here will give better credibility to the results.
An excellent and valuable remark!!! Thanks to it we realized we have made a formal mistake in eq. (5) where HSP values must be declared instead of HEPs (enumeration is ok and with added explanation should be clear now). A new table and explaining text were added:
Communication is one of the most frequently occurring activities that affect the operator's performance in managing the transportation process. Incorrect communication can directly impact the safety of the transportation process. The output of the analysis is the probabilities of correct or incorrect transmission of a certain number of instructions to cooperating employees. We drew data for communication analysis from Table 9, which is a transcription of Table 15-1, column c from the HRA Handbook [17], containing estimated probabilities of errors in recalling oral instruction items not written down. HEPn means HEPnominal.
Table 9. HEP and EP values for communication
|
Instruction No. |
HEPn |
Recommended EF |
|
1 |
0.001 |
3 |
|
2 |
0.006 |
3 |
|
3 |
0.03 |
5 |
|
4 |
0.1 |
5 |
An example of calculating the probability of error in communication with three instructions, applying eq. (1), is as follows:
HEP = 1 − HSP1 × HSP2 × HSP3
9) Table 9: Does "HEPn" mean Nominal HEP? Hence, it should be stated in the manuscript.
Yes, it is HEPnominal. This was explained in response to the previous question.
10) Table 12: The short form of 'Number' is 'No.' NOT 'No" Otherwise you can use '#' sign to represent 'Number'. Also 06-12, 12-18, 18-24, should be properly labeled with units i.e. 'Hours' or 'Hrs'.
Corrected. Similarly in original tables 14 and 15 (now tables 15 and 16).
11) Section 3.4: Kindly explain why 0.25, 1.56, and 0.54 was used in the workplace, A, B, & C respectively. You can do that by making reference to tables 5,6,& 7.
Line 433-434: The value of the used PSF was 0.25. It was achieved as the product of all HEP Multipliers in the last column of Table 5.
Line 446: The value of the used PSF was 1.56. It was achieved as the product of all HEP Multipliers in the last column of Table 6.
Line 462: The value of the used PSF was 0.54. It was achieved as the product of all HEP Multipliers in the last column of Table 7.
12) Lines 515, 517, etc.: What does "ca" mean? Typo error? please correct.
We wanted to say “cca”. Corrected to “about” in all cases.
13) PSF. Check this in Abbreviations. It is first used on line 146 of this manuscript. I think it should be 'Performance Shaping Factor'. Otherwise, the authors should give a reference where it (PSF) is the "Power Shaping Factor" and how it is different from the "Performance Shaping Factor" in the HRA context.
You’re right, it must be “Performance Shaping Factor”, corrected.
Author Response File: 
Author Response.pdf
Reviewer 2 Report
Please see the attached file.
Comments for author File: Comments.pdf
Author Response
RESPONSES TO REVIEWER 2 (IN THE ENCLOSED DIRTY MANUSCRIPT DONE IN BLUE)
The authors assess human error and reliability in railway traffic operations with different level of automation, using a technique for estimating human error probability (HEP) of the 1st generation.
The manuscript is well structured and written in almost perfect English (some exceptions see below).
Nevertheless, I have quite a number of issues which are all minor but need to be addressed before publication. Overall, the theoretical section (1. Introduction) and the methodological section (2. Materials and Methods) are of better quality of presentation than sections 3 (Results), 4 (Discussion), and section 5 (Conclusions).
Minor issues
Section 1 (Introduction)
(1) The cited literature is comprehensive and well-structured. I only suggest including two more classic pieces of Human Factor literature that are relevant to the field of the study:
- The "Swiss cheese model" of system failures and accidents by Reason. (Reason, J. (1990). Human Error. Cambridge University Press).
New text and reference added:
Accidents in complex systems occur through the accumulation of multiple factors and failures. James Reason [X] proposed the model of "Swiss cheese" to explain their occurrence as a series of factors that line up in just the wrong way, allowing seemingly-small details to add up to a major incident.
- The levels of automation by Parasuraman and colleagues that provide a framework and an objective basis for deciding which system functions should be automated and to what extent (Parasuraman, R., Sheridan, T. B., & Wickens, C. D. (2000). A model for types and levels of human interaction with automation. IEEE Transactions on systems, man, and cybernetics-Part A: Systems and Humans, 30(3), 286-297).
The latter should be included in the description of the three workplaces analyzed in the manuscript.
Control systems in railways, as in other fields, are of different types and use different technologies to determine their level of automation. Parasuraman et al. [46] provide a framework and an objective basis for deciding which system functions should be automated and to what extent. Three workplaces chosen for our analysis can be characterized as follows:
(2) Improvements of English language and verbal precision:
- Abstract, line 4: "We use a technique for Human Error Rate prediction of the 1st generation of Human Reliability Analysis.
Corrected (not in blue)
- Line 15: "determine" is too deterministic for human behavior. Use "affect" or "impact" instead.
Corrected
- Line 24: "viewpoint from which" instead of "angle at which".
Corrected
- Line 96: Include the term "semi-automated" from the abstract in the description of the relaybased system.
Included.
Section 2 (Materials and Methods)
Improvements of English language and verbal precision:
- Line 169: "estimate" instead of "determine". (Probabilities cannot be determined empirically.)
Corrected
- Section 2.1., lines 188-219: Use (repeat) the terms "manual", "semi-automated" and "almost fully automated" in the description of the three workplaces.
Corrected.
- Line 221-222: Put the URL in the references, i.e. assign a number and state the author and the date of access.
Corrected.
- Lines 233-256 and Table 1: Use exactly the same words as in the enumeration in lines 227-232 in order to help the reader to identify the types.
Corrected
All the terms "electro-mechanic" in the text have been changed to the uniform term "electro-mechanical" (not only in Table 1, but also Lines 8 and 94).
- Line 264: Leave out "In"
Corrected
- Lines 274 and following. Choose a different type of enumeration (A, B, …, H) from that of the workplaces (A, B, C).
Corrected
- Tables 2, 3, 4: Indicate the number of persons for which the operations were counted. (In a later section, the authors stated that it was one person, but it seems almost inconceivable that e.g. in Table 2 a single person conducted 5 operations with ETD in one minute, at 10.31 hrs.)
We are still talking about one person (this fact has been emphasized in the changed description of Tables 2,3 and 4). It may seem almost inconceivable, but only at first sight - it is real. For explanation, I am showing here an illustrative fragment of the entry into the Electronic Traffic Diary. A cell in 1 column corresponds to one task.
- Lines 356, 362: What does the symbol ¸ mean? Please explain or replace.
Replaced
- Figure 2, white box within the grey box: What does "≥ 1" mean? How can a probability be greater than one? Please explain.
Corrected
Section 3 (Results)
The presentation of results should be structured more clearly, the contents be presented more concisely, and the parts (Tables, Figures) of the subsections be better related to each other, to the Methods, and the Discussion. One problem might be some unmotivated changes of word usage, and lacking references between Tables, Figures and subsections. In detail:
- Figure 3-5, Table 8: Where in the later parts of the manuscript are these results taken up again for further evaluation or for discussion?
Due to the existing technological differences, the comparability of specific realized tasks is complicated, if not impossible. At the beginning of 3.1, we added a text (also as a reaction to another reviewer’s comments – in red) about why we decided to group the tasks into 3 groups. This grouping has made a comparison of performed tasks possible, at least on some approximate level. Also, a new text was added to the”
“Materials and Methods” part:
Tables 2, 3, and 4 indicate that the types of tasks are different at individual workplaces, and performing the same operation (e.g., setting up the main route) will require a different number of them. In order to make at least an approximate comparison of workplaces possible, we assume the subsequent grouping of tasks into more easily comparable groups of a similar type (Section 3.1).
“Discussion” part:
Figures 3, 4, and 5 show how the performed actions are distributed over time. For illustration, the same work shift (from 6:00 a.m. to 12:00 p.m.) was chosen at all 3 workplaces. Although neighboring workplaces on the same line were chosen, the volume of work (processed traffic volume) can hardly be considered the same (effects of specific local operations within the railway stations, time shift in the transfer of traffic load through the railway line, etc.). However, the results give an idea of the time distribution of the operator's workload in a given work shift. Table 8 gives a more comprehensive idea of the distribution of the operator's actions. The obtained data show:
- In the considered 3 groups of operations, there are no significant changes in individual work shifts
- The smallest number of operations is expected at workplace A, which is connected to their manual (and often physically demanding) nature; on the other hand, the operator at the semi-automated workplace B faces the biggest workload.
- Full automation at workplace C results in the lowest number of actions requiring the need for human communication and a significant limitation of actions related to ETD.
It is worth reminding that, from the safety point of view, the purpose of railway interlocking and signaling systems is to control or completely replace (if possible) the unreliable human factor, and therefore to reduce the number of actions performed by humans.
- Equation (5), after line 411: Why does the communication fail if and only if all three instructions are not understood (multiplicative term in the equation)? Please explain.
Corrected. There was a mistake in Eq.(5) – a special paragraph and table were added here to make it clear (as a previous reaction to another reviewer – in red).
- Line 417: "… we assigned nominal values of HEPs". How did you assign these values, i.e. why did you assign exactly these values, and on which theoretical grounds?
Related to and answered in the previous paragraph.
- Line 422. "… the values given in the tables": Which tables and which values do you refer to?
The confusing (and also incorrect) text was re-written and a new sub-chapter 3.14 has been added to show how all the text parts are interconnected and used for the calculation of the HEP value related to a particular task.
…
The previous Tables 10, 11, and 12 (given for Workplaces A, B, and C, respectively), show nominal HEPn values that are multiplied by recommended EF, giving us the resulting HEP and HSP values for each identified and considered operation.
3.14 “How to Apply” Example
Each task consists of a certain number of operations. Once we have all the data available, we can quantify human failure in performing individual tasks. Using the example of the task "setting up the main route", we can show how to apply the presented procedure. The “main route” means a route set up for a train. In the case of a semi-automatic relay-based control system operated at Workplace B, the task “setting up the main route” requires pressing two appropriate three-position buttons:
- The start button, delimiting the place at the main signal where the main route begins, and
- The end button, delimiting the place at the next signal where the main route ends.
Selection and pressing of the buttons are carried out on the control panel.
We must create a task tree representing the procedure of the task. In this case, we can use the task tree that has already been given and explained in Figure 1. The operator selects from a large number of buttons, which are distinguished by the colors of the button heads (in our case either green or white) and the names of the assigned signals to which they belong.
The calculation of HEP for the task in question is as follows:
??? = 1 − ???1 ∗ ???2 = 1 − 0.985 ∗ 0.985 = 0.03.
The calculation of the probability of successful task performance is:
??? = ???1 ∗ ???2 = 0.97.
where HSP1 = 0.985 is Human Success Probability calculated and declared in Table 11 for the correct operation of the 3-position return button (here considered as a start button), and similarly, HSP2 = 0.985 is Human Success Probability calculated and declared in Table 11 for the correct operation of the 3-position return button (here considered as an end button).
The next step includes taking into account the PSF factor, which was set to a value of 1.56 for Workplace B (achieved as the product of all HEP Multipliers in the last column of Table 6).
Then the final value of HEP for setting up the main route at Workplace B can be reached as:
??? = 0.03 ∗ 1.56 = 0.047,
which means that the operator will make an error in approximately 4.7% of cases when performing the task in question.
3.15 Final Overview of Most Frequent Tasks
The same approach as presented in Section 3.4 was applied to the most frequently performed tasks at each Workplace. The overview of the final results for Workplace A is available in Table 13.
…
- Tables 12-14: Di the tasks indicated in the first column relate to any of the tasks described earlier in the paper? Please use identical words for identical concepts as much as possible, and/or give proper cross-references within the manuscript.
After answering the previous comment, this should be clear now. The inserted application example cross-referenced the whole procedure through the text.
Section 4 (Discussion)
In the discussion section, no new results should be presented.
- Line 493. "Already mentioned above". Where exactly?
Corrected. The statement “Already mentioned above” was removed as redundant (the way of getting PSF values was explained and emphasized in the previous manuscript text).
The “Discussion” part was newly re-written.
- Figure 6: The content of the figure is nothing else than the contents of the Tables 5-7. The figure can thus be left out and be replaced by a simple reference to the Tables. Alternatively, replace the Tables 5-7 in section 2 by the Figure 6. If you keep the figure, use exactly the same words in the Figure legend as in the Tables 5-7. Moreover, there is no discussion of the values visualized in Figure 6 (stated in Tables 5-7). Please discuss them here.
Figure 6 was removed. Discussion re-written.
- Figures 7-9: Please do not present new results in the Discussion section. Either put the Figures in the Results section and discuss them here (what role do they play in your main line of reasoning?), or leave them out, if they have no impact for your main line of reasoning. Again, be sure to use the same words in the figure legends as in the main text.
Figures moved to Results. Some legends changed.
- Lines 498, 512, 519, 529, 533, 538, 548: The headings of the subsections 4.1 -4.7 (Setting up the Main Route, etc.) are not clearly related to the main line of reasoning / the rationale of the paper. Please indicate more clearly why you are discussing this point here.
Moved to Results
- Line 506: "50 to 70 tasks". Please give a reference for these numbers within the paper. The same holds for the HEP values in this paragraph.
Added references and explanations.
- After Line 567: Please discuss limitations of your methods and results in a special subsection of the Discussion section.
Included
Section 5 (Conclusions)
What are the implications of your study for Human Factors theory and methodology, and for practice of railway operations? Please discuss briefly.
The optional part Conclusions was canceled, implications included in the Discussion part.
For details, please, see dirty version of the manuscript (blue texts are responses to above).
Author Response File: Author Response.pdf
Reviewer 3 Report
The paper present interest for expert from railway domain. The human errors can have a large implication in ensuring the safety of train traffic. The method used by the authors is interesting and is in correlation with other research methods.
The results are well presented. However, a better graphic representation helps the readers to understand the conclusions expressed by the authors.
Also in the introduction I think that a reference to Markov chains is useful. The method presented in this paper can be extended in the future using the interaction between the equipment and the human operator.
Author Response
RESPONSES TO REVIEWER 3 (IN THE ENCLOSED DIRTY MANUSCRIPT DONE IN CYAN)
The paper presents interest for an expert from the railway domain. Human errors can have a large implication in ensuring the safety of train traffic. The method used by the authors is interesting and is in correlation with other research methods.
The results are well presented. However, a better graphic representation helps the readers to understand the conclusions expressed by the authors.
Also in the introduction I think that a reference to Markov chains is useful. The method presented in this paper can be extended in the future using the interaction between the equipment and the human operator.
A new text added:
The method presented in this paper can be extended in the future using the interaction between the equipment and the human operator. Chen et al. [45] facilitate a Hidden Markov model on top of a human cognitive model to capture the sequential faults of a production line worker, who suffers from work stress. The Markov chain has the discrepancy of time and state. This characteristic is consistent with the changing law of human factors and can be used to predict the risk of human factors. A human factor evaluation model based on the Set Pair Analysis method and the Markov chain was proposed and applied by [46].
[45] Chen, C.-N.; Liu, T.-K.; Chen, Y.J. Human-Machine Interaction: Adapted Safety Assistance in Mentality Using Hidden Markov Chain and Petri Net. Appl. Sci. 2019, 9, 5066. https://doi.org/10.3390/app9235066
[46] Purkait, M.K.; Mondal, P; Changmai, M.; Volli, V.; Shu Chi-Min. Hazards and Safety in Process Industries. Case Studies. 1st ed.; CRC Press, 2021; pp. 267-296.
Author Response File: Author Response.pdf
