Determining the Reliability of Critical Controls in Construction Projects

Abstract

Across the global construction industry, fatalities continue to occur from high-risk activities, where the risk controls have been defined; however, these were unreliable. In the mining industry, Critical Control Risk Management has provided positive results in reducing major accidents, which raises the question, could the Critical Control approach reduce the fatality rate in the construction industry? This study analyzed 10 years of serious and fatal incident investigation reports from four international construction companies to (i) assess the reliability of their Critical Controls (CCs) and (ii) assess the factors that affect the reliability of CCs. The results show the reliability of CCs, measured by implementation and effectiveness, averaged just 42%. Insight into human performance and organizational factors, including risk identification, decision-making and competency, together with supervision, job planning and communication, were identified as opportunities to improve the reliability of CCs. The study used bowtie diagrams with real event data to find the actual CC reliability. This appears to be the first published study that reports on the reliability of critical risk controls in construction. It demonstrates a feasible method for determining and communicating control effectiveness that can be used to deliver meaningful insights to industry practitioners on actual control performance and focus areas for improvement. In addition, actionable findings directly related to individual CCs can be derived that enable the participating organization to focus resources on improving specific verification processes. The results confirm the applicability of CCs for the Major Accident Event hazards analyzed and highlights that further reviews are required on the factors that need to be considered when implementing a CC program. This paper details our methodology and results, to assist others applying CCs as a risk management tool.

1. Introduction

Accident prevention research has identified complex models of accident causation [1], identifying multiple factors and numerous safety controls [2,3] to prevent incidents from recurring. However, within the construction industry, serious and fatal incidents continue to result from recurring causes [4]. Construction industry fatalities result from high-risk work activities (e.g., operating heavy plant machinery, lifting using cranes, working at height [5]), where the interaction between human factors and the activity gives rise to personal-safety-related fatalities. Equally, construction risk management strategies designed to prevent fatality events (e.g., Life Saving Rules) have relied on human action and interventions to identify hazards, assess risks and then treat the risk by defining and applying controls in the workplace [6,7].

Human actions and interventions can introduce errors through variability in hazard identification and assessment within dynamic construction environments [8,9], with workers identifying, on average, only 53% of fatal hazards in the workplace [10]. In addition, human factors affect the compliance to critical controls [11], risk tolerance [12] and decision-making [13], all of which influence the efficacy of control implementation and effectiveness. Selleck and Cattani [14] proposed the construction industry focus on risk treatment and applying a critical control approach to prevent fatalities and to learn from similar programs being applied to process safety in the oil and gas or mining industries. Critical Controls (CCs) are specific safety barriers, which (i) directly prevent the unplanned release of energy, which cause major accident events, (ii) directly prevent the escalation of event consequences or (iii) are unique controls within an event pathway.

The concept of safety barriers as a method of preventing and mitigating unwanted events has been used extensively to identify the controls needed to address event causes and consequences [15]. The bowtie method is often used to facilitate the identification of controls for an unwanted event. The bowtie method was developed by joining fault tree and event tree (cause and consequences) surrounding an unwanted event [16]. The bowtie method has been used extensively in the aviation, nuclear, oil and gas and chemical processing facilities to assess potential failure modes and quantify the adequacy of controls to prevent accidents through risk assessment estimation techniques [7,17,18]. The process industries have an established practice of identifying barriers as independent protection layers, with a preference for hardware and technology reliability as barrier controls over human reliability. The barriers are perceived as discrete onion-like layers, formed by mechanical devices, instruments, alarms, administrative controls and post- release mitigation measures, all acting independently [7]. However, an underlying factor is the influence of human action and organizational factors, which affect the reliability of the barrier [7,19,20,21].

The reliability of control barriers is influenced by organizational psychological mechanisms, such as confirmation bias, normalization of warnings, consensus mode decision- making and group think, which occurs within work teams and across organizations [19]. Reliability of barriers is also affected by human factors (e.g., competence) and human actions in the detection of threats or changes in barrier functionality, diagnose what action is required and then act [20,21,22]. Construction accident causation analysis [23] identified worker actions are heavily influenced by supervision and risk management through planning and risk control at different levels across the organization, emphasizing the need for a holistic approach to managing fatal risks and the use of barriers.

The safety barrier methodology has been applied in the mining industry through Critical Control Risk Management (CCRM). CCRM is focused on risk treatment by specifying and verifying the implementation and effectiveness of critical controls (barriers) in a model addressing organizational and inherent human factors using the principals of High-Reliability Organizations [24,25,26]. An adaptation of CCRM was piloted on an Australian construction project [27]; however, further understanding of Critical Control reliability was identified.

For construction organizations to invest in the development and implementation of a safety barrier approach such as CCRM, organizational leaders who are accountable for fatality prevention will need assurance that the controls being defined will prevent fatalities (are they the ‘right’ controls?) and how will the reliability of the controls be measured? Hassall et. al. [24], in a study on selection and optimization of risk controls, identified control performance as the product of reliability in the control to perform within the work environment and the adequacy of the control to prevent and/or mitigate unwanted events across normal and abnormal situations (Figure 1). When considered in the context of construction fatalities, regulators across multiple jurisdictions reported that between 85% and 90% of fatalities are events occurring from common high-risk activities, where controls that prevent the incident are defined within organization safety management systems, but still result in single to two-person fatalities [4,28,29]. In summary, construction industry fatality events continue to be caused by the same high-risk activities and hazards due to failures in control reliability and less from novel or abnormal situations, where they are not defined or are inadequate in preventing the novel events.

Risk-control reliability is a factor of the availability and use of the control when required (i.e., control is implemented) and the effectiveness of the control to eliminate or minimize exposure to a threat or mitigate consequence severity [24]. Measuring reliability of risk controls can apply quantitative [6,30], semi-quantitative [7,21,31,32] and qualitative processes [24], depending upon the control to be measured, if the events where a control is challenged in the normal environment can be tracked or if the control can be tested under controlled conditions. In the simplest form, the effectiveness of a risk control is the ratio of the number of failures of the control when challenged to the number of occasions the control was challenged [32], i.e.,:

$$\mathrm{Risk}\mathrm{control}\mathrm{effectiveness}=1-\frac{\mathrm{Number}\mathrm{of}\mathrm{failures}\mathrm{of}\mathrm{the}\mathrm{control}\mathrm{when}\mathrm{challenged}}{\mathrm{Number}\mathrm{of}\mathrm{occasions}\mathrm{the}\mathrm{control}\mathrm{was}\mathrm{challenged}}$$

(1)

However, when being considered within a safety barrier program by a construction organization, the implementation and/or availability of the control needs to be assessed together with the adequacy of the control. Figure 1 provides a visual representation of control performance measurements, as derived from Hassall et al. [24].

The reliability of the control (barrier) is also a factor of the type of barrier being used and the interdependency on human action and the effectiveness of the safety management system supporting the reliability of the barrier [33]. The selection of barriers will apply the Hierarchy of Control as a means of reducing the risk of an event and improving the reliability of the barrier by selecting the most effective control type practicably available [34]. Controls related to hardware barriers (i.e., physical and/or engineered control mechanisms or systems) only have indirect human involvement and are less likely to contribute to accidents [21]. Despite the importance of understanding control reliability, no publications could be found that analyze the reliability of the suite of controls used to manage construction-related fatality risks. This study aims to begin to address this gap by first developing and testing a practical method for industry practitioners to use to determine control reliability. It also explores the use of bowtie diagrams as a means of presenting the results in a manner that helps reveal insights on risk-control vulnerabilities to decision makers. Specifically, the study explores critical control reliability through analysis of historical fatality and serious-event investigations across four construction organizations to understand the historical performance of critical controls. The study aims to:

• Evaluate if known critical controls, as documented within existing high-risk activity performance standards and organizational specifications, address known construction safety risks.
• Identify performance factors that affect the reliability of critical controls to assist in the implementation of safety barrier programs within construction organizations.

2. Methods

The research involved sourcing incident investigations for potential accidents, coding the investigation reports and analysis of CC reliability as outlined in Figure 2.

Incident investigation reports for serious and fatal incidents which occurred over a 10-year period (2010 to 2020) were sourced from four construction companies based in Australia, South Africa, Canada and the USA. To be included in the study the incidents had potentially fatal consequences and a root cause analysis investigation had been completed. Sourcing of actual, potentially fatal consequence investigation reports from construction companies is problematic as the reports are highly confidential and often subject to legal privilege. The value of the research to the participating organizations and grouping of data across multiple companies made the research possible given the sensitivity of the events and causal factors. Alternative open-source data sources were explored (e.g., FACE database [35]); however, details on root causes were not able to be mapped to CCs and information was unclear on CC reliability factors.

The incident investigation reports were collated by the researcher for analysis by a focus group panel of four HSE professionals. The incident investigation reports were grouped into 11 event categories (e.g., Working at Height) and assessed to determine if the quality of the report was sufficiently detailed to identify Major Accident Event (‘MAE’) hazards, applicable controls and causal factors. If the investigation report details were insufficient, the report was excluded from the study.

The four HSE professionals (i.e., 1 representative from each company) each had more than 15 years’ construction experience (ranging from 15 to 25 years of experience) including competence in incident investigation, which enabled the analysis of the incident investigation reports. The HSE professionals were assigned an MAE category and assessed all incident events from all four companies applicable to the category. The focus group members were trained in the assessment methodology using worked examples with a follow-up session once five incident event assessments had been completed by each member to ensure alignment and consistency of assessment and coding of the events. A workshop was conducted following the completion of all analysis, where the outcomes were reviewed from each event and focus group members challenged the assessment rating until consensus was reached.

2.1. Critical Control Categorization and Assessment

Each investigation report was assessed to determine the mechanism (s) of failure to match the event to the MAE hazard (threat), then compare controls detailed in the investigation to known Critical Controls (CC) defined in the Major Accident Prevention (MAP) model (Figure 3, event classification method). Each applicable CC was assessed to:

• Determine if the CC was a contributing factor in the event (yes/no).
• Determine if CC had been implemented (yes/no).
• Had the CC performed as required, i.e., was it adequate to prevent events using a rating of good, needs improvement or inadequate?
• What was the mechanism of the injury? (List)
• Were improvements in the of CCs required? (Yes/no, free text)
• Were any improvements in CC specification or additional CCs required? (Free text)

The mechanism of injury list used in the assessment process was in accordance with AS1885.1-1990 [36]. The assessment also identified improvements in the application of CCs and gaps in the MAP model of CCs. The free-text comments were reviewed by the panel and collated into common themes. The assessment details were recorded in an online Microsoft Form^® database stored on a secure site.

2.2. Transformation of CC-Coded Data

Each MAE category data group was exported into MS Excel for transformation and consolidation by CCs. Scoring of the data was applied by converting the three CC assessment ratings into numerical values as per

Table 1. CC data transformation.

Event Assessment Criteria	Code Response	Output Variable	Calculated Output
CC Contributing factor	Yes	Number of times CC Challenged	∑n(CC = “yes”)
	No	-	-
CC Implemented	Yes	Number of times CC implemented	∑n(CC = “yes”)
	No	Number of times CC not implemented	∑n(CC = “no”)
CC Adequacy rating	Inadequate	Adequacy score	∑n(CC = “adequate”)
	Poor
	Good

NOTE: n = number of times CC was assessed.

to produce the variables used for calculating control reliability.

2.3. Calculating Control Reliability

Two values were calculated from the CC assessment and converted into percentage values:

$$\mathrm{Implementation}\mathrm{ratio}(\%)=\frac{\mathrm{number}\mathrm{of}\mathrm{times}\mathrm{the}\mathrm{CC}\mathrm{was}\mathrm{implemented}\times 100}{\mathrm{number}\mathrm{of}\mathrm{occasions}\mathrm{the}\mathrm{control}\mathrm{was}\mathrm{challenged}}$$

(2)

$$\mathrm{CC}\mathrm{effectiveness}(\%)=\frac{\mathrm{number}\mathrm{of}\mathrm{times}\mathrm{the}\mathrm{CC}\mathrm{was}\mathrm{rated}\mathrm{adequate}\left(\mathrm{i}.\mathrm{e}.,\mathrm{good}\right)\times 100}{\mathrm{number}\mathrm{of}\mathrm{occasions}\mathrm{the}\mathrm{control}\mathrm{was}\mathrm{challenged}}$$

(3)

CC Reliability (%) = Implementation ratio × CC effectiveness ratio

(4)

Critical control reliability percentages were mapped against the MAE hazard Bowtie. The mapping of the result from applying real data calculations for individual CC reliability to the bowtie is a novel extension of bowtie analysis that visually highlighted control gaps and provided feedback on the performance of control pathways and improvements required in the verification processes.

2.4. Failure Rate by CC Hierarchy of Control Type

CC reliability ratings were compared by hierarchy of control type of CC for each MAE category to review the reliability of CC type. Observations on critical control gaps and improvements were collated and provided to the participating organizations.

2.5. CC Comparative Performance by Implementation and Effectiveness Ratios

The data were analyzed using R statistical package [37] applying exploratory analysis steps to understand the relationships and strength of relationships between variables. One-way ANOVA was applied to the implementation and effectiveness ratio (%) variables to understand the importance of the measures in assisting construction projects to improve CC management.

2.6. Human Ethics Statement

The research was conducted in accordance with Edith Cowan University Human Research Ethics Committee (HREC) approval for Project number 20293 Selleck granted on 12 June 2018 (valid from 12 June 2018 to 31 March 2022) which meets the requirements of the National Statement on Ethical Conduct in Human Research.

No harm has resulted from the focus group process or the analysis of the reports.

3. Results

Sourcing of actual potentially fatal consequence investigation reports from construction companies is problematic, as the reports are highly confidential and often subject to legal privilege. The value of the research to the participating organizations and grouping of data across multiple companies made the research possible given the sensitivity of the event and causal factors. This resulted in 186 serious and fatal event investigation reports collated, covering a period from July 2011 to December 2019. Five investigations were rejected due to insufficient detail on contributing causes. The events were sorted by MAE category (

Table 2. Number of L4/L5 Event Reports by category.

Fatal Risk (MAEs)	Number of Events	Fatal Risk (MAEs)	Number of Events
Excavations	3	Marine Operations	2
Fall of Ground	12	Mobile Equipment	30
Fire & Explosion	4	Falling & Rolling Objects	1
Lifting Operations	49	Stored Energy	33
Light Vehicle	10	Working at Height	36
Machinery & Equipment Safeguarding	0

), with all events assessed; however, statistical analysis was limited to MAE categories where there were greater than 30 event reports, which included: Lifting Operations; Mobile Equipment/Light Vehicles; Stored Energy and Working at Height.

Lifting Operation comprised the strongest frequency rate (27%) of all events and mobile equipment, stored energy and working at height represented, collectively, 87% of all events analyzed. Where the event report did not provide sufficient information to assess the event or the event related to another failure mode, these were rejected (

Table 3. Data analyzed by MAE category.

High Risk Activity	Number of Event Report	% Events	Number Rejected
Operating Mobile Plant and Equipment	40	22%	4
Lifting Operations	49	27%	0
Stored Energy	33	18%	7
Working at Height	36	20%	5

).

The most frequent MAE hazards included ‘driving interactions and operator error’, ‘lifting operations—dropped load’, ‘uncontrolled electrical energy release’ and ‘falls due to access/egress from plant or unstable ground’ (Figure 4).

3.1. Critical Control Performance Measures

Implementation of CCs across all MAE categories was analyzed at an average 57%, with a standard deviation of +/− 35.5%, indicating considerable variation in the implementation of CCs. Effectiveness of the CCs when implemented averaged 41.2%, with a standard deviation of +/− 38.6%. (Figure 5). The performance of CCs had limited reliability (23%), with a high rate of variability (+/− 37%) in preventing or mitigating MAE threats or consequences.

Comparison of CC performance measures (implementation, effectiveness) by MAE category (Figure 6) identified Mobile Equipment (77%) and Lifting Operations (77.2%) as having the strongest CC implementation rate, with Stored Energy (24.5%) having, overall, the weakest CC implementation rate. The CC effectiveness rate was, on average, 30% lower than the CC implementation across the MAE categories, except for Stored Energy.

Comparison of CCs by the hierarchy of control types was conducted using the control types defined in the CC data set used for comparative analysis [27]. In this model, the higher levels of the hierarchy of controls ‘elimination’ and ‘substitution’ are not applied, as the focus is on action and verification in the field. The ‘administrative’ controls are broken down by the action taken, e.g., ‘inspection’, ‘monitoring’, ‘procedural’ and ‘competency’. The comparison identified Engineering controls as having the strongest rate of implementation (73.3%), with the other control types ranging between 46.7% and 53.6%. Engineering and administrative procedural CCs had similar effectiveness ratings at 47.6% and 45.5%, with the rest performing between 34.5% and 35.9% (Figure 7a–d).

Engineering controls are closely monitored by field construction managers and project engineers as part of monitoring the integrity of the facility being constructed, with the extra focus reflected in the higher implementation rate and, to a lesser extent, the adequacy of the engineering controls compared to the other control types.

Comparing hierarchy of CC types across the MAE categories, the Stored Energy CCs have a consistently lower rate of implementation yet deliver a higher rate of effectiveness (Figure 7c).

The best performing control type was procedural CCs when conducting Lifting Operations, with a high rate of implementation (69.3%) and being effective 79.1% of the time (Figure 7a). The weakest performance was monitoring controls in the Stored Energy MAE category, with no monitoring type of controls across the category having been implemented (e.g., verify monitoring of pressurized systems to be within design and test limits).

Implementation of administrative-type controls (e.g., competency) in the Mobile Plant and Equipment was 97.0% (Figure 7b) and Lifting Operations (Figure 7a) was analyzed at 82.2%, demonstrating a high rate of implementation compliance. The effectiveness of the CCs for the same control type once implemented was weak, with Lifting Operations competency controls only being effective 28.3% of the time when implemented and Mobile Plant and Equipment only 62.5% of the time.

In total, 119 CCs were assessed across the four MAE categories, and all were found to be a primary causal factor in a minimum of one MAE incident when the CC was not implemented or effective. This was a fundamental assessment of whether controls being evaluated were Critical Controls. It was observed that CCs could also be contributory factors in MAE incidents.

3.2. Lifting Operations

Lifting Operations MAE hazards had the strongest level of implementation for CCs, with 15 of the 32 CCs having a greater than 80% implementation rate, with an overall average of 77.2% (Figure 6).

Activities involving the stability of the crane or lifting device had the lowest rate of CC implementation, 48.8% of CCs. Lift plans, risk assessment, inspection of ground conditions, stability devices and exclusion zones had low-implementation ratings. The CCs applicable to the design of hoists, lifts and winders used in construction to move people and materials had a low-implementation rate at 44.4%. Two CCs involved in managing moving and swinging loads specifically, line of fire risk assessments and assessing environmental conditions had low-implementation rates at 57.1% and 50%, respectively. Work pressure was identified as a contributory cause in lifting events due to limited windows in the day’s schedule being available to complete lifts.

The effectiveness of lifting operation CCs has an overall average of 46%, with five CCs being 100% effective and six CCs being 0% effective (Figure 8). The stability of crane or lifting devices has the least level of prevention control, with six of the seven CCs having weak effectiveness ratings, with an average of 11.5%.

All Lifting Operation MAE hazards compromised CC prevention pathways with two or more CC effectiveness measures being compromised by having a 50% or lower failure rate when the control is implemented (Figure 8).

3.3. Mobile Plant and Equipment

Mobile Plant and Equipment MAE hazards had, overall, a high level of implementation of CCs, with 11 of the 22 CCs having a greater than 80% implementation rate, with an overall average of 77% (Figure 9), marginally behind Lifting Operations (Figure 5).

CCs that managed Operator Error Hazards had a strong implementation average at 85%, with the two lower implementation rates (67%) associated with operating within vehicle specifications and driving off road with roll-over protection. The weakest level of CC implementation (47%) was heavy vehicles or plant operators not responding to alarms. The CC implementation and effectiveness ratings for the Unsecured Loads MAE hazard are indicative only as the CCs were only challenged twice by the assessment of incident events.

The effectiveness of mobile plant and equipment CCs has an overall average of 48%, with two CCs associated with emergency response drills being 100% effective and two CCs (excluding the Unsecured Loads MAE hazards noted above) being 0% effective. One of the two completely not effective was heavy vehicles or plant operators not responding to alarms (Figure 9).

The vehicle failure MAE hazard has the strongest level of prevention control, with all CCs in the prevention pathway having CC effectiveness ratings above 62%, with an average of 74.9%. The least effective prevention pathway is associated with driving on site, where the effectiveness ratings of five from seven are weak (<50%) and range between 0 and 33.3% (Figure 9). Three of the four Mobile Plant and Equipment MAE hazards have compromised CC prevention pathways, with two or more CC effectiveness measures being compromised by having a 50% or lower failure rate when the control is implemented.

3.4. Stored Energy

In total, 36 major accident events were analyzed that were associated with stored energy. Uncontrolled Electrical Energy Release was the most common MAE event by which personnel were harmed, with inadequate isolation methods and application of exclusion zones around live systems. Contributing to the failure of isolation methods was due to perceived schedule pressure, either from the issuing of permits without full validation (“we needed to get the permit issued as work had already been held up”) or isolation placed on the wrong system (“crew were waiting to start”).

An average 4.4% of events analyzed identified the critical controls as not implemented as the primary failure. Failure to apply isolations and/or exclusion zones was identified as a common failure across all Stored Energy MAEs (Figure 10).

3.5. Working at Height

In total, 36 working at height major accident events were analyzed. Falling Down: access and egress and working on unstable ground together with Dropped Objects were the most common MAE events by which personnel were harmed due to inadequate design of access/egress, inspections and maintaining exclusion zones and inadequate risk and simultaneous operation assessments.

An average 10.5% of events analyzed identified the critical controls were not implemented as the primary failure. Failure to undertake inspections of work environment, pre-start/pre-use inspections and fall protection and inadequate job planning were identified as a common failure across all Working at Height MAEs (Figure 11).

3.6. CC Improvements and Gap Analysis

In total, eighteen (18) CC performance improvements and nine (9) gaps in CC specifications were identified during the event analysis (

Table 4. Identified improvements and gaps in Critical Controls by MAE category.

	Identified CC Performance Improvement	Identified Gaps in CC Specifications
Lifting Operations	The quality, definition and details provided in lift plans Identifying, delineating and communication of line of fire exclusion zones Control of exclusion zones (requirement for trained and competent spotters) Communication between crane operators and riggers Competency of crane operators/riggers used for the task being performed	Mechanical locking system mandated for storage of crane booms during transit Overhaul and/or major maintenance service of lifting devices to apply NDT to all critical welds and joints and ensure lubrication/inspection of critical components. Safety critical materials (e.g., rigging components) required for lifting operations are identified, sourced, and applied as designed.
Mobile Plant and Equipment	Load factors for trucks and mobile equipment not defined or applied in work activities Malfunction of automated processes, vehicle proximity alerts/alarms—inadequate inspection, maintenance, and testing. Deliberately disabled. Operator fitness for work—fatigue, under the influence of drugs/alcohol, mental distraction, and physical conditions. Personnel operating within blind spots, line of fire and inadequate use of the spotters for tramming, reversing, and loading/unloading operations Inadequate traffic/pedestrian segregation	Development of loading/unloading critical controls—positions/lifting/offloading with heavy equipment
Stored Energy	Personal discipline to use isolations and lock out system. Identification, installation, and monitoring of exclusion zones Permit to work application—wrong systems identified, systems not de-energized and inadequate lock out/tag out.	Risk assessments extend beyond project perimeter to include tramming route of mobile equipment (e.g., overhead power lines) Line of fire risk assessments to include securing systems (e.g., chains, clamps)
Working at Height	Line of fire assessment Engineering and design reviews of new scaffolding/barrier systems Competency of personnel installation/using scaffolding (e.g., overloading) and managing materials, tools and equipment when working at height Integrity of work surfaces—multiple trips/slips on work platforms Design of working at height systems, anchor points and hookup by work team members	Design, inspection and loading specifications of temporary works including loading platforms

). The recommendations provide insight into the type of errors that contributed to the incident events, including design failures, system errors and human factors. The CC performance improvements and gaps were not able to be validated beyond the statements provided within the historical investigation reports. The recommendations provide insight into the type of CC performance errors occurring historically and areas for management focus in current projects.

4. Discussion

The research evaluated historical incident investigation reports of significant construction incidents for four international construction companies across a ten-year period. The study evaluated known CCs, as documented in existing high-risk activity performance standards, to identify performance factors that affect the reliability of CCs. The relative control reliability level for each of the CCs was calculated to provide a baseline measure for future assessment of construction critical controls. The analysis does provide insights into the applicability of CCs for the construction industry and factors affecting CC reliability across the different hazard categories.

4.1. Validity of Construction CCs

One of the key questions asked by the construction companies participating in the study and one of the aims of the study was to determine whether the CCs being applied in their organizations are the ‘right’ CCs to prevent major-accident events. The CC verification process requires management investment in resources to undertake the verification tasks, monitor performance, report on the risks and is expected to demonstrate management duty of care in respect to MAE risks. The CCs applied in the companies were reviewed by internal construction and safety professionals. However, no definitive review against major-incident events was conducted and the organizations continued to experience significant incidents post the implementation of the CC verification process. The study confirmed all 119 CCs being applied by the organizations were valid, with a further 7 CCs being recommended. The additional seven CCs were recommended for MAE hazards where the threat was not identified (e.g., loading/unloading from haulage vehicles) or there were gaps in the control specification.

The type of CC gaps occurred across a range of control types, including engineering, inspection and procedural, which focus on the higher end of hierarchy of controls. By contrast, observations on factors affecting implementation of the CCs identified gaps in lower-level hierarchy controls. The gaps included procedural, administrative and training associated with human performance factors, resulting in CCs not being implemented.

All four major-accident event categories were found to have a high proportion of weakly or not implemented Critical Controls and, therefore, were not effective in preventing the release of hazardous energies. The CCs rated ‘weak’ (<50% reliability) were considered unreliable as they failed more times than the CC was effective. The ratings (weak, needs improvement, strong) highlight where construction organizations need to prioritize action to improve implementation and the quality of the CC being considered. The ratings also inform where CC verification programs need to prioritize organizational effort to validate CC reliability. In the case of Working at Height events (Figure 8), three of the control pathways (i.e., falling down, working from scaffolding, working from man cage) identified each Critical Control as being weakly implemented or not effective. For example, Falling from Scaffold identified three CCs as being implemented: design of the scaffold, inspections on standard of scaffold being built and scaffold foundation inspections; however, only the design CC was assessed as being only 20% effective. Similarly, when assessed in the overall context of the study, the CCs that had a high reliance on human performance (e.g., operating plant and vehicles, inspections, maintaining exclusion zones) had a higher rate of failure (Figure 6), which aligns to hierarchy of control principles [23]. Human performance factors that affect either the implementation or quality of the Critical Control, including decisions to intervene when a CC is not performing as specified, need further consideration.

The Stored Energy hazard category provides a case in point, with Stored Energy events having the least proportion (18%) of incident events in the study. Arguably, Stored Energy should have the best CC performance. Comparing hierarchy of CC type across the MAE categories, the Stored Energy CCs have a consistently lower rate of implementation yet deliver a higher rate of effectiveness (Figure 6). All four Stored Energy MAE hazards had a minimum of two CCs assessed as having a 100% reliability rating (Figure 8). These CCs were engineering and inspection-type controls and, whilst overall more effective in the absence of other CCs (i.e., those relying on human performance), the incident events still occurred.

4.2. Human Performance Factors

The analysis of the incident investigation reports identified a range of organizational, supervisory and human performance factors contributing to poor implementation of CCs. Eighteen (18) recommendations on improving implementation of CCs (Table 3) provide insight into the type of human performance factors affecting CC implementation and effectiveness. These are observations made by the experienced panel members to assist construction organizations intending to implement CCRM or improve management focus on the verification of CCs.

The failure to recognize hazards was identified across multiple incidents, particularly when working in and around mobile plants, where personnel were working in blind spots (reversing plant), in the line of fire (swinging loads), during loading/unloading of equipment and working above others. Failure to recognize hazards adversely impacts the effectiveness and reliability of critical controls, as human actions are not applied either to implement the Critical Control or act when the Critical Control deviates from the required specified standard [22]. The analysis identified multiple MAE incidents where an erosion in control integrity or changes in barrier functionality (e.g., exclusion barriers, maintenance of scaffold in use, proximity alarms) were tolerated by the work team and supervision. Where the risks become normalized through repetition or familiarity (e.g., continuously working around mobile plant, working on scaffolding), workers are desensitized to the risk exposure and become ‘complacent’ [19]. Under these circumstances, workers are less likely to respond to changing conditions, resulting in the type of ‘line of fire’ incidents observed in the study. This has implications in the design, implementation and operational integrity of a CCRM program where the reliability of the CC can be eroded.

Failures were identified in the competency of crane operators and riggers, application of work permits to isolate stored energy, spotters failing to maintain exclusions zones around plant and equipment or ineffective communication with mobile plant operators (Table 3). The incident investigations readily identified competency, (i.e., inexperienced or untrained workers) as a factor when CCs were not implemented. Competency, as a factor in CCs that were not applied to the standard required, is more complex. Worker competency is linked to their ability to either adapt the standards to the work or decide to stop work and seek clarification from supervision and management [38,39]. In both options, the CC system must provide direction on how to manage deviations [38], as major-incident investigation studies identified deviations from controls (rules/barriers) that are inevitable in high-risk industries, including construction [12,39,40,41]. One option to improve competency and consistent application of controls (rules) was to improve the specificity of the control and detail the control tolerance limits [13].

CC reliability was attributed to an individual’s decision making, which resulted in aberration from accepted safety standards (e.g., not fit for work, not applying danger lock and tag), substandard actions (e.g., inadequate inspections) or errors and lapses (e.g., wrong system isolated) (Table 3). Individual risk-based decision making in the application of CCs (rules and/or barriers) is influenced by a complex interface of personal, work team, organizational and psychological factors [13,19,38]. Rules are perceived as ‘guidance’, with workers applying adaptive thinking to achieve work tasks and goals [42]. Further investigation into individual’s decision making and the impact on CC implementation and effectiveness would benefit construction organizations looking to improve CC reliability.

Maintaining risk awareness is an inherent duty of supervisors through job planning and risk reviews, which focus on the hazards inherent in the tasks being undertaken and how hazards will be controlled [43,44,45]. Both factors were identified as being inadequate and contributed to the events analyzed. Winge [23] identified immediate supervision as strongly connected to worker actions, with the effectiveness of supervision a direct factor of job planning and risk management. In the absence of effective supervision, workers are less likely to act to implement or maintain CCs.

A major impact on job planning is the reactive nature of construction due to delays in the provision of materials, plant, equipment or labor, which causes compression of the schedule [46]. The delays result in perceived production pressure to ‘get the job done’, meaning work teams and supervisors become focused on task completion and fail to recognize changes in the work environment or hazards [8,47] or continue to work in the absence of effective safety supervision [23]. Where production pressure adversely affects safety performance through compression of work schedules [46,48] or rework from poor quality of execution [49], this also impacts performance of control barriers that rely on human action [7].

By focusing on CCs, construction organizations become more resilient as risk assessment, integrated into all systems; the verification process identifies and eliminates problems before they occur [48]. The study used historical incident data where the risk maturity of the participant organizations was reactive or, at best, risk compliant [48,50]. As organizations further develop and improve CCRM, the verification audits provide additional data to model safety performance. This shifts management focus from incidents (lagging measure) to proactive risk management and provides opportunities for predicting risks.

4.3. Limitations

The calculated control reliability level is biased and over represents the failure rate, as the assessment was conducted on incident events with known control failures and does not represent every time a Critical Control was challenged when executing work. The Critical Controls assessed did not cover all construction high-risk activities and were limited to four hazard categories. Equally, the study did not assess various cultural factors (e.g., language, religion, societal structures) and commercial and delivery strategies (e.g., self-perform, subcontractor, joint ventures), which potentially impact control of construction project fatal hazards.

5. Conclusions

The study confirmed that the controls identified for the four MAE hazard categories (Lifting Operations, Mobile Plant and Equipment, Stored Energy, Working at Heights) were valid as CCs through the control of energies associated with high-risk construction activities. Implementing and maintaining a CCRM is a significant investment in time, resources and cost, all of which are significantly constrained in the construction environment [48]. Senior managers want assurance that the investment in CCRM delivers safety improvements, which, in the absence of incidents, is difficult to quantify. Construction organizations participating in the research questioned the validity of CCRM to prevent potentially fatal accidents, specifically how does the organization know effort is invested in the ‘right’ CCs? The study was able to validate CCs for the four MAE hazards tested and identified gaps in CC standards within the safety management system (s), which the organizations were able to act upon. The methodology of CCRM incident analysis provides a basis to improve incident investigation root cause analysis by comparing incident root causes to CCs generating focused improvement actions. The study did highlight a need for further research how to measure the impact CCRM has in preventing serious incidents within a construction project.

The study provided insight into the individual and organizational factors, which potentially impact the reliability of CCs. Human performance factors, including hazard identification, personal decision making and competency, were common findings in the investigation reports analyzed. Worker competency was attributed to inexperience or lack of training or the lack of competency to assess, adapt and apply CCs to the work activity being conducted.

In complex construction environments, individuals need to be adaptive in the application of the CC to the situation, not just follow a black and white ‘rule’. It is the competency to apply CCs to the work environment that individuals need to develop, which informs their decision to stop work when the ‘rule’ is found not to apply to the situation. In the absence of an organization providing clear direction regarding CC deviations, failures will occur as workers influenced by their own risk perceptions will decide on how and whether to apply the CC and to what standard. The human performance factors can be addressed by the organization improving worker competency to assess and apply CCs across all high-risk tasks and, critically, the actions and/or behavior of competent supervisors to verify CC implementation and effectiveness for the given task being undertaken.

Organizational factors also contributed to the reliability of CCs. Supervisors having reacted to changes in construction schedule, materials and labor resourcing failed to undertake the CC activities, including job planning, risk assessments or communicating the risks and CCs to the work team.

The study benefits construction organizations applying CCs as a risk management tool as the results confirm the applicability of CCs for the MAE hazards analyzed and highlight the factors that need to be considered when implementing a CC program. Organizational processes need to ensure supervision and workers are trained and competent in the application of CCs, direction is provided to manage deviations and management oversight to ensure implementation and quality is maintained. The method presented and the use of the bowties to illustrate the results represents a novel contribution to the literature on controlling fatal risks on construction sites. Future work to continue the contribution to research is planned to extend the analysis to additional risks and additional construction projects.