Next Article in Journal
Reforming Health, Safety, and Environmental Regulation for Offshore Operations in China: Risk and Resilience Approaches?
Previous Article in Journal
Reviewing the Poyang Lake Hydraulic Project Based on Humans’ Changing Cognition of Water Conservancy Projects
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

A Systems Analysis Approach to Identifying Critical Success Factors in Drinking Water Source Protection Programs

1
Graduate School of Engineering Science and Technology, National Yunlin University of Science and Technology, Yunlin 64002, Taiwan
2
Department of Civil and Construction Engineering, Graduate School of Engineering Science and Technology, National Yunlin University of Science and Technology, Yunlin 64002, Taiwan
3
Graduate School of Environmental, Health and Safety Engineering, National Yunlin University of Science and Technology, Yunlin 64002, Taiwan
*
Author to whom correspondence should be addressed.
Sustainability 2019, 11(9), 2606; https://doi.org/10.3390/su11092606
Submission received: 1 April 2019 / Revised: 27 April 2019 / Accepted: 3 May 2019 / Published: 6 May 2019

Abstract

:
The success of source protection in ensuring safe drinking water is centered around being able to understand the hazards present in the catchment then plan and implement control measures to manage water quality risk to levels which can be controlled through downstream barriers. The programs in place to manage source protection are complex sociotechnical systems involving policy, standards, regulators, technology, human factors and so on. This study uses System Theoretic Process Analysis (STPA) to analyze the operational hazards of a typical drinking water source protection (DWSP) program and identify countermeasures to ensure safe operations. To validate the STPA results a questionnaire was developed based on selective grouping of the initial countermeasures identified and distributed to specialists in DWSP in Taiwan, Australia and Greece. Through statistical analysis using Principle Components Analysis (PCA), the study identified four critical success factors (CSFs) for DWSP based on the questionnaire responses. The four CSFs identified were “Policy and Government Agency Support of Source Protection”, “Catchment Risk Monitoring and Information”, “Support of Operational Field Activities” and “Response to Water Quality Threats”. The results of this study provide insight into the approach of grouping of source protection measures to identify a series of targeted CSF for operational source protection programs. Using CSF can aid catchment management agencies in ensuring that the risk level in the catchment is managed effectively and that threats to public health from drinking water are managed appropriately.

1. Introduction

1.1. Role of Source Protection in Safe Drinking Water Supply

The multi-barrier approach to drinking water safety, which includes source water protection, is recognized internationally as best practice and is advocated for in many national drinking water frameworks and relevant legislation. In most drinking water schemes, catchment management and source protection are the first barriers in the supply of safe drinking water [1,2]. A well-developed and implemented source protection program can not only help ensure safe drinking water, but effective source protection can also be an economical way of offsetting water treatment costs through capitalizing on ecosystem services to control water quality [3].
For drinking water catchments, two of the most common threats considered in source protection are the potential public health impacts from contaminated water and the potential to compromise downstream processes from water quality parameters that inhibits the ability of the process to operate as required for safe drinking water production. Protection of drinking water catchments provides key services in the supply of safe drinking water through reducing the number of pathogens and organic matter entering downstream treatment facilities. From a system-wide perspective, controlling pathogens reduces acute enteric risks, and controlling organic matter reduces unwanted disinfection by-products (DBPs) formed during treatment and disinfection [4]. The health concerns surrounding DBPs in drinking water supplies is growing worldwide. A good example is chlorite produced from the use of chlorine dioxide for disinfection of waters with high dissolved organic carbon content. In the European Union the compliance with chlorite has proved challenging for many member countries due to the continued updating of the European directive for drinking water in response to a growing body of evidence on the potential health risks. In some cases where disinfection and treatment has been improved chlorite remains an issue due to the poor quality of surface water sources [5].
The success of source protection in ensuring safe drinking water is centered around being able to understand the hazards present in the catchment then plan and implement control measures to manage water quality risk to levels which can be controlled through downstream barriers. Investigations of contributory factors of public disease outbreaks due to drinking water contamination show that common factors related to source protection include inadequate knowledge of source water hazards, fecal contamination from live stock or wildlife, as well as extreme weather [6]. In most cases drinking water contamination incidents experienced can be attributed to multiple causes [7]. Drinking water catchments are complex systems, subject to continual dynamic changes resulting from environmental fluctuations over time, impacts of interactions of multiple agencies and stakeholders in the catchment area as well the actions of drinking water management agencies. The nature of complex systems is that they are inherently hazardous and require defenses to guard against failure [8]. When these guards in complex system such as source protection programs and drinking water, systems fail there is potential for significant loss through public illness and potential death.
A well-known example of the result of waterborne disease outbreak is the Walkerton, Canada tragedy in 2000. As the result of drinking water being contaminated with Escherichia coli 0157:H7 and Campylobacter jejuni, 2300 people experienced gastroenteritis, 65 were hospitalized and 7 died. Detailed investigations of the incident discovered the cause was a multiplicity of failures consistent with other outbreaks experienced in the developed world that went unresolved [9]. In the case of the Walkerton investigations, while a lot of attention was focused on the role of the water service providers responsible for operating the water supply, there were several other latent systemic failures in the drinking water supply system. From a source protection perspective there were some key failings; a good example is the government policy and regulation at the time. The government policy and regulation controls in place failed to prevent farming practices from contaminating bores which were known to be vulnerable to contamination from surface activities [10].
For safe drinking water supply the World Health Organization (WHO) advocates a catchment to consumer approach to risk management that includes all steps in the drinking water supply process [11]. Use of risk-based approaches in the supply of drinking water systems is well established in the water industry. Risk-based approaches enable the management focus to be on the greatest threats to drinking water quality and public health. For source protection there is limited literature and research on the management of the complex operational risk in the implementation of source protection programs.

1.2. Source Protection Programs

Understanding the risks present in a drinking-water supply requires a detailed and documented hazard assessment, that considers factors such as system performance, controls in place, assumptions made, data used, etc. Hazard assessment outcomes are typically summarized in documents such as water safety plans, catchment management strategies and other operational management plans. Such operational documentation should provide a detailed description of the operations undertaken and the procedures that guide them to ensure safe drinking water [1].
When assessing the risks and planning operations to protect drinking water relying on water quality sampling alone provides a limited picture of the activities and hazards in the catchment. From the numerous potential drinking water contaminants, pathogens present the greatest risks to public health. Infections can result from minute doses, also there is potential for rapid concentration variances within a short period of time. Furthermore, the public can be exposed to waterborne pathogens before any testing results are received. Therefore, frequent sampling of supplied drinking water alone is not reliable for confirming the safety of a drinking water supply [11]. From a chemical safety perspective, detecting of chemical contamination in the final water supply often indicates that significant contamination has taken place. Identifying potential chemical hazards in the catchment area and controlling them before impact occurs reduces the overall chemical risk to drinking water quality.
The preventative approach to controlling drinking water risk has been in practice in the water industry for many years. To ensure drinking water safety, management focus should be placed in the performance of the entire system including protection of water sources, appropriate treatment, disinfection, and distribution management. Developing an effective source protection strategy is contingent upon having a good understanding of the characteristics of the catchment and the downstream water quality barrier characteristics. In drinking water systems, the term hazards is typically used to refer to biological, chemical, physical or radiological agents with the potential to result in public illness or result in water quality which is unacceptable for consumers [12]. When developing a whole-of-system water safety planning approach, the process of hazard identification should extend beyond the direct inputs driving the microbial and chemical parameters and examine the threats to the proper functioning of the system components [11]. For this study the focus is on the hazards to the proper functioning of the operational program to implement source protection measures for the protection of public health.
The ecosystem services and water quality management infrastructure that influence the drinking water quality outcomes in catchment areas require continuous monitoring and verification of performance against required levels of service to ensure that water quality objectives are met [13]. Whereas conventional water treatment process has a certainty of outcome and can be assessed readily with conventional-based hazard analysis techniques, when it comes to source protection and catchment management conventional hazard analysis techniques have limited applicability. Numerous methods are frequently employed to evaluate the hazards involved in the operation of complex systems. Many of these component failure-based methods such as Hazard and Operability Analysis, Failure Modes Effect Analysis, Bowtie Analysis and Fault Tree Analysis are used to identify hazards in the assessment of drinking water systems [14]. Managing catchments as a water treatment asset poses a set of unique challenges compared to conventional water supply infrastructure. Conventional hazard analysis methods focus on reducing the components of larger system to an “assembly” of individual components. This use of analytical reduction and potentially overlook hazards resulting from the component interactions [15]. Rasmussen [16] proposed that risk management should be approached as a control problem, which requires a system orientated approach using functional abstraction. The complex sociotechnical structures of source protection systems rely heavily on safe interaction of system components to ensure overall system safety. A key example is the complexity resulting from multiple ownership of land and multiple land managers both private land owners and government agencies. The resulting challenge with managing drinking water catchments as a water treatment asset is that often a large portion of the catchment area is not under water utility ownership [17]. Furthermore, many of the risk control actions are undertaken by organizations not under the direct control of the water service provider. The effective control of activities in catchment areas requires targeted multifaceted programs involving the various agencies which engage landholders and other catchment users in achieving a common outcome. Often there is lack of clarity and confusion around the roles of water utilities and different regulatory agencies in drinking water catchments [18]. The complexity of source protection operations requires hazard analysis methods which can understand both the component interactions as well as the implications of the complex organizational arrangements between agencies involved in natural resource management in catchments.

1.3. Need for Further Research

Internationally, within the water industry, the role of catchment management and source protection is receiving growing attention. For example, in Australia the intended future addition of the Health Based Targets (HBT) to the Australian Drinking Water Guidelines (ADWG) is planned to provide greater clarity around various catchment pathogen-based risk factors which drive the vulnerability level and the required downstream barriers [19]. Following this advice, catchment management agencies can influence their catchment risk profiles to better align with the downstream drinking water barrier requirements. To capitalize on these advancements for the provision of safe drinking water, source protection processes in the water industry need to adapt and evolve to meet the growing sophistication of drinking water standards such as the HBT requirements set out the ADWG. Such an evolution requires new innovative ways to analyze the operational risks of source protection programs to ensure the catchment risk control measures are performing as expected.
The operational risk for drinking water supplies is typically centered around the reliability of system components such as the failure of a specific treatment step [20]. The belief that reliability is equal to safety is common throughout the engineering field [21]. In the case of drinking water source protection (DWSP), the assumption would be if that the equipment or processes used in the source protection process perform constantly to a defined standard without deviation, then the water produced by the catchment should be deemed safe. However, it is possible to have a high-reliability system that does not enforce safety of the overall system. Investigations into incidents involving complex systems have shown many accidents occur without the failure of any single component, instead, the failure has come from the way in which the system components have interreacted [21].
When assessing risk in drinking water supplies frequently broad assumptions are made on the effectiveness of control measures to protect drinking water quality. For the management of catchment risk, the risk tools used play a big role in development of the systems and processes to control ensuring drinking water safety [20]. A common standard industry approach is to use risk matrices which use an analysis of inherent risk without controls applied and residual risk after a control has been implemented [14], semiqualitative risk assessment using a risk matrix supports the analyst’s role in providing a risk assessment where there is no reliable quantified probabilistic data on control performance or reliability. Instead the analyst uses the opinions of experts and staff with intimate knowledge of the process of interest [22]. The risk analysis process typically combines the worst possible consequence and associated likelihood of the hazard occurring to define the level of risk, and in most cases the water supplier usually defines the consequence and likelihood. In this risk assessment process, the analyst is required to make broad assumptions regarding the performance and effectiveness of the control measures and the effectiveness of processes that support them [23]. The resulting high level of uncertainty arising from gaps in knowledge must be recognized during the risk assessment process [24].
In source protection, risk control actions are diverse and can include livestock management, land management controls, statutory land use planning, regulatory enforcement actions, access barriers and government policy. Not only are there a vast number of control actions, there are almost as many agencies responsible for the control actions as there are control actions. The challenge then becomes how to ensure the control actions not only reliably perform as expected but correctly enforce safety constraints to meet the assumptions made. Therefore, the issue of operational safety in source protection extends beyond component reliability and takes on board component interactions, multiple organizations, government policies, etc. In more recent times the growing sophistication of technologies such as those used in automation and remote monitoring has introduced new failure modes and operational hazards. There is limited literature exploring the key factors of these programs which can influence successful implementation of risk-based DWSP programs.
The accident model Systems Theoretic Accident Modelling Process (STAMP) was introduced in response to observing that many conventional hazard analysis methods are limited in their ability to work effectively for contemporary sociotechnical systems [25]. Being based on systems theory, the STAMP model views safety as a control problem, in that the feedback and control actions in the system determine the overall system safety. In this study, the STAMP-based System Theoretic Process Analysis (STPA) is used a tool to identify hazards in a typical source protection program. Integrating STPA into the hazard analysis process has the potential to overcome the heuristic biases which can contribute to flawed risk assessments [26]. The complexity of analyzing catchment performance and overall drinking water system risk is significant. This complexity is well suited to the use of STMAP-based STPA to better comprehend how system constraints and hazards impacts on the overall ability of source protection programs to deliver safe drinking water. STAMP-based incident analysis methods have been used to analyze the intricate systemic catchment to tap causal factors of the 2000 Walkerton drinking water contamination event [10].
Much of the guidance literature on the requirements for effective source protection programs is based on the judgement and professional experience of the authors and selected groups of peers. The approach of this study is to use hazard analysis focusing on water quality and public health risk management in drinking water catchments to identify potential hazard control strategies. For this study, STPA is used to complete a comprehensive process hazard analysis of a source protection program for a typical surface water drinking to develop a set of comprehensive of system requirements for effective DWSP programs based on current industry best practice. The requirements are based on countermeasures for controlling hazards identified in typical DWSP processes. In this study, effective source protection is the prevention or reduction of the potential for public illness due to contamination resulting from catchment contamination.
The purpose of this study is to investigate the critical success factors (CSFs) for DWSP programs based on the perceptions of catchment management professionals and specialists. This is achieved through first classifying and aggregating a large number of countermeasures for controlling hazards in DWSP program safety identified through STPA. The countermeasures are then organized into groups based on common functions. Then, a questionnaire was developed to capture industry professionals’ perceptions on the importance of the countermeasure groups identified. Using the questionnaire results the final CSFs for successful DWSP programs are obtained using Principle Components Analysis (PCA). In this type of application PCA aids the identification of components and relationships which cannot be observed solely from the collected measures or indicators [27]. Moreover, PCA can help reduce the overall number of initial factors selected without losing any key information [28]. The principle components found through the PCA results of the surveys completed by catchment management specialists and professionals in Taiwan, Australia, and Greece provides an insight on the perceptions of the CSFs that influence the successful day to day operational management of DWSP programs.

2. Materials and Methods

This study examines the process risks in the delivery of source protection programs for theoretical example of a typical surface water drinking water catchments using process hazard analysis. Process hazard analysis is common in many high-risk industries and is a fundamental step in risk assessment of any technical system and its processes [29]. Many of the conventional methods focus on equipment failure and fail to compressively identify hazards in complex sociotechnical systems [25], like those typically involved in source protection programs. This study uses STPA to take a holistic look at the process hazards involved in DWSP programs. The results from the STPA process are then categorized and refined into final CSFs based on PCA of expert opinions. The detailed description of the two methods used are presented in the next two subsections.

2.1. STPA Methodology

Typical DWSP programs involve complex sociotechnical systems, which can include technology, multiple agencies, government policy, private land holders, etc. Hazards can arise from the control actions of the various actors in the safety system as well as the interaction of the various components in the system. STAMP-based models are well suited to understanding safety systems involving a network of stakeholders such as government authorities and their corresponding influence over the process operations through regulation and policy [30]. Before commencing STPA-based analysis, there is no need for a completed design of the safety process, allowing the development of the safety system to be based on the STPA outputs [21]. Thus, STPA can be utilized in the formation of the safety system design and support refinement as the system evolves and changes, enabling enhancements of safety requirements of the safety system design [31]. The flexibility of the STPA methodology can support the analysis of the theoretical source protection program presented in this study.
The STPA methodology takes a top down view of the dynamic interaction of the various components of the system through a series of control loops, therefore the scope of the assessment can extend to identifying the hazards associated with catchment management process, as well as the wider system safety controllers such a government administration of regulations and links with other government agencies. The safety control structure provides a hierarchical overview of the source protection program, which if effective will enforce the safety constraints of the overall system. The control structure functions as a representation of the system model consisting of an assembly of control loops [31]. The generic format of the STPA control loop is shown in Figure 1. Each feedback loop consists of a controller responsible for initiating the control action, control actions, the process being controlled, and the feedback collected by the controller. Each control action initiated by the controller is based on model of the state of the controlled process to govern the control action required [17]. Within the safety structure of the source protection program controller include, organizations such as government, water service providers, technology and people.
The system hazards are derived through assessing how the control actions in the safety system can lead to scenarios which violate the stated safety constraints of the system. The unsafe control actions (UCAs) are the result of scenarios where the control actions can potentially violate safety constraints. The following four prompts are used to guide the process of identifying scenarios which can result in UCAs: control action not provided, control action provided, control action provided too early or too late or in the wrong sequence, and control action provided too long or too short. In typical practice, the STPA methodology consists of four key steps [31].
  • Define the purpose of analysis—defining the process is used to identify unacceptable system losses, system-level hazards and corresponding safety constraints.
  • Model of the control structure—a safety system model comprised of a series of connected feedback and control loops.
  • Identify unsafe control actions (UCAs)—the control actions which in a worst-case scenario will result in a hazard.
  • Identify loss scenarios—these are the scenarios that result from the combination of causal factors that lead to UCAs and potential loss(es).
In conjunction with the identification of loss scenarios in step two of the STPA process, the causal factors (CFs) and related scenarios leading to a UCA were considered. In this process, CFs are the primary factors which may result in the control actions becoming UCAs [31]. Following the identification CF for the UCAs, to provide information on how to reduce risk associated with UCAs, the next step is to identify suitable countermeasures for each CF. The countermeasures are actions required to either prevent the causal scenario from occurring or to reduce the impacts of the relevant CF for the scenarios considered [32]. It is these countermeasures that are included in the program requirements to reduce the risk of unsafe or out of spec water being produced by the catchment. The STPA was completed using STAMP Workbench V1.0.0 (Information-Technology Promotion Agency, Tokyo Japan).

2.2. Identification of Critical Success Factors

Measuring the perceptions of importance for each of the countermeasures identified from the STPA process is impractical due to the large number of countermeasures generated by the process. For large systems with significant amounts of information, a common approach is the use of CSFs. CSFs are key items that must go well to ensure safe management of a process. When carefully selected, CSFs are proven to be highly effective in supporting planning and requirements analysis [33]. In this study, an initial set of system requirement groups were created from the STPA results and countermeasures identified, by reviewing each countermeasure and grouping them according to common features. These system requirements then become the initial factors from which CSFs can be derived. The process for generating CSFs was to first find the key functional groups of the countermeasures generated, then to validate the countermeasure grouping using a survey of catchment management professionals and dimensional reduction through PCA.
To validate the summarized findings of the STPA results, this study uses a targeted survey constructed from the countermeasures groups identified to measure the perception of the importance of common source protection operational measures. The targets of the survey questionnaire were industry specialists and professionals in Taiwan, Australia and Greece involved with DWSP programs. The survey was open to different countries to ensure the results received represented were not biased towards the practices of any specific country. There is significant diversity in how the target countries manage the supply of drinking water. In Taiwan the Water Resources Agency is the government agency responsible for the management of drinking water catchments. The potable water treatment and distribution services are provided by the Taiwan Water Corporation. Drinking water catchment management arrangements in Australia can vary greatly. Depending on state and location is catchment management is the role of state-owned water service providers, state-owned catchment management agencies, and local governments. In some situations, the treatment and supply of potable water can be from the same agency managing the drinking water catchment; in other circumstances, the treatment and distribution may involve one or more additional agencies. In Greece there are two state-owned water utilities responsible for the supply and sanitation of drinking water for Athens and Thessaloniki. Outside of the major cities drinking water supply and sanitation are the responsibility of individual municipal utilities.
The questionnaire was structured based on a 5-point Likert-type scale for a series of statements where the respondent provides a response ranging from 1 (strongly disagree) to 5 (strongly agree). Likert-type questionnaires are frequently used in the collection of data on personal opinions or perceptions using questionnaires. The final CSFs were then established by using PCA to group the initial countermeasure factors based on the questionnaire results. PCA uses a defined methodology to identify groups of related variables and through taking many variables are grouping them together to create a smaller set of variables that are relatively independent of each other [34]. This supports PCA as an ideal technique for creating a more easily understood construct for management frameworks [35]. Through reducing several highly correlated variables to a smaller number of principle components which account for the majority of variance in the observed results, PCA can provide a set of CSFs for effective source protection program outcomes. The use of PCA is intended to reduce the variables contained in the questionnaire to a select number of focused CSF, as well as to provide insight into how the various factors relate to each other. The key steps involved in PCA are verification of data suitability, construction of a correlation matrix, generation of principle components, and interpretation of principle components produced. For this study, SPSS V18 by IBM (International Business Machines Corporation, Armonk, NY, USA) was used for the processing of questionnaire results and identifying the principle components.

3. Results

3.1. STPA Results

3.1.1. Purpose of Analysis

The principle accident or unacceptable loss considered is public illness and death from unsafe water produced by the catchment. The measure of this accident varies internationally depending on the relevant standards adopted; based on current guidance in [36], an accident would be drinking water quality which results in a community disease burden above the acceptable levels for pathogens. Also considered is the impact of chemical contaminants in drinking water on public health. Based on the accident selected, there are two main types of system-level hazards considered in this study. The first system-level hazard is the potential for regulatory requirements failing to protect public health and the second system-level hazard is the activities or events in the catchment resulting in contamination of catchment runoff.
Following the identification of the hazards relating to the STPA, the safety constraints are the requirements included in each relevant control action or can be requirements for the system to reduce losses in the event of a hazard occurring [31]. Selecting the constraints for a source protection program centered around the protection of public health through the prevention of runoff contamination. Table 1 lists all accidents, hazards and safety constraints identified for the analysis of source protection programs.

3.1.2. Source Protection Safety Control Structure

The safety control structure provides a hierarchical overview of the source protection program, which if effective will enforce the safety constraints of the overall system. The control structure functions as a representation of the system model consisting of an assembly of control loops [31]. The generic format of the control loop is shown in Figure 1. Each feedback loop consists of a controller responsible for initiating the control actions, for the process being controlled, and the feedback collected by the controller. Each control action initiated by the controller is based on a model of the state of the controlled process to govern the control action required [26]. The safety structure of the source protection program includes organizations such as government, water service providers, technology and people. The full list of the actors, respective roles and control actions in the safety system is provided in Table 2. The final hierarchical safety control structure for a typical source protection program for a surface water source is provided in Figure 2.

3.1.3. Unsafe Control Actions

The high-level safety control structure identified 46 high-level control actions related to the system-level hazards and safety constraints related to source protection programs. Following the identification of the control actions, the next step was to assess the potential scenarios in which the control actions could be unsafe. The four ways a control action can be potentially unsafe are; not providing, providing causes a hazard, provided to early or late and the control action lasts too long or is ceased too early. All four scenarios for UCAs will not necessarily be applicable for every control action and some scenarios may result in multiple UCAs. For the 46 source protection control actions there were a total of 155 UCAs identified. The coverage of the UCAs identified covers control actions of all the actors in the safety system as well as the actions of the equipment and systems which support source protection functions. A sample of the UCAs identified is presented in Table 3 to illustrate the results produced.

3.1.4. Loss Scenarios and Countermeasures

The loss scenarios are the set of causal factors which can result in UCAs and then the realization of system hazards [31]. Each of the 155 UCAs were reviewed to identify all credible causal factors based on typical operations and functions of source protection operations. The process yielded 317 individual CFs based on the loss scenarios considered. For the formal STPA method, the identification of causal factors is normally the last step, but, in this study, countermeasures were considered for each CF. The process of identifying countermeasures consists of reviewing each causal factor and identifying requirements to either prevent the CFs from occurring or reduce the potential for causal factors to result in a UCA [32]. The initial investigation of the CFs showed that more than one viable countermeasure can be applied to many CFs. A complete review of the CFs identified resulted in the identification of 222 countermeasures to reduce the impact of or eliminate the chance for system hazard to occur. The selection of countermeasures was based on typical good practice for catchment management and source protection. The guidance on good practice was largely based on [37], and other freely available literature focused on identifying, assessing and managing drinking water risks in surface-water catchments. A sample of selected causal factors and corresponding countermeasures for the UCAs identified is presented in Table 4.

3.2. Critical Success Factors

The STPA process is focused on identifying of system hazards; in the standard STPA procedure there is no step which provides an evaluation of importance. The STPA results can provide a multitude of countermeasures for ensuring safe operational control of DWSP programs with no practical or efficient way to evaluate each of the UCAs or countermeasures identified. Therefore, in this study, the validation of countermeasures is determined based on the perceptions of source protection professionals. Measuring the perceptions of importance for each of the countermeasures is impractical, therefore the countermeasures needed to be reduced to several smaller groups based on common traits and functions.
For large systems with significant amounts of information a common approach is the use of CSFs. CSFs are key items that must go well to ensure success in managing a process. When carefully selected, CSFs are proven to be highly effective in supporting planning and requirements analysis [33]. Developing an initial set of CSFs based on the countermeasures identified involved a review of each countermeasure and grouping it according to similarities between the countermeasures. The authors’ review of the countermeasures revealed they can be reliably grouped based on the key theme and function of the countermeasure. The countermeasure groups for each of the key functions described is shown in Table 5.

3.3. Questionnaire Development

For this study the questionnaire comprised of 20 statements taken from the countermeasure groups 1–20 in Table 5. All the items in these questions related to how an agency would plan, implement, and evaluate source protection programs. Items from 21–26 though representing important results from STPA are more related to the internal business processes within an organization used for supporting source protection programs and were excluded from the scope of this study.
The final statements included in the questionnaire are as follows: 1) Current government policies provide robust protection of drinking water catchments, 2) Government agencies are active in implementing policy for the protection of drinking water quality catchments, 3) Government provides timely policy responses to emerging threats to drinking water catchments, 4) The Government engages with relevant stakeholders when developing or reviewing policy for the protection of drinking water catchments, 5) Natural Resource Management agencies have a good understanding of drinking water catchment management principles, 6) Natural Resource Management agencies actively support the protection of drinking water catchments, 7) Natural Resource Management agencies use their authority effectively to manage activities within in drinking water catchments, 8) Natural Resource Management agencies effectively manage risk of natural disasters (i.e., Fire, flood, erosion/landslides), 9) Natural Resource Management agencies see drinking water management agencies as key stakeholders, 10) Effective catchment management planning requires current land use information, 11) Observational information on catchment condition is critical to catchment management, 12) Water quality data is critical to catchment management, 13) Typical catchment management practices are capable of identifying and managing threats to drinking water quality, 14) Regular monitoring of activities in catchment areas is essential to managing risk to drinking water quality, 15) Real time information on catchment activities is required for effective control of risk to drinking water quality, 16) Enforcement of legal controls over activities in drinking water catchments are effective in reducing drinking water quality risk, 17) Data collected through inspections/surveillance is valuable in planning field operations to reduce risk to drinking water quality, 18) Water quality data is essential in planning field operations to reduce risk to drinking water quality, 19) Engagement with public in catchment areas is essential in reducing risk in drinking water catchments, and 20) Continuous training of field staff is critical to effective catchment operations for the protection of drinking water.
Prior to full distribution, a copy of the survey questionnaire was provided to four selected source protection professionals to verify that the questions were suitable for collecting representative data on source protection practices. The experts involved in the review included three Australian and two Taiwanese specialists with extensive experience in the water industry as well as having backgrounds in source protection practice, operations and research. To validate the final responses provided the questionnaires included the following screening questions, category of current role, current organization, and years of experience in source protection.

3.3.1. Questionnaire Respondents

Both the electronic and paper-based copies of the surveys were sent to professionals and specialists in Taiwan, Australia, and Greece. The target respondents were people working in drinking water catchment management related roles from industry, consulting, academia. Due to the specific skill set of the target respondents and the number of DWSP professionals the target pool was restricted. There was a total of 63 completed questionnaires returned; however, two paper responses were discarded due to missing data, resulting in a total of 61 responses. Once the questionnaires had been returned the results were then analyzed.
The distribution of responses for the three countries included 40, 13 and 8 questionnaires completed from Taiwan, Australia, and Greece, respectively. The current roles and years of experience in source protection of catchment management reported at the time are presented in Table 6 and Table 7, respectively. The survey shows that 44% of respondents reported of having 5 years or less experience. The remaining 56% of respondents reported having 5 or more years’ experience. Most respondents (51%) reported having between 5- and 20-years’ experience in catchment management or source protection.

3.3.2. Test of Reliability and Validity

In terms of the survey responses, “Reliability” refers to the internal consistency and stability of the questionnaire results and is most frequently expressed in terms of Cronbach’s α [38], with a value exceeding 0.7 indicating a high degree of reliability [39] and being deemed satisfactory [40,41]. The Cronbach’s α for the responses of the responses from the three countries combined is 0.92, showing that the internal consistency of the 20 questions is significant enough to proceed with the analysis.
With responses representing three different countries, multiple agencies, roles within respective organization, and different levels of experience there is potential for differences in the responses which could influence the results presented. To test for the potential difference in answers between the three countries a comparison of the means using ANOVA was completed for the responses to each of the 20 questions. Of the responses there was only one significant difference at the p = <0.01 significance level, with item 8 being for role. Item 8 relates to Natural Resource Management agencies effective management of risk from natural disasters, given the diversity of roles represented, some difference is not unexpected. Given the difference is only with a single item and characteristic it was considered prudent to continue with analyzing the data as a single group. The ANOVA F-test results for each of the 20 questions are presented in Table 8.

3.3.3. Questionnaire Responses

The mean responses for each of the questions are shown in Table 9 with relative rankings for the organization types. In general, the mean values are quite high, indicating the measures selected are considered important to the overall success of source protection programs. The factor rankings vary depending on the various agency reported but tend to show similar patterns. This difference is possibly due to variances in perceptions influenced by the organization of the respondent; for example, respondents from government agencies ranked question 2 “Government agencies are active in implementing policy for the protection of drinking water quality catchments” notably higher than respondents from other organizations. This result is possible due to the bias resulting from being close to the inner workings of the respective organization. Overall questions relating to catchment management planning tended to rank highest across most organizations and for all organizational types combined. Catchment management planning is typically directly under the control of agencies tasked with source protection, resulting in greater importance placed on the tasks associated with management planning. Whereas the policy related items tended to have comparatively lower means and lower ranks in the 20 factors identified. For catchment management agencies the influence over policy is usually as a stakeholder in the development and implementation processes. These processes can be protracted and take years to complete. Therefore, there is less direct control over the outcomes and the benefits gained can take a long time to be realized.
The factors are as follows: 1) Current government policies provide robust protection of drinking water catchments, 2) Government agencies are active in implementing policy for the protection of drinking water quality catchments, 3) Government provides timely policy responses to emerging threats to drinking water catchments, 4) The Government engages with relevant stakeholders when developing or reviewing policy for the protection of drinking water catchments, 5) Natural Resource Management agencies have a good understanding of drinking water catchment management principles, 6) Natural Resource Management agencies actively support the protection of drinking water catchments, 7) Natural Resource Management agencies use their authority effectively to manage activities within in drinking water catchments, 8) Natural Resource Management agencies effectively manage risk of natural disasters (i.e., Fire, flood, erosion/landslides), 9) Natural Resource Management agencies see drinking water management agencies as key stakeholders, 10) Effective catchment management planning requires current land use information, 11) observational information on catchment condition is critical to catchment management, 12) Water quality data is critical to catchment management, 13) typical catchment management practices are capable of identifying and managing threats to drinking water quality, 14) Regular monitoring of activities in catchment areas is essential to managing risk to drinking water quality, 15) Real time information on catchment activities is required for effective control of risk to drinking water quality, 16) Enforcement of legal controls over activities in drinking water catchments are effective in reducing drinking water quality risk, 17) Data collected through inspections/surveillance is valuable in planning field operations to reduce risk to drinking water quality, 18) Water quality data is essential in planning field operations to reduce risk to drinking water quality, 19) Engagement with public in catchment areas is essential in reducing risk in drinking water catchments, and 20) Continuous training of field staff is critical to effective catchment operations for the protection of drinking water.

3.4. Principle Components Analysis

3.4.1. Data Suitability

The total valid responses (n) received was 61 for the 20 questions (p) results in a factor to respondent ratio of 3:1. The literature on the minimum sample size required for PCA varies greatly and therefore there is no a straightforward guideline on the minimum sample size required for PCA [40]. The minimum sample size required varies depending on the quality and type of the data. Though the sample size was quite low, communalities for the 20 factors were all above 0.6 with exception of factors 4 and 15 which were 0.49 and 0.57, respectively. The relatively high levels of communality for the factors indicates that the data was of suitable quality to continue with PCA.

3.4.2. Correlation Matrix

In this study, the sample size of 61 means that based on the Central Limit Theorem the sampling distribution would have a normal distribution [42]. To validate this assumption for the data set collected, the Shapiro-Wilk Test was used to check for normal distribution for each of the factors included in the survey. The Shapiro-Wilk Test is a robust test which can be applied to all types of distributions and sample sizes [43]. The results of the Shapiro-Wilk Test showed that the responses received for all the factors were normally distributed at the p = <0.000 level. Therefore, parametric tests are suitable for analysis of the survey data collected for the study [44]. The correlation matrix for PCA was developed through calculating the Pearson Correlation values (r) for each of the factors pair together to form a matrix. Most correlations in the matrix display significant values with many correlations over an r value of 0.4, indicating that several significant relationships exist between the factors. With the large number of strong correlations between factors, the data is suitable to continue with PCA. Interestingly, the strongest correlations are between 10 and 11(r = 0.88), 10 and 12(r = 0.82), and 11 and 12(r = 0.84). Given that these factors relate to the different data used in the planning process, such as observational data and water quality data, as well as land use information, there is likely to be a strong relationship between them. Water quality data has long been the key focus in determining catchment risk and identifying potential risks to public health and planning control measures. However, the formal use of observational data to provide information on potential threats to drinking water quality in catchment is comparatively recent.
Before progressing to the extraction of the factors, an assessment of the data for suitability for PCA was made. The tests included Bartlett’s Test of Sphericity to verify that the correlation matrix does not represent an identify matrix, as well as the KMO Measure of Sampling Adequacy to ensure the matrix is suitable for PCA. The KMO index has a potential range from 0 to 1, where a result greater than 0.50 indicating the data is suitable for PCA. This test is important where the factor-to-respondent ratio is less than 5:1 [45]. Additionally, the Bartlett’s Test of Sphericity should be significant at p = 0.05 level. The KMO index value in this case is 0.83 indicating the matrix is highly suitable for PCA despite the low factor to respondent ratio. Furthermore, the Bartlett’s Test of Sphericity returned an approximate Chi-Square value of 912.6 with 190 degrees of freedom, resulting in a p-value of 0.000, further indicating the suitability for continuing with PCA.

3.4.3. Extraction of Principle Components

Eigenvalues indicate the amount of variance explained by each principal component [34]. The scree plot of eigenvalues and component values shown in Figure 3 shows that a large proportion of the variance is explained between component number one and component number four. At around component number 4 there is an inflection point where the scree plot flattens. The final factors were extracted using the Kaiser rule, where components with eigenvalues of 1 or above are retained [46]. The results contained four principle components with an eigenvalue above 1; therefore, a 4-factor model was selected.
Following the calculation of the initial components, varimax rotation was performed to help with the interpretation of the results. The varimax rotation process simplifies components through maximizing the variance of the loadings within components. The loadings which are high post extraction become higher after rotation, and loadings that are low become lower [45]. After rotation all the components in the rotated component matrix with a value of 0.55 or greater were retained. According to [47], a value of 0.55 can be considered good and indicates 30% overlapping variance. The resulting component groups, loadings and associated variances are shown in Table 10.

4. Discussion

With the PCA completed, the next step is to interpret the results and understand the implications for implementation of source protection programs. The PCA results are interpreted as a series of refined CSFs based upon the components which when combined account for 71.5% of the total variance observed.

CSF 1: Effective policy and government agency support of source protection

This group is comprised of factors 1 through 9, which capture government action on policy and the support of government NRM groups, together they account for around 40% of the total variance seen in all components. Successful source protection requires an established link between land use control and water quality outcomes. While there are recognized benefits from government and government agency support, the benefits take significant effort to cultivate and require ongoing effort and commitment from all involved to sustain. Policy and legislation can take years to develop and effectiveness is based on the commitment and the engagement and support of the various stakeholders in the catchment areas. The development of new environmental policy related to source protection is often incremental and done in response to a crisis [48].
Drinking water catchments are often multiuse and fall under the jurisdiction of multiple government agencies for the management of various natural resources or environmental protection roles. For water utilities there of often limited statutory power to enforce requirements for the protection of drinking water quality. Therefore, the actions natural resource management agencies operating in catchment areas have substantial influence over water quality produced by the catchment. Active stakeholder engagement between water service providers and natural resource management agencies is fundamental to ensuring safe drinking water. The approach taken should be that of a partnership which leverages off each agencies strength in protecting drinking water quality outcomes.

CSF 2: Catchment condition information and risk monitoring

This group is comprised of factors 10, 11, 12, and 14, which relate to information on catchment condition and water quality. The respondents to the survey ranked these items the highest overall, and together they account for almost 20% of the variance observed in the PCA. In this group, the items are related to the information required to effectively plan for the operations to manage risk in the catchment. Often these items are of high importance as they are key to the planning and are easily managed and controlled by water service providers and catchment management agencies. The three key sources of risk information included are conventional water sampling, observational monitoring of catchment condition as well as inspection of conditionally approved activities.
Data collected on catchment condition and water quality plays an important role in the risk assessment and control activities of water utilities. Water quality sampling is a long-standing practice; however, water quality samples provide a limited view of what has happened in the catchment area. Furthermore, to sample for all possible contaminants is very costly and highly impractical. On the other hand, data obtained through observations if understood correctly can provide information on what threats to drinking water quality may occur. Using observational information and together can provide a more accurate representation of the risks in the catchment and the performance of control measures. In the ADWG [2], the guidelines for data collection, setting targets and responding to targets presents the same process for both water quality monitoring and observational monitoring.

CSF 3: Support and enabling of operational activities

The three factors that make up this group (19, 20 and 17) are centered around enabling the collection of appropriate information to support effective catchment management planning. CSF 2 is based on the use of catchment information to assess risk and plan controls. In CSF 3, the functions are centered around field staff being able to use current catchment information to plan and execute their functions in collection of data relating to the overall catchment condition. Catchment areas can be very large, covering significant parcels of land, and monitoring catchment condition is a very resource-intensive task. Additionally, many of the high-risk activities can be spatially or temporally transient in nature. To ensure that field operations realize the required outcomes in managing drinking water risk the planning of operational activities needs to be based on the most accurate catchment information available. Furthermore, field staff across most agencies are required to complete a vast portfolio of tasks in the course of their regular duties in addition to response to incidents such as wildfire. For field staff, being able to successfully execute the large variety of tasks and keep up to date with advancing practices in source protection, there needs to be a program of continual training and skills development.

CSF 4: Response to water quality threats

This CSF is comprised of factors 13 and 16. Though CSF 4 contains only two factors, it was retained as there was good alignment between the two items, as well as the good reliability of their scores. The factors that make up CSF 4 are related to the identification of emerging threats and responding to the threats to water quality using regulatory powers. For the protection of drinking water quality often regulatory instruments are used to control human access, land use, polluting activities, etc.; however, regulatory controls, to be effective, need to be actively enforced. Successful enforcement often requires multiple agencies to work together using their respective powers. Emerging threats to catchments referred to in this study arise in two main ways, the first being potential for the discovery of certain compounds to have potential health impacts and significant changes to land uses or activities in the catchment which could impact water quality.

5. Conclusions

First and foremost, for the supply of safe drinking water is the need to ensure the protection of public health for the consumers without the burden of excessive supply costs. Protecting drinking water catchments reduces health risks associated with enteric pathogens, chemical contamination, toxic compounds produced by cyanobacteria, and disinfection by products. Capitalizing on catchment ecosystems and their services to producing clean runoff has been shown to reduce or offset costs when serving alongside necessary conventional built infrastructure to provide an integrated system to deliver cost effective drinking water [49].
The biggest challenge in understanding risk in catchment management systems is that control structures for drinking water catchments are complex and often not understood well at a systems level. Without a full understanding of the performance of risk control measures the final risk assessments made can provide misleading information on the risks present in the drinking water supply system. As shown in the study, the programs in place to manage source protection are complex sociotechnical systems involving policy, standards, regulators, technology, human factors and so on. With technology, automation, remote monitoring and other technological advancements play a greater role in catchment management and source protection these systems only become more complex. The inherent nature of source protection programs means that many conventional hazard analysis methods are not well suited to such a system and there is a need to investigate more novel methods of hazard analysis.
In this study, we showed the potential of the STAMP-based STPA method for hazard analysis in source protection programs to identify and plan countermeasures to address operational hazards to ensure safe drinking water. The top-down approach of STPA provided a clear picture of how the risk control measures for preventing catchment contamination are managed all the way from government policy and regulation through to field operations. Having this whole-of-system perspective is essential when trying to understand and assess the effectiveness of control measures for ensuring safe drinking water during the risk assessment process. For a water utility, this type of hazard analysis tool is beneficial for understanding where weakness in current operational risk systems may be present and guide the implementation of targeted strategies. Furthermore, for a catchment management agency, the higher level of detail in understanding hazards and control measures and their performance has the potential to provide a higher level of confidence in catchment management systems. Having a high level of confidence that management systems are effective in controlling the process risks in delivering water quality outcomes results in the potential for investing in catchment management interventions being a viable alternative to more costly treatment solutions. The targeted questionnaire used in the study captured the perspective of professionals and specialists in Taiwan, Australia, and Greece involved in the management of drinking water catchments to validate source protection measures identified through STPA. The view of catchment management specialists surveyed aligned well with principles of good practice in the planning and execution of source protection operations based on the STPA results. The PCA of the questionnaire responses identified four CSFs based on the 20 questions. These CSFs give an insight into the factors which influence the overall success for source protection programs based on the perceptions of professionals and experts working in the field. The CSFs identified included “Effective Policy and Government Agency Support of Source Protection”, “Catchment condition information and risk monitoring” and “Response to water quality threats”. The CSFs represent the diversity of factors which influence the success of source protection programs.
The water industry is continually striving to meet the public’s expectations for safe water through setting higher standards to address health concerns from DBPs, pathogens, chemical contaminants, etc. The introduction of new risk management frameworks such the proposed HBT, operational management systems need to become more sophisticated to ensure the objectives set are being continually met. Without advancements in hazard identification and understanding of CSFs a potential exists for asynchronous evolution where the advancements in risk management frameworks place greater pressure on risk controls without adequate understanding of operational hazards in delivering those controls. This study highlights the value of STPA in identifying potential hazard in DWSP programs through being able to handle the complexity of a typical DWSP program.
While the survey results showed there is a fundamental understanding of good practice in catchment operations, to support advancements in source protection practices needed due to evolving standards and expectations more research is required to understand the barriers to successful implementation of source protection programs. Understanding these barriers is fundamental to better optimizing the efforts of the catchment management agencies to get the most from current policy and organizational constraints, as well as in identifying the requirements to support future improvements in source protection practice. Further research such as follow on structural equation modeling (SEM) of the CSFs identified will provide further insight to the management constructs that influence the development of source protection programs. Progressing towards developing a better understanding of system hazards in the operational management drinking water catchments can support relevant agencies manage and invest in catchment areas as a cost-effective alternative to water treatment infrastructure.

Author Contributions

Conceptualization, H.C.M., W.T.C. and J.J.H.; methodology, H.C.M., W.T.C. and J.J.H.; software, H.C.M.; formal analysis, H.C.M.; resources, W.T.C. and J.J.H.; writing—original draft preparation, H.C.M.; writing—review and editing, H.C.M. and W.T.C.

Funding

This research received no external funding.

Acknowledgments

The authors would like to acknowledge the valuable input of the selected specialists who provided their time and expertise to review the initial questionnaires and provide invaluable feedback, as well as all the respondents that took the time to complete the surveys and provide their valued opinions. The authors are also grateful for the valuable comments and suggestions provided by the reviewers.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. WHO. Guidelines for Drinking-water Quality: Fourth Edition Incorporating the First Addendum. World Health Organisation: Geneva, Switzerland, 2017; ISBN 978-92-4-154995-0. [Google Scholar]
  2. NHMRC; NRMMC. Australian Drinking Water Guidelines Paper 6 National Water Quality Management Strategy; National Health and Medical Research Council; National Resource Management Ministerial Council, Commonwealth of Australia: Canberra, Australia, 2011; ISBN 186-496-5118.
  3. Postel, S.L.; Thompson, B.H. Watershed protection: Capturing the benefits of nature’s water supply services. Nat. Resour. Forum 2005, 98, 98–105. [Google Scholar] [CrossRef]
  4. Davies, J.M.; Mazumder, A. Health and environmental policy issues in Canada: The role of watershed management in sustaining clean drinking water quality at surface sources. J. Environ. Manag. 2003, 68, 273–286. [Google Scholar] [CrossRef]
  5. Azara, A.; Castiglia, P.; Piana, A.; Masia, M.D.; Palmieri, A.; Arru, B.; Maida, G.; Dettori, M. Derogation from drinking water quality standards in Italy according to the European Directive 98/83/EC and legislative Decree 31/2001-A look at the recent past. Ann. Ig. 2018, 30, 517–526. [Google Scholar] [CrossRef] [PubMed]
  6. Hrudey, S.E.; Hrudey, E.J.; Pollard, S.J.T. Risk management for assuring safe drinking water. Environ. Int. 2006, 32, 948–957. [Google Scholar] [CrossRef] [PubMed]
  7. Wu, S.; Hrudey, S.; French, S.; Bedford, T.; Soane, E.; Pollard, S. A role of human reliability analysis (HRA) in preventing drinking water incidents and securing safe drinking water. Water Res. 2009, 43, 3227–3238. [Google Scholar] [CrossRef]
  8. Cook, R.I. How Complex Systems Fail; Cognitive Technologies Laboratory, University of Chicago: Chicargo, IL, USA, 2000; pp. 1–2. [Google Scholar]
  9. Hrudey, S.E.; Payment, P.; Huck, P.M.; Gillham, R.W.; Hrudey, E.J. A fatal waterborne disease epidemic in Walkerton, Ontario: Comparison with other waterborne outbreaks in the developed world. Water Sci. Technol. 2003, 47, 1–14. [Google Scholar] [CrossRef]
  10. Leveson, N.M.; Daouk, M.; Dulac, N.; Marais, K. Applying STAMP in Accident Analysis. In Proceedings of the 2nd Workshop on the Investigation and Reporting of Accidents, Hampton, VA, USA, September 2003; pp. 177–198. [Google Scholar]
  11. Bartram, J.; Corrales, L.; Davison, A.; Deere, D.; Drury, D.; Gordon, B.; Howard, G.; Rinehold, A.; Stevens, M. Water Safety Plan Manual: Step-by-step Risk Management for Drinking-water Suppliers; World Health Organisation: Geneva, Switzerland, 2009; pp. 1–5. ISBN 978-924-1-56263-8. [Google Scholar]
  12. Nadebaum, P.; Chapman, M.; Morden, R.; Rizak, S. A Guide to Hazard Identification & Risk Assessment for Drinking Water Supplies—Research Report 11; The Cooperative Research Centre for Water Quality and Treatment: Salisbury, Australia, 2004; pp. 7–18. ISBN 1876616121.
  13. Hokstad, P.; Røstum, J.; Sklet, S.; Rosén, L.; Pettersson, T.J.; Linde, A.; Sturm, S.; Beuken, R.; Kirchner, D.; Niewersch, C. Methods for Risk Analysis of Drinking Water Systems from Source to Tap—Guidance Report on Risk Analysis; TECHNEAU report: Marigni, France, 2009; pp. 50–70. Available online: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.367.4044&rep=rep1&type=pdf (accessed on 2 January 2019).
  14. Sulaman, S.M.; Beer, A.; Felder, M.; Host, M. Comparison of FMEA and STPA safety analysis methods: A case study. Softw. Qual. J. 2017, 1–39. [Google Scholar] [CrossRef]
  15. Merrett, H.C.; Chen, W.T.; Horng, J.J. Managing water catchments as water treatment assets for water quality and source protection. In Proceedings of the 6th Annual International Conference on Architecture and Civil Engineering, Singapore, 14–15 May 2018. [Google Scholar] [CrossRef]
  16. Rasmussen, J. Risk Management in a dynamic society: A modeling problem. Saf. Sci. 1997, 23, 183–213. [Google Scholar] [CrossRef]
  17. McInnes, R.R.; de Groot, J.; Plant, R.; Chong, J.; Olsazk, C. Managing Catchments as Business Assets: An Economic Framework for Evaluating Control Measures for Source Water Protection; Research Report No. 83; Water Quality Research Australia: Adelaide, Australia, 2008; ISBN 1876616334. [Google Scholar]
  18. Bennet, G.; Carroll, N. Gaining Depth: State of Watershed Investment 2014. Forest Trends Ecosystem Marketplace 2014. Available online: www.ecosystemmarketplace.com/reports/sowi2014 (accessed on 2 January 2019).
  19. WSAA. Manual for the Application of Health-Based Targets for Drinking Water Safety; Water Services Association of Australia: Sydney, Australia, 2015; pp. 15–25. ISBN 1-920760-68-7. [Google Scholar]
  20. Pollard, S.J.T.; Strutt, J.E.; Macgillivray, B.H.; Hamilton, P.D.; Hrudey, S.E. Risk analysis and management in water utility sector: A review of drivers, tools and techniques. Process. Saf. Environ. Prot. 2004, 82, 453–462. [Google Scholar] [CrossRef]
  21. Leveson, N. Engineering a Safer World; MIT Press: Boston, MA, USA, 2011; ISBN 978-0-262-53369-0. [Google Scholar]
  22. Jacinto, C.; Silva, C. A semi-quantitative assessment of occupational risks using bow-tie representation. Saf. Sci. 2010, 48, 973–979. [Google Scholar] [CrossRef]
  23. Cox, A.L. What’s wrong with risk matrices. Risk Anal. 2008, 28, 497–512. [Google Scholar]
  24. Guice, J.; Miller, R.; Deere, D. Risk Assessment for Drinking Water Supplies; CRC for water quality and research: Adelaide, Australia, Report 78; 2009; ISBN 187-661-6288.
  25. Leveson, N. A new accident model for engineering safer systems. Saf. Sci. 2004, 42, 237–270. [Google Scholar] [CrossRef]
  26. Leveson, N. A systems approach to risk management through leading safety indicators. Reliab. Eng. Syst. Saf. 2015, 136, 7–34. [Google Scholar] [CrossRef]
  27. Iyer, K.C.; Banerjee, P.S. Measuring benchmarking managerial efficiency of project execution schedule performance. Int. J. Proj. Manag. 2016, 34, 219–236. [Google Scholar] [CrossRef]
  28. Lingard, H.; Francis, V.; Michelle, T. Work–life strategies in the Australian construction industry: Implementation issues in a dynamic project-based work environment. Int. J. Proj. Manag. 2012, 30, 282–295. [Google Scholar] [CrossRef]
  29. Cameron, I.; Mannan, S.; N’emeth, E.; Pasman, H.; Rogers, W.; Seligmann, B. Process Hazard Analysis, Hazard Identification and Scenario Definition: Are the conventional tools sufficient, or should and can we do much better? Process. Saf. Environ. Prot. 110. [CrossRef]
  30. Gong, Y.; Li, Y. STAMP-based causal analysis of China-Donghuang oil transportation pipeline leakage and explosion accident. J. Loss Prev. Process. Ind. 2018, 56, 402–413. [Google Scholar] [CrossRef]
  31. Leveson, N.; Thomas, J. STPA Handbook. Available online: http://psas.scripts.mit.edu/ home/get_file.php?name=STPA_handbook.pdf. (accessed on 3 September 2018).
  32. Merrett, H.C.; Horng, J.J.; Piggott, A.; Qandour, A.; Chen, W.T. Comparison of STPA and Bowtie Method in the development and testing of an automated water quality management system. European STAMP Workshop & Conference, Amsterdam 31 October–2 November 2018. MATEC Web Conf. 2019, 273. [Google Scholar] [CrossRef]
  33. Boyton, A.C.; Zmund, R.W. An assessment of critical success factors. Sloan Manag. Rev. 1984, 25, 17–27. [Google Scholar]
  34. Tabachnick, B.G.; Fidell, L.S. Using Multivariate Statistics; Pearson: New York, NY, USA, 2013; ISBN 0-205-89081-4. [Google Scholar]
  35. Li, Y.; Ning, Y.; Chen, W.T. Critical Success Factors for Safety Management of High-Rise Building Construction Projects in China. Adv. Civ. Eng. 2018. 1516354. [Google Scholar] [CrossRef]
  36. WHO. Health Based Targets; World Health Organisation: Geneva, Switxerland, 2010; Available online: https://www.who.int/water_sanitation_health/WHS_WWD2010_health_targets_2010_8_en.pdf (accessed on 2 January 2018).
  37. Rickert, B.; Chorus, I.; Schmoll, O. Protecting Surface Water for Health. Identifying, Assessing and Managing Drinking-water Quality Risks in Surface-water Catchments; World Health Organisation: Geneva, Switzerland, 2016; ISBN 978 92 4 151055 4. [Google Scholar]
  38. Hinkin, T.R. A Brief Tutorial on the Development of Measures for Use in Survey Questionnaires; School of Hotel Administration, Cornell University: Ithica, NY, USA, 1998; Available online: http://scholarship.sha.cornell.edu/articles/521 (accessed on 2 May 2018).
  39. Hinton, P.R.; McMurray, I.; Brownlow, C. SPSS Explained; Routledge: New York, NY, USA, 2004; ISBN 978-1-315-79729-8. [Google Scholar]
  40. Gay, L.R. Educational Research: Competencies for Analysis and Application; Merrill: Englewood Cliffs, NJ, USA, 1996; ISBN 0675205069. [Google Scholar]
  41. Nunnally, J.C. Psychometric Theory; McGraw Hill: New York, NY, USA, 1978; ISBN 0070474656 9780070474659. [Google Scholar]
  42. Field, A. Discovering Statistics Using SPPS; Sage Publications: Thousand Oaks, CA, USA, 2009; ISBN 978-1-84787-906-6. [Google Scholar]
  43. Razali, N.M.; Wah, Y.B. Power comparisons of Shapiro-Wilk, Kolmogorov-Smirnov, Lilliefors and Anderson-Darling tests. J. Stat. Model. Anal. 2011, 2, 21–33. [Google Scholar]
  44. Mir, F.A.; Pinnington, A.H. Exploring the value of project management: Linking Project Management Performance and Project Success. Int. J. Proj. Manag. 2014, 32, 202–217. [Google Scholar] [CrossRef]
  45. Williams, B.; Onsman, A.; Brown, T. Exploratory factor analysis: A five-step guide for novices. J. Emerg. Prim. Health Care 2010, 8, 1–13. [Google Scholar] [CrossRef]
  46. Kaiser, H.F. The application of electronic computers to factor analysis. Educ. Psychol. Meas. 1960, 20, 141–151. [Google Scholar] [CrossRef]
  47. Comrey, A.L.; Lee, H.B. A First Course in Factor Analysis; Lawrence Erlbaum Associates: Hillsdale, MI, USA, 1992; ISBN 9780805810622. [Google Scholar]
  48. Plummer, R.; Velaniskis, J.; de Grosbois, D.; Kreutzwiser, R.D.; de Loe, R. The development of new environmental policies and processes in response to a crisis: The case of the multiple barrier approach for safe drinking water. Environ. Sci. Policy 2010, 13, 535–548. [Google Scholar] [CrossRef]
  49. Gartner, T.; Mulligan, J.; Schmidt, R.; Gunn, J. Natural Infrastructure: Investing in Forested Landscapes for Source Water Protection in the United States; WRI: Washington, DC, USA, 2013; ISBN 978-1-56973-813-9. [Google Scholar]
Figure 1. The generic control structure used in STPA adapted from [26].
Figure 1. The generic control structure used in STPA adapted from [26].
Sustainability 11 02606 g001
Figure 2. Safety control structure of the theoretical surface water catchment suppling a bulk storage reservoir. Note: Solid lines indicate direction of control actions from the controller to a controlled process, dashed lines indicate direction of feedback from the controlled process to controllers in the system.
Figure 2. Safety control structure of the theoretical surface water catchment suppling a bulk storage reservoir. Note: Solid lines indicate direction of control actions from the controller to a controlled process, dashed lines indicate direction of feedback from the controlled process to controllers in the system.
Sustainability 11 02606 g002
Figure 3. Scree plot of the eigenvalue versus the component number.
Figure 3. Scree plot of the eigenvalue versus the component number.
Sustainability 11 02606 g003
Table 1. The accidents, hazards and safety constraints selected for step one of the STPA method.
Table 1. The accidents, hazards and safety constraints selected for step one of the STPA method.
AccidentHazard IDHazardSafety Constraint IDSafety Constraint
Public Illness due to Drinking Water ContaminationH1Regulatory requirements/drinking water standards do not protect public healthSC1All regulatory requirements/drinking water standards must protect public health to an acceptable level
H2Chemical or microbial contamination of drinking water supplySC2All parameters must be below stipulated guidelines
H2.1Microbial/chemical contamination from transient human activity in catchment areaSC2.1Human access is maintained to target levels and locations to prevent microbial contamination above acceptable levels
H2.2Microbial/chemical contamination from domestic livestock waste entering stream flowSC2.2Waste from livestock must not be allowed to enter catchment waterways
H2.3Microbial/chemical contamination of runoff from fixed location human activities in catchment areaSC2.3Runoff from fixed location human activities must not contaminate catchment
H2.4Natural disasters result in microbial/chemical contamination of runoffSC2.4Prevention and response to incidents/natural disasters and natural disasters must minimize microbial/chemical contamination of runoff
Table 2. Principle actors, responsibilities and control actions in the source protection safety system.
Table 2. Principle actors, responsibilities and control actions in the source protection safety system.
Control Structure ComponentDescription of Roles and ResponsibilitiesControl Actions Provided in the STPA Control Structure
GovernmentAssign relevant government departments the task of protection of natural areas, water resources, and public healthPass Law for the protection of natural areas and public health
Provide Resourcing required to meet objectives
Set policy for the protection of natural areas and public health
Natural Resource Management AgenciesThese are the governmental agencies assigned the task of land management and Regulating land uses in catchments which have potential to impact on water qualityEnforce approval conditions
Enforce relevant legislation
Natural hazard reduction
Respond to natural disasters
Public Health RegulatorRegulate public health aspects of public drinking water supply through a formal relationship with the Water Service ProviderSet requirements for drinking water safety
Requirements for safe drinking water
Water Quality PlanningInterpret catchment data to assess the level of risk
Based on the level of risk plan catchment
management operations to monitor and control risk to acceptable levels
Set operational risk management requirements
Set limits for water quality
Incident response protocols
Set limits for observational data
Internal policy requirements for water quality
Set QA requirements for water sampling
Bulk Water SupplierUse the water quality planning information to implement risk management operationsOperational requirements for water quality
Configure remote monitoring system
Set event alert criteria
Create work orders: Field operations
Work planning: Field operations
Set water quality analysis parameters
Incident management
Set conditional approvals
Drinking Water Catchment AreaThe area of land where water for the public drinking water supply is capturedControl water quality to reservoir
Remote Monitoring System Send real-time information to field operators and provide data to water quality planningSystem configuration commands
Drinking Water StandardsSet guidelines for ensuring safe drinking water in public drinking water suppliesSet industry standards for safe drinking water
Corporate Governance—WQPThe executive management of the water service provider. Responsible for the strategic direction and resources for completing the functions of water quality planningProvide oversight
Provide resourcing
Corporate policy
Operators—Field OperationsUndertake the planned actions in the field required to protect drinking water objectives and respond to threats as they ariseEnforce bylaws
Enforce approval conditions
Inspect catchment condition
Incident response
Collect water samples
Collect in situ water quality data
Inspect reservoir condition
Operator—Maintenance/RepairRepair and maintain catchment assets based on relevant work plansRepair assets
Inspect asset/barrier condition
Maintain field monitoring equipment
Inspect asset condition
Water LabAnalyse water samplesN/A (operates as a sensor in STPA)
Corporate Governance—BWSThe executive management of the bulk water supplier responsible for strategic direction and provision of resourcesProvide oversight
Provide resourcing
Corporate policy
Field Operations SupervisorSchedule assigned work and assign resources to catchment operationsWork schedule: Field operations
Maintenance SupervisorOversee that the maintenance team complete all tasks to meet the requirements of the bulk water supplierWork schedule: Maintenance and repair
Table 3. Examples of the control actions and scenarios where the control actions could be unsafe.
Table 3. Examples of the control actions and scenarios where the control actions could be unsafe.
Control ActionNot Providing Control ActionProviding Control Action Causes HazardProviding Control Action Too Early, Too Late, or Out of OrderStopping Control Action Too Soon or Applying Too Long
Set Operational Limits for Water Quality Parameters(UCA13-N-1) Water quality planning do not set operating limits for water quality parameters to protect public health(UCA13-P-1) Water quality planning set operating limits which do not protect public health
(UCA13-P-2) Water quality limits do not match the performance of downstream water treatment processes
(UCA13-T-1) Water quality limits are delayed in being amended to reflect changes in requirements for public health
(UCA13-T-2) Water quality limits are not amended to reflect changes in catchment condition or requirements of the supply train
N/A
Inspect Catchment Area (UCA15-N-1) Field operators do not inspect catchment condition when in the field(UCA15-P-1) Incorrect section of the catchment area inspected when conducting catchment inspections
(UCA15-P-2) High risk activities/conditions are missed when conducting catchment inspections
(UCA15-T-1) Catchment inspections are undertaken after water quality impacts have occurred(UCA15-D-1) catchment inspections are ceased before covering the required areas
Natural Hazard Reduction i.e., Fuel Reduction(UCA36-N-1) Natural hazard levels are not maintained to levels required for protection of drinking water quality(UCA36-P-1) Hazard reduction program fails to reduce risk in the catchment area
(UCA36-P-2) Hazard reduction activities impacts on ecosystem services
(UCA36-P-3) The hazard reduction activities introduce contamination to the water
(UCA16-T-1) The timing of the hazard reduction reduces the effectiveness of the hazard reduction actions(UCA36-T-1) Hazard reduction program ceased before fully completed
Table 4. An example of the causal factors identified, and the countermeasures developed following STPA.
Table 4. An example of the causal factors identified, and the countermeasures developed following STPA.
UCACausal FactorCountermeasure
(UCA13-N-1) Water quality planning do not set limits to protect public health on water sample resultsThere is no industry agreed limit for the parameter of interest (potentially the case for emerging water quality issues)Work with relevant industry bodies to research emerging issues and provide an agreed position on limits to protect public health
(UCA15-P-1) Incorrect section of the catchment area inspected when conducting catchment inspectionsThe field operator misses key risk areas of the catchment when conducting inspectionsField staff are provided with a plan showing all the areas required to be inspected as part of normal surveillance operations which includes fixed location and transient high-risk activities
(UCA36-N-1) Natural disaster hazard levels are not maintained to levels required for protection of drinking water qualityThe natural resource management agency is not aware of the risk that the natural hazard levels in the catchment area have on drinking water qualityProvide information gained during surveillance that indicates potential hazards to drinking water quality from natural disaster events e.g., high fuel loads
Table 5. Countermeasure groups derived from the countermeasures identified following the STPA process.
Table 5. Countermeasure groups derived from the countermeasures identified following the STPA process.
Countermeasure Groups
1.
Effective government policies focused on the protection of drinking water catchments.
2.
Effective implementation of Government policy for the protection of drinking water quality catchments.
3.
Regular reviews of policy in response to emerging threats to drinking water catchments.
4.
Engagement with relevant stakeholders when developing or reviewing policy for the protection of drinking water catchments.
5.
NRM agencies understanding of drinking water catchment management principles.
6.
NRM agencies support for the protection of drinking water catchments.
7.
Use of statutory authority by government agencies including NRM agencies to control potentially polluting activities within in drinking water catchments.
8.
Management of natural disasters risk and response actions (i.e., Fire, flood, erosion/landslides) by applicable NRM agencies.
9.
Relationship management between NRM agencies and drinking water management agencies.
10.
Availability of accurate land use information to support catchment management planning activities and field operations.
11.
Availability of accurate observational information on catchment condition to the catchment management planning process.
12.
Use of water quality monitoring data in the catchment management planning process.
13.
Regular targeted monitoring of land use activities in catchment to support drinking water quality risk management.
14.
Effective catchment management planning practices for identifying and managing emerging threats to drinking water quality.
15.
Operations to reduce or control risk to drinking water quality based on the latest inspection and surveillance data.
16.
Capture of real time information on catchment activities to provide information for guiding operations to control risk to drinking water quality.
17.
Use of water quality data for the planning field operations to reduce risk to drinking water quality.
18.
Enforcement of legal controls over regulated activities in drinking water catchments to reduce drinking water quality risk.
19.
Engagement with private land holders and other users of catchment areas to support risk control and reduction in drinking water catchments.
20.
Provision of continuous training to field staff for effective catchment operations for the protection of drinking water.
21.
Resourcing required to complete source protection related works and tasks.
22.
Effective maintenance planning processes to support risk management operations and infrastructure.
23.
Processes for enabling/supporting of catchment management strategy.
24.
Processes for enabling/supporting of catchment management planning.
25.
Processes for enabling/supporting of catchment management operations.
26.
Processes for work scheduling to ensure all operations are completed as planned.
Table 6. Summary of respondents’ years of experience in catchment management and source protection.
Table 6. Summary of respondents’ years of experience in catchment management and source protection.
Years’ ExperienceFrequencyPercent
<52744%
5–101728%
10–201423%
>2035%
Total61100%
Table 7. Summary of respondents’ role type in their respective organization.
Table 7. Summary of respondents’ role type in their respective organization.
RoleFrequencyPercent
Operations Support2541%
Strategy and Policy1016%
Technical/Scientific Support1830%
Water Quality Planning35%
Other58%
Total61100%
Table 8. ANOVA results for comparison of all groups.
Table 8. ANOVA results for comparison of all groups.
StatementCountry (df = 2)Role (df = 4)Experience (df = 3)Organization (df = 4)
F-TestSig.F-TestSig.F-TestSig.F-TestSig.
12.8980.0631.4580.2102.6180.0600.7670.551
21.0040.3731.2890.2781.6290.1932.9480.028
30.940.3971.1560.3431.4560.2361.1950.323
41.590.2132.1400.0631.2260.3093.2870.017
50.7020.52.5730.0291.0710.3691.5140.211
62.1740.1231.5590.1772.0470.1183.0470.024
70.3280.7221.5180.1904.1450.0103.6470.010
81.9340.1543.5210.005 *1.5130.2212.1430.087
93.5650.0352.0060.0812.0990.1102.3010.070
100.4430.6450.3210.9230.1950.8991.5610.197
111.7210.1880.7140.6400.8680.4632.0030.106
120.3940.6760.8310.5521.0240.3890.5280.715
130.4370.6481.6780.1440.4430.7230.4860.746
142.3260.1070.4730.8250.9120.4410.6390.637
151.4550.2420.6860.6611.5760.2052.2460.076
160.4530.6380.1600.9860.2470.8631.2650.295
174.7230.0131.4530.2121.5110.2220.7710.549
181.4450.2441.0640.3960.7760.5120.6230.648
193.8910.0262.5920.0281.2220.3100.8450.503
202.7360.0730.9310.4800.4970.6862.6480.043
* significant results at the p =< 0.010 level; df = degrees of freedom within the group.
Table 9. The ranked means of the survey results shown by organizational group.
Table 9. The ranked means of the survey results shown by organizational group.
FactorAll
n = 61
Consulting
n = 4
Government Agency
n = 25
Research/ University
n = 10
Water Utility/Water Service Provider
n = 18
Other
n = 4
Mean ScoreRankMean ScoreRankMean ScoreRankMean ScoreRankMean ScoreRankMean ScoreRank
13.61143.50103.72163.33133.74133.1714
23.61133.25114.0853.00163.47163.1714
33.34193.00153.64192.89173.32193.1714
43.69122.50164.0083.22153.74133.5011
53.54153.25113.84123.33123.42183.1714
63.34192.50163.64192.78193.53153.0019
73.39182.50163.80142.78193.47163.0019
83.41172.50163.80153.33133.21203.1714
93.48162.50163.72182.78183.74123.5011
104.5415.0014.5214.6714.5833.679
114.4825.0014.4424.4444.6313.5011
124.4634.7534.4434.4444.5354.003
133.77113.25113.72173.89103.89113.679
144.4144.7534.2844.4434.5344.003
154.2364.5053.84124.5624.4774.331
164.02103.25113.92114.2274.16104.003
174.2074.0084.0854.1184.4763.838
184.1884.5054.0854.0094.3294.172
194.0794.0083.92103.78114.3784.003
204.2854.5053.9694.3364.6314.003
Table 10. Rotated components identified, loadings and variance.
Table 10. Rotated components identified, loadings and variance.
Criteria and Characteristic VariablesLoadingVariance PercentageCumulative Variance Percentage
CSF 1: Policy and Government Agency Support of Source Protection40.13%40.13%
6: Natural Resource Management agencies support the protection of drinking water catchments0.892--
2: Government agencies actively implement policy for the protection of drinking water quality catchments0.835
5: Natural Resource Management agencies have a good understanding of drinking water catchment management principles.0.777
7: Natural Resource Management agencies use their authority to manage activities within in drinking water catchments0.769
3: Government provides timely policy responses to emerging threats to drinking water catchments.0.766
1: Current government policies provide robust protection of drinking water catchments0.746
9: Natural Resource Management agencies see drinking water management agencies as key stakeholders0.759
4: Government engaging with relevant stakeholders when developing or reviewing policy for the protection of drinking water catchments0.668
8: Natural Resource Management agencies effectiveness in managing risk of natural disasters (i.e., Fire, flood, erosion/landslides)0.619
CSF 2: Catchment Risk Monitoring and Information19.77%59.90%
10: Effective catchment management planning requires current land use information0.898--
11: Observational information on catchment condition is critical to the catchment management planning process0.884
12: Effective catchment management planning requires water quality monitoring data0.854
14: Regular surveillance of human activities in catchment areas is essential to manage risk to drinking water quality0.721
15: Real time information on catchment activities is required for effective control of risk to drinking water quality.0.630
CSF 3: Support of Operational Field Activities6.29%66.19%
19: Engagement with private land holders is essential in reducing risk in drinking water catchments.0.839--
20: Continuous training of field staff is critical to effective catchment operations for the protection of drinking water.0.772
17: Data collected through inspections/surveillance is valuable in planning operations to reduce risk to drinking water quality0.568
CSF 4: Response to Water Quality Threats5.32%71.51%
13: Best practice catchment management planning practices can identify and manage emerging threats to drinking water quality 0.788--
16: Enforcement of legal controls over activities in drinking water catchments are effective in reducing drinking water quality risk0.716

Share and Cite

MDPI and ACS Style

Merrett, H.C.; Chen, W.T.; Horng, J.J. A Systems Analysis Approach to Identifying Critical Success Factors in Drinking Water Source Protection Programs. Sustainability 2019, 11, 2606. https://doi.org/10.3390/su11092606

AMA Style

Merrett HC, Chen WT, Horng JJ. A Systems Analysis Approach to Identifying Critical Success Factors in Drinking Water Source Protection Programs. Sustainability. 2019; 11(9):2606. https://doi.org/10.3390/su11092606

Chicago/Turabian Style

Merrett, Hew Cameron, Wei Tong Chen, and Jao Jia Horng. 2019. "A Systems Analysis Approach to Identifying Critical Success Factors in Drinking Water Source Protection Programs" Sustainability 11, no. 9: 2606. https://doi.org/10.3390/su11092606

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop