Are We Objective? A Study into the Effectiveness of Risk Measurement in the Water Industry
Abstract
:1. Introduction
Technical Risk Measurement
2. Materials and Methods
2.1. Transforming Data
2.2. Defining “Objective”
3. Results
3.1. Risk Scoring within Organizations
3.2. Choice of Projects Impact upon Scores
4. Conclusions
Author Contributions
Funding
Acknowledgments
Conflicts of Interest
Appendix A
Q-Q Plots by Project
References
- Council of Standards Australia. Risk Management—Principles and Guidelines; Standards Australia: Sydney, Australia, 2009. [Google Scholar]
- Hubbard, D.; Evans, D. Problems with scoring methods and ordinal scales in risk assessment. IBM J. Res. Dev. 2010, 54, 2:1–2:10. [Google Scholar] [CrossRef] [Green Version]
- Aven, T. Improving risk characterisations in practical situations by highlighting knowledge aspects, with applications to risk matrices. Reliab. Engin. Syst. Saf. 2017, 167, 42–48. [Google Scholar] [CrossRef]
- Ball, D.J.; Watt, J. Further Thoughts on the Utility of Risk Matrices. Risk Anal. 2013, 33, 2068–2078. [Google Scholar] [CrossRef] [PubMed]
- Starr, C. Social Benefit versus Technological Risk. Science 1969, 165, 1232–1238. [Google Scholar] [CrossRef] [PubMed]
- United States Nuclear Regulatory Commission. Reactor Safety Study: An Assessment of Accident Risks in U.S. Commercial Nuclear Power Plants; United States Nuclear Regulatory Commission: Washington, DC, USA, 1975.
- Self, P. Econocrats and the Policy Process: The Politics and Philosophy of Cost-Benefit Analysis; Macmillan: London, UK, 1975. [Google Scholar]
- Beck, U. Risk Society: Towards a New Modernity; Sage Publications: London, UK, 1992. [Google Scholar]
- Von Neumann, J.; Morgenstern, O. Theory of Games and Economic Behavior. [Electronic Resource]; Princeton classic editions; Princeton University Press: Princeton, NJ, USA, 2007. [Google Scholar]
- Lupton, D. Risk; Routledge: New York, NY, USA; Oxon, UK, 2013. [Google Scholar]
- Renn, O. Concepts of Risk: A Classification. In Social Theories of Risk; Praeger Publishers: Westport, CT, USA, 1992; pp. 53–82. [Google Scholar]
- Paté-Cornell, E. On “Black Swans” and “Perfect Storms”: Risk Analysis and Management When Statistics Are Not Enough. Risk Anal. 2012, 32, 1823–1833. [Google Scholar] [CrossRef] [PubMed]
- Pate-Cornell, E. Risk and Uncertainty Analysis in Government Safety Decisions. Risk Anal. 2002, 22, 633–646. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Flage, R.; Aven, T.; Zio, E.; Baraldi, P. Concerns, Challenges, and Directions of Development for the Issue of Representing Uncertainty in Risk Assessment. Risk Anal. 2014, 34, 1196–1207. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Monat, J.P.; Doremus, S. Deficiencies in and Alternatives to Heat Map Risk Matrices for Project Risk Prioritization. J. Mod. Proj. Manag. 2018, 6. [Google Scholar]
- Victorian Managed Insurance Authority. The Victorian Government Risk Management Framework Practice Guide; Victorian Managed Insurance Authority: Melbourne, Australia, 2016.
- Australian Government, Department of Industry, Innovation and Science. Quantification of Consequence/Likelihood Matrices. Available online: https://archive.industry.gov.au/resource/Programs/LPSD/Risk-management/Appendices/Appendix-1-Risk-analysis/Pages/Quantification-of-consequencelikelihood-matrices.aspx (accessed on 15 February 2019).
- Van Der Sluijs, J.P.; Craye, M.; Funtowicz, S.; Kloprogge, P.; Ravetz, J.; Risbey, J. Combining Quantitative and Qualitative Measures of Uncertainty in Model-Based Environmental Assessment: The NUSAP System. Risk Anal. 2005, 25, 481–492. [Google Scholar] [CrossRef] [PubMed]
- Ruan, X.; Yin, Z.; Frangopol, D.M. Risk Matrix Integrating Risk Attitudes Based on Utility Theory. Risk Anal. 2015, 35, 1437–1447. [Google Scholar] [CrossRef] [PubMed]
- Kosovac, A.; Davidson, B.; Malano, H.; Cook, J. The Varied Nature of Risk and Considerations for the Water Industry: A Review of the Literature. Environ. Nat. Resour. Res. 2017, 7, 80–86. [Google Scholar] [CrossRef]
- Thompson, M.; Ellis, R.; Wildavsky, A. Cultural Theory (Political Cultures Series); Westview Press: Boulder, CO, USA, 1990. [Google Scholar]
- Slovic, P. Perception of Risk. In Social Theories of Risk; Praeger Publisher: Westport, CT, USA, 1992; pp. 117–153. [Google Scholar]
- Douglas, M.; Wildavsky, A.B. Risk and Culture. [Electronic Resource]: An Essay on the Selection of Technological and Environmental Dangers; University of California Press: Berkeley, CA, USA, 1982. [Google Scholar]
- Kosovac, A.; Hurlimann, A.; Davidson, B. Water Experts’ Perception of Risk for New and Unfamiliar Water Projects. Water 2017, 9, 976. [Google Scholar] [CrossRef]
- Rae, A.; Alexander, R. Forecasts or fortune-telling: When are expert judgements of safety risk valid? Saf. Sci. 2017, 99, 156–165. [Google Scholar] [CrossRef]
Fictional Projects in Survey | Brief Description |
---|---|
1 (Familiar project) | Pipe replacement along a busy road |
2 (Familiar project) | Construction of a new water pump station |
3 (Familiar project) | Construction of a recycled water treatment plant |
4 (Familiar project) | Public campaign for water conservation |
4A (Unfamiliar project) | Creating recycled water for potable uses |
4B (Unfamiliar project) | Implementation of a new radiation-based water treatment method |
4C (Unfamiliar project) | Removal of fluoride dosing from existing potable water supply |
Consequence/Likelihood | 1 (LOW CONSEQUENCE) | 2 | 3 | 4 | 5 (HIGH CONSEQUENCE) |
---|---|---|---|---|---|
1 (VERY UNLIKELY) | 1 (LOW) | 2 (LOW) | 3 (LOW) | 4 (LOW) | 5 (LOW) |
2 | 2 (LOW) | 4 (LOW) | 6 (MED) | 8 (MED) | 10 (MED) |
3 | 3 (LOW) | 6 (MED) | 9 (MED) | 12 (HIGH) | 15 (HIGH) |
4 | 4 (LOW) | 8 (MED) | 12 (HIGH) | 16 (HIGH) | 20 (EXTR) |
5 (HIGHLY LIKELY) | 5 (LOW) | 10 (MED) | 15 (HIGH) | 20 (EXTR) | 25 (EXTR) |
Risk Rating | Range from (Inclusive) | Range to (Inclusive) |
---|---|---|
Low | 1 | 4 |
Medium | 5 | 9 |
High | 10 | 16 |
Extreme | 20 | 25 |
Project | N Statistic | Range Statistic | Minimum Risk Score | Maximum Risk Score | Mean Risk Score |
---|---|---|---|---|---|
1 | 77 | 19 | 1 | 20 | 6.74 |
2 | 77 | 19 | 1 | 20 | 9.96 |
3 | 77 | 23 | 2 | 25 | 10.65 |
4 | 77 | 19 | 1 | 20 | 8.18 |
4A | 76 | 23 | 2 | 25 | 13.70 |
4B | 76 | 24 | 1 | 25 | 12.63 |
4C | 73 | 24 | 1 | 25 | 10.18 |
Project No. | Range within 1 SD (~68%) | Range within 2 SD (~95%) | ||
---|---|---|---|---|
Project 1 | 3.16 | 10.33 | 1.12 | 15.44 |
LOW | HIGH | LOW | HIGH | |
Project 2 | 4.78 | 15.16 | 1.78 | 22.53 |
LOW | HIGH | LOW | EXTREME | |
Project 3 | 5.35 | 15.96 | 2.17 | 23.39 |
MEDIUM | HIGH | LOW | EXTREME | |
Project 4 | 2.97 | 13.41 | 0.57 | 21.44 |
LOW | HIGH | LOW | EXTREME | |
Project 4A | 7.26 | 20.16 | 3.22 | 29.03 |
MEDIUM | EXTREME | LOW | EXTREME | |
Project 4B | 6.56 | 18.72 | 2.82 | 27.14 |
MEDIUM | EXTREME | LOW | EXTREME | |
Project 4C | 4.24 | 16.14 | 1.16 | 24.98 |
LOW | HIGH | LOW | EXTREME |
Project 1—Pipe Replacement | Range within 1 Standard Deviation (~68% of Data) | Range within 2 Standard Deviations (~95% of Data) | ||
---|---|---|---|---|
Organization 1 | 3.8 LOW | 13.1 HIGH | 1.3 LOW | 19.8 EXTREME |
Organization 2 | 4.1 LOW | 9.7 MEDIUM | 2.1 LOW | 13.5 HIGH |
Organization 3 | 2.8 LOW | 10.2 HIGH | 0.8 LOW | 15.6 HIGH |
Organization 4 | 2.8 LOW | 9.6 MEDIUM | 0.9 LOW | 14.6 HIGH |
Project 2—Pump Station Installation | Range within 1 Standard Deviation (~68% of Data) | Range within 2 Standard Deviations (~95% of Data) | ||
---|---|---|---|---|
Organization 1 | 4.1 LOW | 18.2 EXTREME | 0.8 LOW | 28.9 EXTREME |
Organization 2 | 4.9 LOW | 15.7 HIGH | 1.8 LOW | 23.4 EXTREME |
Organization 3 | 6.7 MEDIUM | 16.8 EXTREME | 3.3 LOW | 23.6 EXTREME |
Organization 4 | 4.6 LOW | 13.1 HIGH | 2.0 LOW | 19.0 EXTREME |
Project 3—Construct Sewage Treatment Plant/Recycled Water Treatment Plant | Range within 1 Standard Deviation (~68% of Data) | Range within 2 Standard Deviations (~95% of Data) | ||
---|---|---|---|---|
Organization 1 | 5.2 MEDIUM | 19.3 EXTREME | 1.5 LOW | 29.6 EXTREME |
Organization 2 | 5.4 MEDIUM | 14.3 HIGH | 2.6 LOW | 20.2 EXTREME |
Organization 3 | 5.5 MEDIUM | 18.6 EXTREME | 1.9 LOW | 28.1 EXTREME |
Organization 4 | 5.6 MEDIUM | 14.7 HIGH | 2.6 LOW | 20.8 EXTREME |
Project 4—Save Water Campaign | Range within 1 Standard Deviation (~68% of Data) | Range within 2 Standard Deviations (~95% of Data) | ||
---|---|---|---|---|
Organization 1 | 2.3 LOW | 11.9 HIGH | 0.3 LOW | 19.4 EXTREME |
Organization 2 | 3.6 LOW | 13.9 HIGH | 1.0 LOW | 21.5 EXTREME |
Organization 3 | 4.2 LOW | 15.6 HIGH | 1.2 LOW | 24.1 EXTREME |
Organization 4 | 2.7 LOW | 13.3 HIGH | 0.4 LOW | 21.6 EXTREME |
“Unfamiliar” Projects | Range within 1 Standard Deviation (~68% of Data) | Range within 2 Standard Deviations (~95% of Data) | ||
---|---|---|---|---|
Project 4A—Using Recycled Water as Potable | 7.3 MEDIUM | 20.2 EXTREME | 3.2 LOW | 29.0 EXTREME |
Project 4B—Using Radiation in Treatment of Drinking Water | 6.6 MEDIUM | 18.7 EXTREME | 2.8 LOW | 27.1 EXTREME |
Project 4C—Removing Fluoride from Drinking Water Supply | 4.2 LOW | 16.1 HIGH | 1.2 LOW | 25.0 EXTREME |
© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Kosovac, A.; Davidson, B.; Malano, H. Are We Objective? A Study into the Effectiveness of Risk Measurement in the Water Industry. Sustainability 2019, 11, 1279. https://doi.org/10.3390/su11051279
Kosovac A, Davidson B, Malano H. Are We Objective? A Study into the Effectiveness of Risk Measurement in the Water Industry. Sustainability. 2019; 11(5):1279. https://doi.org/10.3390/su11051279
Chicago/Turabian StyleKosovac, Anna, Brian Davidson, and Hector Malano. 2019. "Are We Objective? A Study into the Effectiveness of Risk Measurement in the Water Industry" Sustainability 11, no. 5: 1279. https://doi.org/10.3390/su11051279