Improving Risk Assessments by Sanitary Inspection for Small Drinking-Water Supplies—Qualitative Evidence

Abstract

Small drinking-water supplies face particular challenges in terms of their management. Being vulnerable to contamination but often not monitored regularly nor well-maintained, small drinking-water supplies may pose consequences for health of users. Sanitary inspection (SI) is a risk assessment tool to identify and manage observable conditions of the water supply technology or circumstances in the catchment area that may favour certain hazardous events and introduce hazards which may become a risk to health. This qualitative research aimed to identify the strengths and weaknesses of the SI tool as published by the World Health Organisation to inform a review and update of the forms and to improve their robustness. The study identified a number of benefits of the approach, such as its simplicity and ease of use. Challenges were also identified, such as potential for inconsistencies in perception of risk between inspectors, in interpreting questions, and lack of follow-up action. The authors recommend a revision of the existing SI forms to address the identified challenges and development of complementary advice on possible remedial action to address identified risk factors and on basic operations and maintenance.

1. Introduction

Users of small drinking-water supplies face many challenges. Many of these supplies are located in rural areas and, frequently, owners neither have the technical knowledge nor the financial support to maintain them. It is reported by the World Health Organization (WHO) [1,2], independent researchers [3], and the European Union [4] that many small drinking-water supplies are not regularly monitored and inspected or do not deliver drinking-water that is compliant with quality standards. Water quality monitoring activities may be limited in many rural regions, particularly in those where household, local, or non-piped supplies prevail [1]. Small supplies are also vulnerable to inadequate management and breakdown. This can lead to unsafe services or insufficient quantities of drinking-water, which may have consequences for health [1,5]. As evidenced by the WHO [1], surveillance control mechanisms are particularly weak for small drinking-water supplies in many countries. Appropriate tools are needed to support such settings.

Risk-based drinking-water quality surveillance is a fundamental activity for the protection of public health [6]. Hazards (something which is capable of causing harm) and risks (the likelihood that that harm will occur) to health can be identified throughout the supply chain of the water from points of abstraction to final consumption, using a risk assessment and management approach, as advocated by the WHO [7]. Techniques are then used to decide the level of risk and what level of risk is acceptable [8,9]. Risk assessment considers the adequacy of controls in place to prevent contamination of water supplies which may be engineered (e.g., water treatment) or non-engineered measures (e.g., catchment protection, hygiene protocols for repair works on the water distribution).

One of the advantages of robust risk assessments is that they can identify critical risks to water supply in a specific setting and thereby complement water quality monitoring as required by standard-setting approaches [10]. Although risk assessment and management, as an integral part of the WHO-recommended water safety plan (WSP) approach, is suitable for both small and large drinking-water supplies, sanitary inspections (SIs) provide a simple method suitable for small drinking-water supply. The approach supports the identification and management of observable conditions or circumstances that may favour certain hazardous events and introduce hazards which may become a risk to health [7]. Published studies show that SIs have a number of additional applications, such as a surveillance tool [11,12], for operational purposes [13,14], as a component of a WSP [15], in research [16,17,18,19,20,21,22,23,24,25,26], and in outbreak investigation [27,28].

Enforcing standards through compliance-based approaches and undertaking risk assessments in line with recommendations of WHO Guidelines for Drinking-Water Quality [7] are complementary. Both approaches have benefits: standard setting ensures a uniform approach to achieving safe drinking water. A risk-based approach is site specific and holistic by addressing the whole operation, identifying hazards, and making judgments about the likelihood of the risk occurring and its potential severity. Through this process of systematic risk assessment and prioritization, the most significant hazards within the water supply may be identified and managed as the priority.

Risk assessment can vary in complexity reflecting the nature and size of the undertaking. It requires those managing small supplies to think about the likelihood of a hazardous event happening and take steps to avoid hazardous outcomes. This risk assessment process guides decisions where it is best to spend limited resources to get the best health return on investment, which is particularly important for resource-limited small supplies.

SIs were designed to allow persons with limited technical knowledge to use them with basic training. This has the additional benefit of empowering those who may not have technical expertise to engage in water management [29]. Where a standard form is used, SIs can also be used by technically trained people to support surveillance, at low cost, relying on observations rather than costly laboratory testing [16].

During the risk assessment process, difficulties may arise where uncertainties around the detail or level of risk exist, and it is difficult to evaluate the likelihood of it arising [30,31]. In order to make risk assessments as robust as possible, key questions that may need addressing are, how good should the information be on which the assessment is based? How should risks be prioritised and controlled? Should risk only be evaluated by experts or should lay opinion be permitted? How should the perception of risk be incorporated?

Sustainable Development Goal 6, target 6.1, seeks to secure safe and affordable drinking water for all. Therefore, a focus on the management of small supplies, which experience significant challenges, must be maintained to reach this goal and is the emphasis of this paper. SIs have been used since 1997 and with the revision of the WHO Guidelines for Drinking-Water Quality, volume 3, Surveillance and Control of Community Supplies [32] underway it is timely to review the strengths and weaknesses of the forms. This is required in order to inform a review and update those forms to maximize their potential and ensure the approach is as robust as possible to protect public health, to provide greater alignment with the WSP approach, to ensure appropriate technology options are available, and to provide best practice technical and management advice.

2. Materials and Methods

Twenty-six water professionals from 19 countries of varying socio-economic status were identified and invited to participate in a semi-structured interview to identify the benefits and challenges associated with SI implementation. The professionals came from a mixture of government departments, nongovernmental organisations (NGOs), private consultants, and academia and were selected for their field experience of using SIs themselves or in the context of capacity building.

Semi-structured interviews were chosen as the method to employ as it allows a discussion between researcher and participant, guided by a flexible interview protocol and supplemented by follow-up questions, queries, and comments [33]. Open-ended data was collected, to explore participant thoughts, experiences and beliefs about SIs. The interview protocol was developed following a review of published and grey literature relating to experiences using SIs, the adaptation of the SI forms from volume 3 of the WHO Guidelines for Drinking-Water Quality and the use of SIs as part of WSPs. From that review and discussion amongst the research team 11 broad questions were identified to form the frame of the interviews. No distinction was made between small and large drinking-water supplies in the questions; however, responses were primarily related to small drinking-water supplies, which is the focus of this paper.

All interviewees who were approached agreed to participate. Interviewees were sent a copy of the SI forms published by the WHO [32]. Written consent to participate and have the interview audio recorded was obtained prior to the interview. Interviews were conducted by a researcher over the telephone within six weeks of agreement to participate. All interviews were recorded and transcribed. As described by Braun and Clarke [34], themes were identified from the interviews during a review of the transcripts. These were application; use, ease of use; applicability in different settings; wider application in risk management; follow up action and continuous review; level of knowledge required to complete the SI; interpretation of questions; weighting of risks; adaptability of forms; and completeness of evaluation.

Responses were anonymised by the interviewer so that the data analysts were not biased by knowledge of the interviewee’s background.

The transcripts were analysed by two independent researchers who categorised the responses into the identified themes. Key quotations from each respondent were selected to illustrate the themes and are presented with the sector they were from at the time of interview given in parenthesis. This form of thematic analysis was selected as it helps to compare responses when respondents have different understandings of the same issue [34]. The project was given a favourable ethical opinion by the University of Surrey Research Ethics Committee.

3. Results

3.1. Application

The geographical spread of use of the SI forms by respondents spanned a range of low-, middle-, and high-income countries (

Table 1. Geographical spread of use of sanitary inspection forms and number of participants shown in parentheses.

Continent	Country and Number of Participants
Africa	Angola (1), Ethiopia (1), Kenya (2), Malawi (1), Mozambique (1), Nigeria (1), Sierra Leone (1), Uganda (4), Zambia (1), Zimbabwe (2), East Africa general (1)
Americas	Canada (1), Colombia (1), Ecuador (1), Nicaragua (1), Peru (1), Central America general (1), Latin America general (1)
Asia	Bangladesh (4), Bhutan (1), China (1), India (1), Kyrgyzstan (1), Maldives (1), Myanmar (1), Nepal (4), Pakistan (1), Sri Lanka (1), Tajikistan (1), Thailand (1), South Asia general (2), Central Asia general (1), East Asia general (1)
Australasia	Australia (3), New Zealand (1)
Europe	Belarus (1), Georgia (1), Ireland (1), Serbia (1), Spain (1), Ukraine (2), United Kingdom (2)

.)

3.2. Use

Table 2. Context of use of sanitary inspections by interviewees.

Purpose that Sanitary Inspection Form Was Used for	Number of Participants Responding
Systematic identification of risks and control measures to water supplies	26
Planned and unplanned surveillance	6
Regulatory aspects	4
Water supply management/assessment of supplies	4
Project work	4
Prior to WSP activity/integration activity	3
Engagement and empowerment of those involved in water management	2
Pilot trials	2
Capacity building	2
Research	2
Increasing awareness and understanding	2
Outbreak investigation/emergency water, sanitation and hygiene (WASH) services	2

shows the variety of uses that SIs have been used for, as identified by interviewees.

All respondents identified the use of SI forms as beneficial. They were recognised as providing a practical and structured way to assess likely hazards to a water supply by identifying conditions that may trigger contamination. Interviewees identified the systematic approach of SI forms as a method to highlight and overcome hazards which may have previously been overlooked or discounted (‘Familiarity makes many people not realise the risks that are staring them in the face until they go through a structured assessment’ (government)) and rated them are useful as they ‘indicate the point of contamination entry and risk factors’ (government) and ‘help the community to identify intervention control measures that are required to stop contamination’ (government).

The forms were also noted as useful in helping to broaden understanding of the water supply by the users and managers of the supply and to assist with the systematic prioritisation of hazards and remedial action, especially when a large number of hazards are identified and water quality is also monitored. One respondent noted, ‘When SI forms are used in combination with water quality tests, a good insight into the problem may be gained’ (government).

Others discussed the use of SIs to compare results spatially—‘If similar forms are used it is possible to get comparable results across a region allowing prioritisation of action if certain risk factors keep arising’ (government)—and temporally—‘ SIs can be highly supportive in providing a systematic approach to understanding the status and trends (of substantial numbers of individual systems) and to assist in identifying means for improvement’ (academia). A key strength of SIs is the proactive nature of the exercise: ‘[They] provide information on the precursors to risks which may result in actual risks occurring’ (government).

Using SIs as a tool to engage and empower those involved in water management is especially relevant in community-managed water supplies and was identified as being beneficial as community members can engage in water management and take action immediately to rectify issues. One interviewee noted that SI forms ‘broaden the engagement of the local people with respect to water quality. They see how critical the water supply is when they see people paying attention to it’ (WASH consultant).

One respondent highlighted the importance of the forms for use by government and ministerial departments as a means to stimulate remedial action: SI forms ‘provide a tool that can generate comparable and valuable information in settings where that may not otherwise be possible’. The tool was ‘fundamentally empowering to ministries’ (academia).

One interviewee discussed a ‘weakening of the role of environmental health officers in many locations as a result of the separation of the health sector from the environmental health sector’ (academia). Due to this weakening in the sanitarian’s role, the SI tool has also been undermined. This trend is being reversed in many countries, especially in urban areas with the advent of WSPs; however, it is still to be reversed in other settings.

SI forms have been used widely for research—in particular, to try and correlate SI scores with water quality [33,35,36]. However, as expressed by interviewees, this is not what the SI forms were originally intended for and there is inconclusive evidence whether SI forms can reliably predict presence of faecal indicator bacteria [37]. Both positive and negative aspects of this relationship were discussed. On the one hand, interviewees identified SI forms as a fast and easy method compared to water quality testing, as long as there is an awareness of the need to ensure the forms are not a substitute for water quality monitoring. Others raised concerns that false interpretation of water quality data is a risk, but the forms can enable more insight from the testing and helps appropriate interpretation of data, or as an interviewee noted, ‘SI can serve as an entry point to get more value from faecal indicator bacteria testing and to enable its effective interpretation’ (academia). Additionally, one interviewee noted in many ways SI forms are more useful than water quality testing—‘In some countries we have not been able to do water quality testing for many years testing is ad-hoc. SI forms bring the potential for local engagement and local capacity, like a WSP’ (WASH consultant).

The potential for SI forms to be used as a tool during outbreak investigation to help identify causal factors was noted by one interviewee: ‘There is potential for SI’s to help us to understand the causal factors in the water system that led to a waterborne disease outbreak’ (academia).

Application in WSP

All but seven respondents stated that they had used SI forms in some way alongside WSPs. Although SI forms do not assign risks or look at the effectiveness of control measures, they were noted by respondents as being the first step in a WSP in that they can help to identify hazards and hazardous events. Respondents noted that ‘you cannot do a WSP without SI, it is a fundamental element’ (water supply operator) and that ‘they [SIs] have a natural home in the WSP sequence’ (academia). Some respondents saw SI forms as embedded into the WSP framework although there was mixture of experience in using the two in conjunction. Some interviewees noted that although they themselves may not have used SI forms in the WSP process, the potential is there. Others noted that linking SI forms directly to the WSP was a challenge. In particular, clarification was needed on where the SI should sit within the WSP process.

In the past, many legal jurisdictions have tended to focus more on ‘regulations to focus analytical results rather than system safety; that has undermined the credibility of SI forms’ (academia). This is now changing though with the wider use of WSPs.

3.3. Ease of Use

A fifth of respondents noted the simplicity and ease of use of the SI forms as being beneficial: ‘Forms are simple and easy to use–very quick to give the potential user a good idea of the potential risk’ (government).

The ease of use of the forms was noted to enable non-technical persons, to help identify intervention control measures, and take action promptly, which may stop further contamination of water supplies—SI forms make it ’possible to make changes rapidly’ (WASH consultant).

The simplicity also helps to discuss issues with non-water specialists—the concept of SI forms helps to ‘explain risks to water supply to non-water specialists’ (NGO). In the context of surveillance, the simplicity was noted as being able to provide a quick overview of the situation, which can be beneficial to gain a comparable overview of the status of a region to allow for the prioritization of action. In this context, a respondent explained that ‘when you have 300–400 systems, for practical reasons you will probably have to focus on the worst 10%. SI forms are useful for engineers and planners who are running programs’ (academia).

Some respondents noted that the forms need to be adapted for people of low literacy or with no access to the internet. The forms are ‘very powerful as a tool, especially if you can translate it into a pictorial form as an education tool to teach children the importance of looking after water supplies and the potential of how they might get sick from contamination of water supplies’ (NGO).

Challenges regarding the simplicity of the forms were also noted by respondents. The reasons for this were because simplicity leads to certain observations being missed or omits links which are often complex, and with regards to interpretation. SI forms ‘cannot possibly capture each hazard’ (government).

Four respondents referred to issues surrounding the paper-based nature of the forms in the field and the potential to digitize forms to assist in data collection and analysis.

Other challenges identified include the perception of risk which varies between individuals largely depending on historical experience: ‘Persuading people to accept that they have risks and overcoming the human reluctance to accept real risk is a challenge in itself. It may prove difficult to change the attitude of staff members who have performed monitoring in a certain way for much of their career’ (government).

Respondents highlighted the complexities of environmental systems in general, specifically groundwater systems, with regards to accurate completion of the forms: ‘When dealing with groundwater supplies, knowing how far to look for risks is difficult. In addition, many risks that present themselves are very much mediated by issues such as rainfall. It is important to understand the effects of rainfall; looking at a risk on its own will not necessarily give you the full answer’ (government).

3.3.1. Level of Knowledge Required to Complete the SI

Although a strength of the SI was noted as the ability to be conducted by operators with varying levels of knowledge, there are also concerns with regards to the baseline knowledge an inspector requires in order to be able to do the SI thoroughly and, most importantly, to understand why they are doing it. Our interviews highlighted that ‘risk assessments in the form of SIs are a powerful tool that have brought huge benefits to the water community not only in terms of identifying risks to water supplies but also engaging and empowering communities in water management’ as they engage users who may not possess the applicable tools to participate in water management. However, there is a level of concern surrounding the lack of training or baseline knowledge of people completing the forms. Three respondents discussed training in a positive light, one noting that ‘the benefit of SI is that you can explain risks to water supply to non-water specialists—particularly in the absence of major analytical capacity, for example where there is no access to labs for regular or routine testing’ (NGO). Three questioned the requirement for training with regards to needing skilled or trained personnel to manage a water supply, or a baseline training, which can be hard to access in a rural context and interpretation of the forms can be hard for lay people to interpret: ‘SIs require knowledge’ (NGO). If the relevance of the questions is not understood, then inspectors are unlikely to act upon improvements needed. As two respondents commented, ‘Really, we need to understand the nature of the water resource from which the water is being drawn; people tend to focus on the question ‘is there a latrine 10 m from the well?’ Especially in urban areas, it is actually probably the density of the latrines which is more important’ (NGO). ‘It is challenging if an inspector could not think of the questions in the given supply context’ (NGO).

As indicated by four respondents, SIs are rarely comprehensive and there is ‘an intellectual tendency to want to try to catalogue every potential risk’ (academia). Ensuring that the whole catchment is surveyed is also a challenge, especially ‘in piped systems where the source may be some distance from the community. There is always the temptation for the operative not to do that long walk to actually look at everything, so there is a need for oversight and effective training in the conduct of SIs’ (academia). However, as one respondent noted, SI forms can be a good starting point to ‘trigger thinking’ (WASH consultant) where there may be a lack of technical experience.

3.3.2. Interpretation of Questions

As previously discussed, respondents explicitly noted challenges related to interpretation of the questions in conjunction with the potential for inconsistencies in perception of risk between inspectors. Such varying perception may be in relation to a fault, infrastructure component or hazardous event. For example, is drainage poor? Is water puddling? How big does the puddle have to be to call it water puddling? Cracks on concrete floor—what defines a crack? When is a crack a problem which allows pathogens or chemicals to enter (e.g., superficial versus deep crack)?

The flexibility and adaptability of the SI is interpreted as a strength, but it can lead to inaccuracy if the structure of the question is not thought through or piloted. In particular, ‘there is a very strong tendency in the adaptation of WHO forms for people to introduce very subjective questions. It is extremely valuable to make questions as objective as possible both when the focus is on a specific system at hand and especially when one is trying to get an oversight in the case where the findings of one system are to be compared with the findings of other communities because different operators will interpret what they see in different ways—subjective objectivity’ (academia).

Linguistic challenges, including challenges relating to confusion of meaning when translating forms from English, were noted by two interviewees. One noted, ‘The phrasing of the questions is sometimes an issue for the target audience. For example, when is a ‘yes’ (in answering a SI question) a good or bad thing?’ (government). ‘It is difficult at a linguistic level that a “yes” will always be a risk. Sometimes, with the phrasing of the questions, you really have to think whether the answer is a “yes” or “no”’ (government).

3.4. Applicability in Different Settings

SI forms have been produced for a range of water supply types as identified in the WHO Guidelines [10]. Respondents highlighted that the standard SI forms focus on technologies which are managed by communities and not on complex, utility-managed systems. Traditionally, SI forms were aimed at rural settings; however, some respondents noted the forms as increasingly being used in peri-urban and urban settings, too.

The interviewees raised the concern that SI forms presented in WHO Guidelines [32] do not provide information on the functionality and level of performance of a system. Rather, the number of potential hazards and where they lie are identified with no indication of whether the system as a whole works well or not.

3.5. Follow-Up Action and Continuous Review

In order to take appropriate remedial action, accurate interpretation of results is essential. Six respondents noted challenges surrounding accuracy of interpretation of results.

Post-inspection remedial action was raised by three respondents. One noted that a benefit of the form is that it can be used to identify simple, remedial action but two interviewees raised the lack of follow-up action as a challenge. In particular, ‘With regards to enforcement, reports are sometimes prepared and not acted upon. Risks are not always addressed in terms of management of supplies. This may be due to land ownership or access issues’ (government). In order to address this, SI forms ‘need to be embedded with recommendations for follow-up action, not just identify risks without mitigating them’ (government).

Available capacity and commitment or logistics to carrying out SIs regularly was also raised as a concern. One respondent stated that although the SI forms can be used by governments to identify where resources need to be focused; due to resource limitations, remedial action is often directed towards urban areas, whereas the greatest added value of a SI would be in rural areas. Financial constraints limit how many people and frequently they can go into the field. The SI process is ‘very much challenged by the limited mobility of teams and regularity of inspections’ (WASH consultant). It is ‘challenging to roll SI out as part of a water surveillance program and keep it going, and very difficult to do water quality testing in unison with SIs in developing countries’ (WASH consultant). One respondent reported the converse—that the SI process is ‘very suitable for rural or small communities where a high level of resources are not available and where the system is very simple’ (government).

Continuous review of the SI process could also relate to a review of the format of the form itself. One respondent noted, ‘Most countries are still photocopying pages from the old 1997 WHO Guidelines for SIs and using them verbatim even though technology has moved on and been adapted’ (NGO).

3.6. Weighting of Risks

An area identified by four respondents was around the issue of weighting risks. As a result of the standard ‘yes/no’ method of risk scoring employed in volume 3 of the WHO Guidelines [32], it is assumed that each risk has equal weighting and therefore an equal potential to cause contamination, which in practice may not be the case. There is ‘no variation in weighting [which] gives us a problem with prioritizing the mitigation tasks, especially when several risks [are] identified’ (government).

The question of standardization with regards to whether one risk has greater importance than others was noted by several respondents. One respondent commented that standardization of the interpretation of the results of SIs is important ‘in order to be accepted by regulators or by national authorities’ (NGO).

The use of SI forms as part of the WSP process was identified by a number of respondents as a method of overcoming this issue: ‘SI assumes every risk has an equal value—WSP looks at every risk in isolation and assigns a high, medium or low risk’ (NGO).

3.7. Adaptability of Forms

The WHO Guidelines [7] explicitly say that the SI forms should be adapted for local circumstances and evidence was provided for this by our respondents who had adapted the forms to take into account local conditions and context. One respondent said, ‘The most important [adaptation] is wording due to cultural translation and specificity. When you pilot test the wording people want, it is often highly subjective and then you need to explore how to make it more objective’ (academia).

Respondents indicated that the main reasons for adapting the forms were primarily to fit in with differing cultural contexts, i.e., translated to local language and reworded to ensure they are still relevant to the setting or with the addition/removal of certain questions. Questions may need to be adapted quite significantly due to different structures and settings of small drinking-water supplies found in different countries. This was closely linked with the adaptation ability of the forms. Respondents noted the capacity for the forms to be adapted: ‘Forms were adapted so further questions or suggestions could be proposed’ (NGO). ‘Questions have been added, for example about the facility and vicinity of risk factor’ (government). Questions were removed as necessary: ‘We used forms that were relevant (e.g., rainwater was not) and also added some forms that were not in volume 3 [of the WHO Guidelines], such as household storage and public tap’ (government). In other cases, questions were added: ‘We used a bigger set of in total 17–18 questions and phrased them a little more detailed again to be adapted to context’ (government). ‘Sometimes a question is eliminated or added. For example, where chlorination is not practiced, the question is removed. Sometimes a question is added for specific hazards’ (academia). In other cases, the form was adapted into local language or pictures: ‘It is overwhelmingly the outcome of the piloting to make the questions make sense in the local setting’ (academia). Doing this not only led to greater success of the forms but also engaged a wider audience.

A benefit of the form noted is the consistency of approach, even if they have been adapted. One respondent noted that one of the challenges of adapting the forms is in ensuring that gaps are not created which may lead to risks being missed. Seven respondents discussed challenges around adaptation of forms including that forms ‘cannot be everything to everyone’ (NGO) and that the lack of standard forms ‘can make comparisons difficult’ (water supply operator).

3.8. Completeness of Evaluation

Interviewees were divided when asked whether they felt any potential risk factors had been omitted from the forms. While nearly half (44%) stated that the forms were complete, others suggested more emphasis on aspects such as risks around priming of tube wells; impact of storm water runoff, silt build up from construction work, land use and catchment changes, abandoned boreholes in which there are short circuits into groundwater, water quantity aspects as they may impact water quality, and management and maintenance of water supplies more in line with the WSP approach.

4. Discussion

4.1. Application and Use

Results show that SI forms are largely being utilized globally for the purpose in which they were intended, for example to provide a rapid means of identifying and assessing conditions and circumstances that may introduce risks to a water-supply system, including inadequacies and lack of integrity in the system that could lead to contamination [32]. The focus is primarily on small drinking-water supplies, in particular rural communities, as well as surveillance agencies that work with such communities. Our interviews confirmed that the scope of use of SI forms has expanded to urban and larger utility type contexts in some cases; however, literature supporting this is limited.

The current breadth of settings in which SI forms can be used is identified as a benefit of the forms; they can be applied along a water-supply chain from source to consumer in line with the WSP approach. Indeed, a more complex form of SI is useful in larger water supply settings as a part of the WSP process. Further considerations around how to optimize this broader application and whether this is appropriate may be required.

The positive responses surrounding the relationship between the use of SI forms and the WSP process is encouraging for the further integration of the two risk management strategies. Only in countries in which WSPs had not been rolled out at the time of the survey, respondents stated that SI forms had not been used in the WSP process. The use of SI forms in the hazard analysis stage was a promising indicator that WSPs are not being seen to replace SI forms but embed them as a practical tool to inform system assessment, as for example promoted by the WSP field guide of the WHO Regional Office for Europe [38]. This needs to be encouraged further and, as stated by a number of respondents, clearly documented and illustrated to better encourage and develop both SI and WSP practices in unison. The processes complement each other and should evolve to continue to do so. In certain settings with low capacity, SIs may be viewed as the first step on a risk assessment/management continuum and applied as a basic WSP in essence. As experience, capacity, and resources permit, water supply managers may be encouraged to incrementally move from SIs toward full WSPs, tailored to the local context, where SIs can continue to play an important WSP-supporting role for hazard identification and verification activities.

A number of interviewees identified ‘missing risks’ in the SI process to be around management, including hygiene management, and maintenance of the water supply. The WSP practice is more encompassing of behavioural and management practices of water safety than the traditional SI process. Further development and promotion of the relationships between SI forms and WSPs should assist in tackling the risks around management and maintenance of supplies.

4.2. Ease of Use

There are clear strengths associated with the SI process, particularly as a quick and easy to use tool to identify observable conditions or circumstances that may favour certain hazardous events to happen, introducing hazards which may become a risk to health and require control measures to prevent water quality contamination. For almost each strength identified by an interviewee, a corresponding challenge was noted. For example, while the simplicity of SI forms eases their use, the associated challenge is to ensure risk factors are not oversimplified. This interlinking of benefits and challenges indicates that a certain level of technical expertise is required at some point during the inspection and assessment process; whether that is in the adaption of a regional or national form, the interpretation of questions in the forms, or in the analysis of inspection findings.

Risk management approaches are deemed to be a proactive method of managing water supplies. To ensure this, however, the people carrying out the assessment of the water supply and interpreting the findings must be knowledgeable of the local supply context and of what conditions are presented to them on-site and not limit their attention to the questions on the SI form.

Furthermore, the undertaking of an SI by an operative with little or no experience could actually be counterproductive toward ensuring drinking-water safety. There is clearly a need for supporting information to be attached to the questions to provide knowledge as the relevance of the questions. These concerns imply there is a need for effective training in the conduct of SIs and comprehensive explanatory notes to accompany the inspection form. The WHO Guidelines [7] suggest that community members/supply operators learn how to conduct the inspection independently where official visits by the surveillance officer are infrequent. This places great emphasis not only on the importance of objective questions phrased in a simple and robust way but also on the importance of the scientific validity of each question and a minimum level of technical knowledge needed by the inspectors in order to draw valid conclusions from the inspection result and define appropriate improvement interventions.

A key objective of risk assessment is to improve management and operation of water supplies, including the protection of its source. However, this is not the only benefit. Due to the diverse ways in which the findings of SI forms can be utilized, they may be considered not only within the context of a single supply, but also within the context of understanding the broader situation in an area, such as the identification of recurrent failures, inappropriate practices (i.e., in design, construction, operation, maintenance) and the assessment of system level responses to these failures. That knowledge can be used to inform wider improvement programming. Clarity of the settings in which the forms intended purpose is essential, as are associated guidance, training and explanatory documents.

4.3. Improved Use of Technology in the Digital Age

Respondents raised the issue of the use of technology in the data collection and form filling process. Although this is common in many locations now, it is acknowledged that such technology is not always available. Especially in remote, rural locations, additional human and financial focus is required to achieve this. Benefits of using technology were noted by interviewees as reducing the risk of forms being left on the shelf, improving the data processing reliability, especially in times of inclement weather, to ensure questions are not missed and to improve the chances of follow up surveys and action. The use of technology also provides an efficient means to review SI outcomes and an easier way to identify trends over time and prioritize both site-specific action and interventions in rural water supply as part of broader improvement programming. The use of technology has been found to increase the speed of reporting and data quality [39]. Further, the use of mobile application platforms provides greater flexibility and scope for local/regional adaptation of the SI forms than hard copy paper versions (e.g., through the use of question logic flow, where a ‘yes’ answer can prompt additional more detailed questions which may support determination of the level of risk). Mobile phones have been identified as assisting with improving the quality and consistency of surveillance data [40]. It is important to note that any technology used must be applicable, appropriate and sustainable for the individual setting.

4.4. Adaptability of Forms

Evidence from interviews and supported by published literature show that in most cases SI forms are adapted from WHO [32] but many different formats of SI forms are in circulation which take into account factors such as the knowledge base of intended users; the use of SIs as part of WSPs; and the differing contexts in which the inspections were undertaken [15,17].

4.5. Completeness of Evaluation

One area missing from the SI forms identified was to do with the functionality of a water supply. This may have an influence on water quality, as well as quantitative service indicators, such as coverage, cost, continuity, and quantity—all of which should be taken into account when assessing a supply. Deficiencies in any of these service indicators may lead to consumers using drinking water from unimproved supplies or undertaking unhygienic storage procedures at a household level.

When a trend in failures within water supplies is noticed, the SI form provides a tool which allows the operative to work out the cause and how to address it [11,12,41,42]. This allows the identification and prioritization of supplies within a geographical area, which in turn may be used to allow water supply managers and surveillance agencies to prioritize remedial measures within their programs. It may also direct management attention to the most problematic issues in a given supply or region. Prioritization of risk and therefore remedial actions to be taken is identified as a strength of the SI tool.

4.6. Interpretation of Questions

There were several discussions with respondents around the issue of subjectivity throughout the interviews. In answering the SI questions, inspectors may apply different interpretations of the questions and thus the answers may be prone to subjectivity. This suggests there is strong argument to revise the questions and provide support materials for each SI question to ensure consistency in interpretation and reduce misinterpretation and subjectivity, in particular for circumstances where local expertise is not easily available of insufficient. Uncertainties related to the interpretation of questions may result in the operative who performs the SI, making (informed or uninformed) assumptions, which in turn may lead to inaccurate reporting of faults. This is particularly pertinent when the operative does not have the knowledge base to apply any interpretation to the questions or the perception of risk which may occur from a fault, infrastructure component or hazardous event.

Additionally, respondents raised concerns around complexities of supplies, for example, groundwater supplies and the various proximal and distant influences on these supplies which must be considered. For example, the risk associated with a latrine being located a certain distance from a supply varies depending on the local geology and hydrogeology. Even if forms are adapted on a regional or national level, it is difficult to take account of these localized variations, which may result in the risk level outcome being higher or lower than it should be. Therefore, as noted by several respondents, SI forms should favour the use of people’s experience and local knowledge over scientific validity in statements; however, this is then dependent on local knowledge being available and reliable. Solid explanatory notes should come with each SI question to minimise the risk of misinterpretation and subjectivity, in particular for circumstances where local expertise is not easily available or insufficient.

With regards to interpretation and perception of risk, challenges associated with interpretation of risk level, or the results from an individual SI form, was noted by the interviewees. Interpretation of risk can be highly dependent on the level of technical expertise of the assessor, as well as the clarity of the question and associated supporting documentation. Risk interpretation can therefore be inferred and be subjective depending on the level of experience or technical knowledge of a specific hazard or water supply.

4.7. Weighting of Risks

Forms do not allow for the classification of the level of risk posed by each identified failure or problem that may introduce a hazard to the water supply. If the level of risk is not estimated, it is difficult to prioritize which issue to tackle first where human and financial resources are limited. Respondents discussed the need for risk levels to be identified, one noted the use of presence/absence indicators to reduce the subjectivity of risk.

Other respondents suggested the inclusion of the option to classify the risk level (e.g., assign low-, medium-, and high-risk) found for each individual SI form question (i.e., risk factor). This classification, however, would introduce an additional problem. Perception of risk can be highly variable between individuals. Risk perception is defined as a cognitive ability to determine the risk inherent in a situation, involving an accurate appraisal of the external situation and one’s personal capacities [43]. Underestimating the risk in the context of drinking-water supply could lead to illness amongst consumers. Overestimating the risk can also have considerable consequences, possibly resulting in misallocation of scarce resources or consumers seeking an alternative water supply which may be further to access and of poorer quality.

A key issue with the SI form is that it is apparent that people estimate risk in very different ways. One study has concluded from another sector that amongst flight crews, low and medium risk events were overestimated [44]. Other studies concluded that risk perception differs as an effect of cultural, environmental, and governmental influences [45,46]. In our own context, experience in conducting risk assessments, as well as using the water supply on a regular basis will likely influence the results.

In order to further support risk management of drinking-water supplies in general, quantitative tools for risk analysis are needed [47]. An important consideration is that although the level of risk can be rigid or fixed in a given context, it is the perception of the level of risk, acceptance, and an understanding of the resulting consequences that are subject to variation between individuals [48,49]. Quantification of the risk facilitates, for example, comparison with other risks and acceptable levels of risk in absolute terms as well as quantitative estimations of the efficiency of risk-reduction options. Perception of risk could determine whether or not a successful and safe outcome will result for an intended action. In situations where there is a high-consequence, for example in the case of drinking-water becoming unsafe, failure in addressing a crucial factor has the potential to result in the large-scale illness.

Even among professionals, perception of risk may vary; yet a key feature of the SI is that it is intended to be used by lay persons where technical human and financial resources are lacking. If trained professionals are not producing consistent results then there is a real danger that lay persons are missing risks to the water supply, they are assessing, potentially exposing the users to health risks.

The introduction of risk ranking for each individual SI form question (or risk factor) would likely require corresponding technical guidance and training materials, including for example operational definitions and visual standards, to support consistent and objective risk ranking by trained inspectors in the field.

4.8. Follow-Up Action and Continuous Review

One of the key aims of SIs is to prompt action to improve water quality and protect human health. However, the lack of follow-up action in practice was highlighted by several interviewees. If the aim of SI is to improve water supplies, then this concerning assessment by interviewees must be addressed to understand the reasons behind why action is not being taken in response to the data. The interviews in this study were more focused on the forms themselves but a recommendation of the study is to strengthen the linkages between the SI results (i.e., data) and decision-making on necessary follow-up action and to provide practical guidance on appropriate follow-up actions that can be picked up by water supply operators and surveillance authorities to inform implementation of remedial interventions.

As noted previously, a certain degree of technical expertise is required in either the adaptation, completion or interpretation of the SI forms. While SI can be performed by people with limited resources or technical expertise, caution needs to be applied to ensure that important observations and potential hazards are not being missed, skewed or interpreted incorrectly. Incorrect completion of the form could lead to incorrect interpretation and therefore inappropriate remedial actions (and therefore wasted resources in resource-limited settings), deeming the whole process ineffective.

Appropriate training is an essential component of programs stipulating or promoting sanitary inspection. It may address understanding of SI questions, the use of forms in the field, interpretation of results, or how to adapt the form to local context, including how to translate forms from English into local language to ensure the original meaning of questions is conserved. Training must be adapted to the context in which SI forms are being used, targeting the right people at the right level (i.e., engaging the relevant stakeholders), offering follow-up and backstopping and capacitating trainers to further disseminate knowledge in local context. It should be noted that WHO publication [32] has a diagram associated with the form for people with low literacy.

5. Conclusions

The objective of the study was to identify the benefits and challenges of SI as a risk assessment and management tool. The results show that the process undoubtedly has several benefits provided that the inspections are conducted and interpreted correctly. The process is simple to use and adaptable. However, the study also highlighted some challenges. As one of the interviewees from academia pointed out, ‘Safety requires that not only water is safe but that measures are in place to keep it safe’. The SI forms are currently being updated to make them more robust, reflect appropriate technologies alongside current best practice technical and management advice, and to better align with WSPs. This paper has highlighted some of the issues that need consideration in that process, alongside some that require further investigation. We recommend that questions are revised to ensure consistency in interpretation by inspectors. The provision of support materials for each question will aid this—in particular, targeted technical guidance for each SI question to facilitate effective and consistent completion of the sanitary inspection. Complementary information and advice on possible remedial action to address identified risk factors and on basic operations and maintenance would further promote appropriate follow-up action and strengthen the overall outcomes from the approach.