**Stakeholders' Perspectives for the Development of a Point-of-Care Diagnostics Curriculum in Rural Primary Clinics in South Africa—Nominal Group Technique**

#### **Nkosinothando Chamane 1,\*, Desmond Kuupiel <sup>1</sup> and Tivani Phosa Mashamba-Thompson 1,2**


Received: 6 December 2019; Accepted: 7 January 2020; Published: 1 April 2020

**Abstract:** Poor knowledge and adherence to point-of-care (POC) HIV testing standards have been reported in rural KwaZulu-Natal (KZN), a high HIV prevalent setting. Improving compliance to HIV testing standards is critical, particularly during the gradual phasing out of lay counsellor providers and the shifting of HIV testing and counselling duties to professional nurses. The main objective of this study was to identify priority areas for development of POC diagnostics curriculum to improve competence and adherence to POC diagnostics quality standards for primary healthcare (PHC) nurses in rural South Africa. Method: PHC clinic stakeholders were invited to participate in a co-creation workshop. Participants were purposely sampled from each of the 11 KwaZulu-Natal Districts. Through the Nominal Group Technique (NGT), participants identified training related challenges concerning delivery of quality point of care diagnostics and ranked them from highest to lowest priority. An importance ranking score (scale 1–5) was calculated for each of the identified challenges. Results: Study participants included three PHC professional nurses, one TB professional nurse, one HIV lay councilor, one TB assistant and three POC diagnostics researchers, aged 23–50. Participants identified ten POC diagnostics related challenges. Amongst the highest ranked challenges were the following:absence of POC testing Curriculum for nurses, absence of training of staff on HIV testing and counselling as lay counsellor providers are gradually being phased out,. absence of Continuous Professional Development opportunities and lack of Staff involvement in POC Management programs. Conclusion: Key stakeholders perceived training of PHC nurses as the highest priority for the delivery of quality POC diagnostic testing at PHC level. We recommend continual collaboration among all POC diagnostics stakeholders in the development of an accessible curriculum to improve providers' competence and ensure sustainable quality delivery of POC diagnostic services in rural PHC clinics.

**Keywords:** quality HIV point-of-care-diagnostics; nominal group technique; stakeholder engagement

#### **1. Introduction**

Point-of-care (POC) diagnostic testing is defined as timely clinical testing performed during patient consultation when the result will be used to take appropriate action, which will lead to an improved health outcome [1]. POC diagnostic testing plays a critical role in healthcare access in settings with limited laboratory infrastructure. Early diagnosis and rapid initiation of treatment is key to controlling infectious diseases including HIV/AIDS [1,2]. Improving the quality of diagnostic services is therefore essential for improved access to quality primary healthcare (PHC) services. In South Africa an HIV Counselling and Testing (HCT) campaign was first launched in April 2010. During this time in PHC clinics HIV/AIDS testing was done by lay counselors, who were permanently employed staff sent for training specific to performing HIV/AIDS rapid testing and counselling. Their role was solely to provide HIV testing and counselling services. In 2014 the South African Department of Health then announced the gradual phasing out of lay councilors and adding HIV testing duties to professional nurse consultation duties. This shift of duties may lead to added pressure on clinics who already suffer because of staff shortages. Moreover, poor knowledge and adherence to quality assurance as well as HIV testing standards remain a challenge especially in resource limited areas [3]. This is a concern because quality assurance programs, which include regular calibration of instruments, participation to external quality assessment schemes, adherence to standard operating procedures and test operator competency as well as running of internal quality control samples on every test day have been shown to ensure accuracy of POC diagnostic testing [4,5]. Factors contributing to this problem include poor access to laboratory infrastructure, training resources and institutions due to clinics being situated in deep rural areas, time constraints and lack of motivation to engage in new interventions [3,6,7].

Various initiatives have been introduced towards addressing this challenge. They include the Rapid HIV testing quality improvement initiative (RTQII) introduced to seven countries in 2013, including Tanzania, Ethiopia and Kenya [8]. The goal of RTQII was to scale up coverage of HIV rapid testing quality improvement (QI) and assurance activities as well as to improve the quality and safety of rapid testing services [8]. The RTQII findings emphasized provision of standard operating procedures (SOPs), onsite supervision, job aiders and orientation of providers on the importance of compliance to HIV rapid testing standards as the main tactics which led to quality improvement [9].

In this study, we involved PHC clinic-based POC diagnostics stakeholders in a co-creation workshop to identify priority areas for the development of point-of-care (POC) diagnostics curriculum to improve competence and adherence to POC diagnostics quality standards for PHC nurses in rural South Africa. The findings of this study have the potential to inform policy making concerning Continuous Professional Development (CPD) interventions aimed at improving the competency of PHC health workers on POC diagnostics services. This will further contribute towards healthcare systems strengthening, through improving and ensuring provision of quality, reliable and sustainable POC diagnostic services.

#### **2. Materials and Methods**

#### *2.1. Study Design*

The Nominal Group Technique (NGT) was employed to enable engagement with representative key stakeholders from 11 districts of KwaZulu Natal (KZN). We defined key stakeholders as PHC workers with experience in performing POC diagnostic services in rural KZN PHC clinics and researchers in the field of POC diagnostics. NGT is defined as a process to identify strategic problems and to develop appropriate and innovative interventions to address them [10]. The NGT processes is commonly applied to homogenous groups and it involves four main phases—(i) Nominal or silent phase, where participants individually consider their personal responses to a presented question and write them down; (ii) Item generation phase, where individual participants take turns to share their responses with the group. The items generated are recorded without being discussed; (iii) Discussion and clarification phase, where group members discuss and ask questions in order to clarify items on the list and elaborate on their responses. During this phase, items with similar meanings are combined and duplicate items can be removed; (iv) Voting phase, here each participant is asked to prioritize the listed items by assigning ranks to them. The ranking results are then collated to produce a single list of priorities for the wider group [11,12]. Application of this process in this study is discussed below.

#### 2.1.1. Study Participants

We invited key stakeholders of PHC-based POC diagnostics to participate in a Nominal group co-creation workshop. The NGT team comprised of three Professional nurses, one TB professional nurse, one TB assistant, one HIV/AIDS lay councilor, two experienced researchers, the primary researcher (as facilitator) and one research assistant. Detailed characteristics of participants are presented in the next section.

#### 2.1.2. Sampling Strategy

A purposeful sampling strategy was used to select representative clinics to participate in this study. This sampling technique involves identifying and selecting participants with practical knowledge and experience of PHC-based POC diagnostics. This study was conducted as a follow up to the cross-sectional survey of 100 randomly selected clinics in KZN rural PHC clinics [3,13]. The survey was aimed at demining the accessibility, availability and utility of POC diagnostic services in rural PHC clinics [13]. Clinics identified to have the highest availability and usage of POC diagnostics following the survey were selected to participate in this study. We thus included one PHC clinic from each of the 11 KZN districts with the highest availability and usage of POC tests. Clinics with low HIV POC diagnostics availability and usage were excluded due to minimal experience in HIV testing as reported in a previous audit [3].

#### 2.1.3. NGT Process

The PI (NC) facilitated the workshop with the help of a trained research assistant (PS). The four phase NGT was performed to achieve the objective of this study. Prior to the meeting of all key stakeholders, a pre-elicitation technique [14] was employed, where collaborators were sent an invitation together with a brief on the program of the day. The brief included the purpose to create a platform for key stakeholders to come together and determine training-related challenges affecting delivery of quality point of care diagnostics services in PHC clinics and to identify priority areas to be addressed in order to overcome the challenges. The main aim of the NGT was to bring together key stakeholders to identify priority areas for the development of a POC diagnostics curriculum.

At the opening of the session all participants were given an opportunity to introduce themselves, sharing their current positions as well as the number of years of experience in the field of HIV/AIDS testing. Following this, the PI (NC) provided a background to the workshop and presented the program of the day as illustrated in Figure 1. The participants were then separated into two sub-groups of four. The sub-groups were provided with a set of sticky notes, pens, markers and a flip chart sheet.

**Figure 1.** Nominal Group Technique (NGT) session to identify training related challenges with the delivery of quality point of care diagnostics.

The PI (NC) posed the following question to participants, to start the workshop: what training related challenges do you encounter with regards to the delivery of quality HIV/AIDS Rapid tests? The following steps were followed to answer this question:

#### 2.1.4. Silent Brainstorming

Participants were given up to 10 min to consider the question and note down all the relevant ideas that came to mind. Discussions were prohibited during this period, however the participants could raise their hands for the attention of the facilitator if in need of clarity on the above question.

#### 2.1.5. Group Discussion

Group members were given another 10 min to share their ideas within their groups, group them into themes as they emerged and then stick them on the flip chart sheet to be presented to the whole workshop group.

#### 2.1.6. Group Presentations and Clarification

Each sub-group selected one representative to present their ideas according to the themes they had agreed upon. The facilitator encouraged questions and discussions during the discussion sessions. This process was also used as an opportunity to probe the presenters for further explanations as well as for the wider team to discuss and clarify presented ideas. During this process the research assistant collated all the ideas and together with the facilitator highlighted similar themes and removed duplicates. The collated results were presented to the wider group as priority areas to be ranked during the ranking session.

#### 2.1.7. Ranking of Ideas

The ranking process followed the strategy suggested by Delbecq et al. [15] of ranking ideas through assigning a value to an idea according to its priority. Ranking is usually preferred by many researchers, because scores can be quickly tallied and the results can be interpreted and discussed within the same session [16]. Participants were given a break, while the facilitator through the use of an online form software (Google® Forms, Google LLC, Mountain View, CA, USA) with the help of a trained research assistant created a ranking questionnaire. The questionnaire consisted of 1l challenges presented by the two groups combined.

The questionnaire was handed to each participant for ranking ideas using a Likert scale of 1–5 scores with 1 representing very low priority and 5 representing highest priority. The ranking process was conducted independently and without discussion. The results were collated and analyzed using a spreadsheet as explained in the data analysis section below.

#### 2.1.8. Data Management

During the nominal group discussions, two types of data were collected; qualitative and quantitative data. Each type of data obtained was first managed individually and then the findings were combined to fully address the main aim of this study. Qualitative data was recorded in chart sheets as well as through a recorder to be analyzed at a later stage. Quantitative data obtained from the ranking tool was entered onto a google spreadsheet for further analysis as explained in the section below. This enabled us to immediately report the results back to the participants and more qualitative data was obtained during clarification to elaborate on the ranking data

#### *2.2. Data Analysis*

Two nominal groups consisting of four participants each were conducted. Data analysis was ongoing and an overview of the process followed is provided in Figure 2. This involved ranking of ideas and thematic analysis of the qualitative data. All stages are described in detail below, where analysis will be discussed in the context of three terms: ideas, priorities and themes [12].

**Figure 2.** Overview of the NGT Data analysis Process.

#### 2.2.1. Raw Data Analysis

Ideas raised by participants in the silent brainstorming of each nominal group were presented to the wider group. All the ideas were put up for participants to view simultaneously and the facilitator went on to facilitate grouping of all the ideas according to emerging themes agreed upon by the wider group. Similar ideas were grouped together and duplicates were removed. The themes then became priorities to be voted upon in the ranking stage.

#### 2.2.2. Quantitative Data Analysis

The most common technique to analyze and describe nominal group data is summing the votes allocated to each idea to determine the overall priority score [17–19]. This method was employed in this study, where quantitative data obtained from the participants ranking of ideas on a scale of 1–5 was analyzed through the summing of votes allocated to each idea. The overall priority score for each theme was then calculated. This was done through capturing ranking responses into google forms and calculating overall priority scores. A priority list of responses was then drawn and presented to the wider group.

#### 2.2.3. Qualitative Data Analysis

We conducted qualitative analysis of the first five highest overall priority scores. The recorded qualitative data were collected from the participant's presentations, where the rationale for selecting these themes was provided. More qualitative data collected during the discussion and clarification of the priority list was also utilized to elaborate more on the selected themes.

#### **3. Results**

#### *3.1. Characteristics of Study Participants*

In total the NGT team comprised of 8 participants from ages 23–50, with each group comprising of four participants. The attendance rate was 72% since 11 participants were expected. Reasons for nonattendance included other work commitments, inability to organise transportation, other appointments and protest actions on the road. All the stakeholders in attendance reported involvement in POC diagnostics and their specific roles are reported in Table 1.


**Table 1.** Key stakeholder characteristics and involvement in point-of-care (POC) diagnostic services.

#### *3.2. Nominal Group Ranking*

Stakeholders identified a combined total of 18 challenges (Appendix A), which were then categorised into 10 themes. Ranking results of these themes from highest priority to lowest are presented in Table 2.


**Table 2.** Ranking results in descending order.

#### *3.3. Thematic Analysis of Top Five Priorities*

The study was aimed at determining priority areas for development of a POC diagnostics curriculum for PHC nurses in rural South Africa. From the ten priority areas determined by key stakeholders during the workshop, the top five ranked priorities were: Absence of POC testing curriculum for nurses (90%); absence of training of staff on HIV testing and counselling as lay counsellors are being phased out (86%); absence of Continuous Professional Development (84%); HPIC Tracking and registering patients before testing: Correct Record Keeping (82%) and Staff involvement in POC Management program (80%). Each theme is presented below with supporting quotes.

#### 3.3.1. Absence of POC Testing Curriculum for Nurses

Stakeholders ranked the absence of a curriculum specific to POC diagnostic services for PHC nurses as the highest challenge to be addressed urgently. The main reasons they highlighted were that personnel who performed HIV testing relied on the knowledge they obtained in their past training, each other's experiences as well as the internet accessed through personal phones. Furthermore, they reported that there were no structured follow-ups to validate what they were doing. Priority is more on getting tests done and meeting targets. When asked about standard operating procedures they responded that these were not available in their clinics and one participant who worked at a clinic that did have one said that SOPs were not readily available and not easily accessible: "not having a standard training is very unfair because we do not even have internet to search when we do not understand something."

#### 3.3.2. Absence of Training of Staff on HIV Testing and Counselling as Lay Counsellors Are Being Phased Out

Stakeholders reported that they were concerned that no formal training was provided to professional nurses in order to gain skills needed for the provision of quality POC testing, particularly during this period of phasing out of lay counsellors. Participants highlighted that lay councillors were well trained to provide high quality HIV testing and counselling services. In addition, the main duty for lay counsellors was HIV testing, therefore all their attention was on providing this service to the best of their ability: "there is a great need to motivate, encourage and re-train Professional nurses to take over the quality duties of phased out HIV/AIDS lay-councillors, because lay councillors provided more quality testing than professional nurses who have other priorities."

#### 3.3.3. Absence of Continuous Professional Development

In addition to not having a POC curriculum, stakeholders reported lack of financial support to enrol on additional courses. Stakeholders reported that the shortage of staff in clinics prevented them from securing leave days to attend workshops. They also reported a lack of such opportunities and that they rarely receive invitations to attend workshops. As a result, PHC professionals rely on secondary information from senior nurses who are usually invited to attend workshops: "majority of Professional nurses obtained their qualifications many years ago, having them just be given HIV testing duties on top of their current workload without some form of training is not right"

"From time to time nurses need to be updated or retrained as new technologies change all the time"

#### 3.3.4. HPRS Tracking and Registering Patients before Testing: Correct Record Keeping

Health Patient Registration System (HPRS) is a tracking system that works via the Health Patient Registration Number (HPRN) with a purpose to track and keep a unique record for each patient. Stakeholders ranked the utility of this system for correct record keeping amongst high priority challenges because there are shortages of HPRS registers and there are no other means to support this system to ensure that correct information is captured. Moreover, this system is not linked across different health centres in the country, which then largely affects the reliability of HIV statistics reported for South Africa.

#### 3.3.5. Lack of Staff Involvement in POC Management Programs

Stakeholders who were still on their lay-counsellor duties expressed their disappointment by the lack of interest of professional nurses on POC Management programs. Amongst these programs they highlighted procurement, quality assurance and proficiency testing: "professional nurses always complain about how overworked they are as it is, adding HIV testing on top of all that they have to do is really worrying"; "POC management programs will suffer even more with having no one whose main focus is HIV testing"

#### **4. Discussion**

This study has identified priority areas for the development of a POC diagnostics curriculum to improve competence and adherence to POC diagnostics quality standards for primary healthcare (PHC) nurses in rural South Africa. From the five highest ranked priorities, four concern the absence of training, Continuing Professional development (CPD) opportunities for PHC professionals and poor staff involvement in HIV programs. The fourth highest ranked priority (HPRS Tracking and registering patients before testing: Correct Record Keeping) is the only priority addressing a different issue. However, this particular priority is a significant and very interesting finding to be looked at further, as it touches issues of record keeping and HIV statistics in the country. The findings of this study also identified gaps in the management and proper implementation of HIV diagnostics in South Africa. This is particularly due to the phasing out of HIV lay counsellors and adding HIV testing to professional nurse duties without a structured training in place to ensure proper transition as well as the sustainability of quality testing services. The findings of this study also demonstrate that identifying of priority areas towards improving the quality of POC diagnostic services has an impact on the achievement of the 90:90:90 goal by 2020, which is a local health priority. In recent years South Africa has made great progress in getting more people tested for HIV; where in 2017 the first of the 90-90-90 targets was met, with 90% of people living with HIV being aware of their status up from 66.2% in 2014 [20,21]. The statistics look impressive, however achieving these targets may lead to increased pressure on nurses performing HIV rapid tests, which could compromise the quality of tests performed as highlighted by the stake holders in this study.

The findings of this study support the wider literature in emphasizing that incompetency of health professionals, lack of CPD, training resources and time constraints remain as barriers to the provision of quality POC services in resource limited settings [3,4,22]. Furthermore, participants in a study conducted in a resource limited setting in India also highlighted high workloads, lack of willingness to participate in POC testing programs, missing support (including training) and pressure to meet targets as contributors to poor quality testing [23]. A report focusing the spotlight on the full achievement of the 90:90:90 goal highlighted healthcare workers attitudes as one of the barriers for patients who want to know their HIV status as well as those who want to return to care [24]. Stakeholder who participated in our study also raised concerns about poor health worker attitudes and lack of staff involvement in HIV management programs as a result of lack of training or too much work pressure, as lay counsellors are being phased out.

In comparison to other research approaches like focus group discussions and in-depth interviews, Stakeholder engagements through the NGT were shown to be effective in identifying priority areas for the development of a POC diagnostics Curriculum. Furthermore, the findings of this study support findings of recent studies conducted in a similar setting [2,3,21,22]. These studies also highlighted lack of support, training opportunities, high workloads and lack of staff involvement as challenges or priority arears to be addressed towards ensuring the provision of quality POC diagnostic services. Stakeholder engagements were further recommended for ensuring effectiveness of future diagnostics as well as for the true potential of POC testing to be realized [4,23].

Strengths of this study include that healthcare users' views and priorities are increasingly being recognized by policy makers in contributing towards improvement of current policies and practices [12]. A limitation to this study in comparison to other NGT studies may be due to poor availability of stakeholders from four KZN districts, however the participating stakeholders represented a wide variety of relevant role players in the implementation of POC diagnostic services. Moreover, since there were only two groups, all the participants had a fair opportunity to express their views and more time was available for probing and clarifying of ideas. Furthermore, there were no challenges in terms of comparing and presenting larger data sets, which is normally the case for multiple group NGTs [12,25].

Recommendations

Based on the success of the NGT in identifying and ranking priority areas with supporting reasons for the development of a POC curriculum for nurses in rural clinics, we recommend more stakeholder involvement in the development of a context specific POC diagnostic services curriculum for onsite training. This curriculum must be supported by the availability of quality assures in each clinic and regular meetings for staff members to review SOPs and discuss POC diagnostics related issues with management to ensure sustainability.

#### **5. Conclusions**

This study has presented key stakeholders' views on the development of a POC diagnostics curriculum to improve competence and adherence to POC diagnostics quality standards for primary healthcare (PHC) nurses in rural South Africa. Phasing out of HIV lay counsellors without a POC curriculum in place, continuous professional development and lack of staff involvement in HIV testing programs were ranked as the highest priority areas that need more focus to ensure delivery of quality POC diagnostic testing at the PHC level. We recommend continuous collaborations between all POC diagnostics stakeholders for the development of an effective and accessible curriculum to ensure quality and sustainable POC diagnostic services in rural PHC clinics.

**Author Contributions:** N.C. and T.P.M.-T. Conceptualized the study. D.K. carried out first data analysis during the NGT. N.C. carried out further data analysis and interpretation of results and then produced the first draft of the manuscript. T.P.M.-T. commented on the draft and contributed to the final version. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was funded by University of Kwazulu-Natal College of Health Sciences, grant number 589063.

**Acknowledgments:** The authors would like to thank all stakeholders, for their time and sharing their knowledge and experiences for the benefit of this study. We would also like to thank the Department of Health KwaZulu-Natal for giving permission for their employees to take time away from their clinical duties to participate in this study as well as the College of Health Sciences, University of KwaZulu-Natal, for resources to assist with setting up and conducting of this research study.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **Appendix A**

Identified list of Priorities for the development of a POC diagnostics Curriculum


#### **References**


© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

### *Article* **Key Stakeholders' Perspectives on Implementation and Scale up of HIV Self-Testing in Rwanda**

#### **Tafadzwa Dzinamarira 1,2,3,\*, Collins Kamanzi <sup>2</sup> and Tivani Phosa Mashamba-Thompson 1,4,5**


Received: 20 January 2020; Accepted: 13 February 2020; Published: 1 April 2020

**Abstract:** Introduction: The World Health Organisation recommends HIV self-testing as an alternative testing method to help reach underserved populations, such as men in sub-Saharan Africa. Successful implementation and scale-up of HIV self-testing (HIVST) in Rwanda relies heavily on relevant stakeholders' involvement. We sought to explore HIVST key stakeholders' perceptions of the implementation and scale-up of HIVST in Rwanda. Method: We conducted in-depth interviews with personnel involved in HIV response projects in Rwanda between September and November 2019. We purposively sampled and interviewed 13 national-level key stakeholders from the Ministry of Health, Rwanda Biomedical Center, non-governmental organizations and HIV clinics at tertiary health facilities in Kigali. We used a thematic approach to analysis with a coding framework guided by Consolidated Framework for Implementation Research (intervention characteristics, inner setting, outer setting, characteristics of individuals involved in the implementation and the implementation process). Results: Key stakeholders perceived HIVST as a potentially effective initiative, which can be used in order to ensure that there is an improvement in uptake of testing services, especially for underserved populations in Rwanda. The following challenges for implementation and scale-up of HIVST were revealed: lack of awareness of the kits, high cost of the self-test kits, and concerns on results interpretation. Key stakeholders identified the following as prerequisites to the successful implementation and scale-up of HIVST in Rwanda; creation of awareness, training those involved in the implementation process, regulation of the selling of the self-test kits, reduction of the costs of acquiring the self-test kits through the provision of subsidies, and ensuring consistent availability of the self-test kits. Conclusions: Key stakeholders expressed confidence in HIVST's ability to improve the uptake of HIV testing services. However, they reported challenges, which need to be addressed to ensure successful implementation and scale-up of the HIVST. There is a need for further research incorporating lower level stakeholders to fully understand HIVST implementation and scale-up challenges and strategies to inform policy.

**Keywords:** HIV self-testing; implementation; scale-up; key stakeholder

#### **1. Introduction**

Globally, it is approximated that only 79% of individuals who live with HIV are aware of their HIV status [1]. In Rwanda, the findings of a 2018–2019 national HIV survey in Rwanda indicated that 17% of adults living with HIV were unaware of their status [2]. By sex; 15% of HIV-positive women and 20% of HIV-positive men did not know their HIV status [2]. Improving the uptake of HIV testing services (HTS) remains one of the main strategies to combating HIV [3,4]. In 2016, the World Health Organization (WHO) provided the first global guidelines on HIV self-testing (HIVST), as an additional model for improving uptake of HTS [5]. Based on statistics from the WHO, 77 nations have adopted HIV self-testing policies, whereas several others are presently developing them [1]. WHO guidelines have been aimed at supporting the implementation and scale-up of ethical, effective, acceptable, as well as evidence-based approaches to HIVST [1]. Along with other sub-Saharan African countries, the Rwanda Ministry of Health in 2017 recommended the use of HIVST as an additional strategy [6].

HIVST refers to the process where an individual collects his own specimen, which could be blood or oral fluid, and thereafter carries out HIV testing and decodes the result, in most cases in a private setting [5]. HIVST has the potential to overcome some of the main barriers which are associated with the current testing models [7–11]. The notable barriers include stigma, discrimination, as well as non-confidential testing environments [7,8]. Acceptability studies have provided mixed findings. In South Africa, a very low rate of acceptability of HIVST (22%) was reported among conveniently-sampled adults [12]. Similarly, low acceptability (44%) was reported in China among men who have sex with men [13]. High acceptability rates have been reported among university students in the Democratic Republic of Congo (82%) [14]; men who have sex with men in Peru and Brazil (87%) [9] and a general adult population in western Kenya (94%) [11]. Globally, it has been documented in various studies that HIVST is acceptable in general populations [9,10,15–17]. A systematic review by Krause et al. presents evidence that HIVST is highly acceptable among key populations [8]. In Rwanda, a qualitative study on men's perspectives towards HIVST revealed that men found HIVST acceptable; however, lack of awareness, cost of the kits, and concerns over potential social harm and possible adverse events were reported as potential barriers to uptake [18].

With a goal to end AIDS by 2030 [19,20]; this goal calls for strategic implementation and scale-up strategies that result in increased uptake HTS. Within SSA Africa, Kenya has effectively implemented guidelines on programmatic approaches to HIVST. The Kenyan guidelines for HIVST describe the package of support services, commodity management systems, the coordination mechanisms, quality assurance measures, and describing some of the monitoring and evaluation strategies [21]. These guidelines played a key role in ensuring effective implementation and scale-up of HIVST in Kenya. The relevance of stakeholders in the implementation of policies in every healthcare delivery cannot be over-emphasized. Evidence has demonstrated the important role of stakeholders in the successful implementation of health policies [22,23]. The implementation and scale-up efforts for HIVST in Rwanda will rely largely on informed strategies that ensure improved uptake [24]. Adequate involvement of all the relevant stakeholders is crucial for the overall success of the implementation and scale-up efforts. This study therefore sought to explore the perspectives of key stakeholders concerning the implementation and scale-up of HIVST in Rwanda.

#### **2. Materials and Methods**

This study was conducted as part of a large study entitled: Adaptation of a Health Education Program for Improving Uptake of HIV Self-Testing among Men in Rwanda. The protocol for the main study is under consideration for publication elsewhere.

#### *2.1. Ethics*

This study has been ethically reviewed and approved by four institutional review boards: the Rwanda National Ethics Committee (Approval number: 332/RNEC/201; May 29th 2019), University Teaching Hospital of Kigali Ethics Committee (Approval number: EC/CHUK/0111/2019; June 17th 2019), Rwanda Military Hospital Institutional Review Board (Approval number: RMH IRB/036/2019; July 12th 2019) and the University of KwaZulu Natal Biomedical Research Ethics Committee (Approval number: BE/280/19; June 24th 2019). Study participants were provided with an information sheet explaining the objectives of the study, and all participants signed informed consent forms prior to participation.

#### *2.2. Study Setting*

Kigali Province is the capital city of Rwanda. It consists of three Districts, namely Gasabo, Kicukiro and Nyarugenge, 35 sectors, 161 cells, and 1183 villages in Kigali [25]. Kigali City houses all national-level stakeholders in the HIV program in Rwanda [26]. The overall HIV prevalence among adults in Rwanda is 3.0% [2]. Annual incidence of HIV among adults (defined as those aged 15–64 years) in Rwanda was 0.08% [2]. This corresponds to approximately 5,400 new cases of HIV annually among adults in Rwanda [2]. With a goal to end AIDS by 2030 [26], Rwanda is intensifying evidence-based interventions such as HIVST to further reduce HIV incidence.

#### *2.3. Study Sample*

In the study, the sample consisted of 13 purposively-selected key stakeholders. In particular, the sample was drawn from the Ministry of Health, Rwanda Biomedical Center, non-governmental organizations and HIV clinics at tertiary health facilities in Kigali.

#### *2.4. Data Collection*

We collected qualitative data from the key stakeholders using in-depth interviews. Interviews were conducted by trained researchers using an interview guide (Supplementary File 1), which contained open-ended questions. Interviews were conducted between September 2019 and November 2019 in different settings where the stakeholders or public health officials serve. Interviews were conducted in English and in Kinyarwanda languages and continued until saturation was reached; when no additional information was emerging from the interviews [27]. Interviews conducted in Kinyarwanda were translated by a professional translator with back translation to ensure no loss of data.

#### *2.5. Data Entry and Analysis*

The interviews conducted were audio-recorded and transcribed verbatim in Microsoft Word 2016. Verbatim transcription of all interviews, with study participant's checking [27] to seek points of clarification in relation to issues arising from interviews, was performed to ensure the validity of the interviews. All interview transcripts were uploaded into NVivo v12 software (QSR International Pty Ltd., Melbourne, Australia) for analysis. Framework-based thematic analysis was performed by TD and CK, in parallel guided by the Consolidated Framework for Implementation Research (CFIR) [28]. The framework-based synthesis approach followed these steps: familiarization; identifying a thematic framework; indexing; charting; and mapping and interpretation. This approach has been applied in policy-related research questions [29]. This approach enabled domains identified in advance in the CFIR to be explicitly and systematically considered in the analysis, while also facilitating enough flexibility to detect and characterise issues that emerged from the transcripts [30]. First, the authors familiarized themselves with the content of the transcripts. Secondly, participants' responses were coded into categories based on the CFIR domains, which were then grouped into nodes. Using the relationships module of NVivo, the nodes were grouped into similar concepts that reflect key stakeholders' perspectives on implementation and scale-up of HIVST in Rwanda. Finally, mapping and interpretation of the themes and sub-themes was done.

#### **3. Results**

In total, we interviewed 13 participants, including HIVST key stakeholders in Rwanda's HIV programs. In this study, we defined HIVST stakeholders as professionals working within the Rwanda Ministry of Health and its partners responsible for the implementation of HIVST in Rwanda. The individuals formed part of the national-level technical working group for HIVST implementation with oversight of implementation and scale-up efforts nationally. The characteristics of study participants are outlined in Table 1.


**Table 1.** The presentation of key stakeholders by age, gender, highest qualification, number of years' experience in the HIV response and role in HIVST implementation in Rwanda.

Key stakeholders were all well-aware of HIVST intervention and perceived HIVST as a potentially-effective initiative which can be used in order to ensure that there is improvement in uptake of testing services, especially for underserved populations in Rwanda.

#### *3.1. Researcher, Stakeholder*

"HIV self-testing basically refers to the process through which an individual who is interested in knowing their HIV status carry out the HIV test by themselves. They also interpret the result in private. It is just one of the ways through which individuals can get to know their HIV status after buying the self-test kits. It was introduced in order to address the challenges of stigma and confidentiality associated with routine provider initiated or voluntary counseling and testing. It offers the potential for HIV testing to reach more people than previously possible, including those who do not seek testing in our health facilities here".

#### *3.2. Supply Chain Specialist, Policy Advisor*

"It is an additional or a new approach in Rwanda to boost the existing HIV testing services that we currently have. It is a screening test which detects HIV antibodies. Currently in our country, it is in the pilot phase, not yet well implemented in the whole country. We have kits distributed in different pharmacies in Kigali and those vendors explain to those who come to buy it on how they can use it accordingly".

#### **4. Emerging Themes**

Three main themes emerged: HIVST is a potentially effective initiative to improve uptake of HIV testing services; challenges hindering effective implementation; and potential strategies which can be adopted to ensure effective implementation and scale-up of HIVST. The emerging themes and sub-themes are presented in Table 2.



A detailed framework analysis of key stakeholders' responses guided by the CFIR is presented on Supplemental File 2.

#### *4.1. Theme 1*

Key stakeholders perceived HIVST as an acceptable intervention with the potential to bridge the gap in the uptake of HTS. In addition to the general population, most stakeholders listed men, female sex workers, men who have sex with men, and the rick/famous people as potential groups that could benefit from HIVST initiative in Rwanda.

#### 4.1.1. Program Manager, Policy Maker

"Thank you, so (HIV) self-testing is one of the approaches that we are using to make sure that people know their HIV status. It was started as an additional option for people who want to know their HIV status and especially those who are not willing to use conventional HIV testing methods which are facility-based. This was chosen as an approach that will be increasing the number of people who are aware of their HIV status".

#### 4.1.2. Researcher, Stakeholder

"I think it (HIVST) is a noble intervention, which will play a key role in ensuring that there is an improvement in the uptake of HIV testing services, more so among the people who are otherwise normally hard to reach. It comes with a number of advantages, which generally include privacy, convenience, confidentiality, and ease of use".

#### 4.1.3. Program Manager, Policy Maker

"HIV testing services are available, free of charge, and available to all public health facilities, but we have some groups who don't reach those services for their own reasons, maybe self-stigma in the case of sex workers, maybe being too busy like men, or they are rich and famous. All these groups can now access [HIV] self-testing".

#### 4.1.4. Laboratory Specialist, Policy Maker

"Currently, we have only one kit available on the local market and at health facilities. OraQuick (manufactured) by Orasure Technologies. We performed in-house validations here at the reference lab. We obtained over 90% sensitivity for oral fluid. So as (HIV) program, we are confident this test can help improve our numbers on the first 90. More people can now get tested and interpret their own results. The only caveat is those results are not final; patients would still need to visit a health facility for retesting and confirmation".

#### *4.2. Theme 2: Challenges Hindering Implementation and Scale up of HIVST*

Key stakeholders alluded to challenges that were impeding the implementation and scale-up of HIVST in Rwanda. The participants presented a mix of intervention characteristics, outer and inner settings challenges. Most participants strongly felt the need to prioritize addressing these challenges before efforts to implement and scale-up HIVST are commenced in Rwanda.

#### 4.2.1. Sub Theme 1: Lack of Awareness

A general lack of awareness among the users of the kits emerged as a sub-theme. Stakeholders noted some concerns relating to lack of awareness of/on HIVST among the general population. Further, stakeholders noted that HIVST is still inaccessible to rural populations that are also resource-limited in terms of health facilities. Key stakeholders perceived that not much awareness has been created. They also noted that some of the people with access to the kits do not have adequate levels of education and knowledge on how to use them.

#### 4.2.2. Researcher, Stakeholder

"Before the government decides to ensure that it is rolled up on a large scale, I think they need to ensure that people are aware of the kits. At the moment I think that not so many people know about them. Again, the government should work on the costs that are associated with the acquisition of the kits. At the moment, it is beyond the reach of most of the people who are the main targets."

#### 4.2.3. Sub Theme 2: Cost of the Kits

Key informants perceived the costs of purchasing the kits as one of the main barriers to the effective implementation and scale-up of HIVST in Rwanda.

#### 4.2.4. Health Care Provider, Policy Maker

"I think that the ministry of health is still facing different kinds of challenges, which needs to be addressed to ensure effective implementation of self-testing. For instance, the general lack of the guidance on HIV self-testing, and the lack of low-cost test kits and the systems to assess and regulate them, have been a key barrier to implementation. They need to be addressed effectively. The ministry of health should also adopt policy guidelines that inform the adoption of suitable HIVST test kits, taking into consideration who exactly we want to get tested."

#### 4.2.5. Sub Theme 3: Results Interpretation

Key stakeholders noted their concerns on the interpretation of results following HIVST testing. Key stakeholders were of the perspective that reported cases of discordant results between the HIV

self-test and repeat testing at a health facility were mainly user-based. Issues with the interpretation of results were revealed as key in these discrepancies.

#### 4.2.6. Program Manager, Policy Maker

"HIV self-testing is very new in Rwanda. So far, I only can think of two challenges. For instance, we have received cases of false positives and negatives. This creates a problem. Even though the cases are few, it is enough to raise concern as it may affect uptake. Once a bad message is passed about HIV self-testing giving incorrect results, it may lower uptake significantly. And we don't want that. ( ... ) Basically, it comes down to ensuring those that are selling in private pharmacies or distributing at our (government) facilities are trained so they can train the users and the issue of regulation of sale of these self-test kits."

#### *4.3. Theme 3: Strategies to Improve Implementation and Scale-up of HIVST in Rwanda*

Key stakeholders presented their views on strategies which can be used to ensure successful implementation and scale-up of HIVST. These include the following: the creation of awareness; training those involved in the implementation process; regulation of the selling of the self-test kits; reduction of the costs of acquiring the self-test kits through the provision of subsidies; and ensuring consistent availability of the self-test kits.

#### 4.3.1. Sub Theme 1: Creation of Awareness

The need for the creation of awareness on HIVST in order to ensure uptake was noted by most key stakeholders. Community mobilization strategies proposed by some stakeholders include decentralized campaigns, community-led advocacy through the monthly *umuganda* community meetings, and radio jingles.

#### 4.3.2. Supply Chain Specialist, Policy Advisor

"The uptake of HIV self-testing is still very low, and as you may be aware, not much has been done to ensure that it is available to all the people in Rwanda, including those in the remote areas. So far, the focus has been only in Kigali City. There are also people who are still not aware of HIV self-testing. The government, therefore, needs to do much more to ensure that more people are made to be aware of HIV self-testing".

#### 4.3.3. Program Manager, Policy Maker

"I recommend improving awareness, to encourage those groups who don't get HIV testing and then to make availability of test kits at a low price."

#### 4.3.4. Sub Theme 2: Training of the People Involved in the Implementation Process

Key stakeholders emphasized the need for providing training to all parties involved in the implementation of the HIVST intervention in order to ensure successful implementation.

#### 4.3.5. Health Care Provider, Stakeholder

"The government should just ensure that it (implementation) is done in the right manner by training all the people involved, doing mass campaigns to ensure that more people are aware of the existence of the programs, and ensuring that there are proper distribution channels of the HIV (self) test kits".

#### 4.3.6. Sub Theme 3: Proper Regulation of the Kits

A common theme, on the strategies to ensure effective implementation and scale-up of HIVST implementation, was the need to ensure proper regulation of the self-test kits. Stakeholders noted that currently in Rwanda HIV self-test kits are available for purchase online. Stakeholders perceived the need for ensuring guidelines are followed with regard to certification for use of these kits.

#### 4.3.7. Laboratory Specialist, Policy Maker

"I also believe that proper regulation can play a key role in ensuring that some of the key challenges I have discussed earlier are addressed. We validated for use in Rwanda the test kits currently in circulation. We are also involved in ensuring quality assurance for new kit lots and surveillance for cases of false positive or negative results. We monitor those as well."

#### 4.3.8. Sub Theme 4: Reducing the Costs of the Kits

Most of the stakeholders were of the viewpoint that the kits are currently going for 5000 RWF for one test, which is not within the reach of most of the people who are targeted. As a result, effective implementation and scale-up of the kits needs measures to be put in place to subsidize the costs of acquiring the kits. This will make the kits to be within the reach of the users.

#### 4.3.9. Researcher, Stakeholder

"I think the ministry should look at the processes that are currently being charged. The kit currently costs 5000 Rwandan Francs in pharmacies in Rwanda and I think this is too expensive. While the government is working to ensure that the kits are available in the entire country, they need to ensure that there are measures aimed at lowering the prices of the kits. For such an important intervention, it is reasonable to provide subsidies to cushion users".

#### 4.3.10. Sub Theme 5: Ensuring Availability of the Kits

When it comes to effective implementation and scale-up of HIVST, key stakeholders perceived the need for the government of Rwanda to ensure that there are measures in place to ensure constant availability of kits. This means ensuring the availability of stock and proper distribution channels to improve uptake.

#### 4.3.11. Program Manager, Policy Maker

"I think here the most important point is to see how we can increase kit availability by ensuring adequate stock levels at MPPD always, and appropriate space where the kits will be distributed. So far, we have them limited at pharmacies, a few health facilities and online purchases. To improve uptake when we scale-up, there is a need to have a wider range of distribution channels".

#### **5. Discussion**

This study presents perceptions of HIVST key stakeholders on the implementation and scale-up of HIVST in Rwanda. Key stakeholders perceive HIVST to be a highly effective intervention for helping the underserved populations access HTS. This corroborates well with WHO recommendations on HIVST as an additional strategy to improve uptake of HTS [5]. Interventions aimed at improving uptake of HTS as an important step to attaining the UNAIDS 90-90-90 target of 2020 have been underscored in the Rwanda 2019–2024 Fourth Health Sector Strategic Plan [26]. Our findings reveal that key stakeholders perceive HIVST as an important gateway to realization of the UNAIDS 90-90-90 targets. Theme two identified key stakeholders' perceived challenges for implementation and scale-up of HIVST; lack of awareness of the kits, high cost of the self-test kits, and concerns over results interpretation. Low awareness was mainly attributed to the intervention being relatively new in Rwanda and still in pilot phases. Theme three: key stakeholders' perceived measures of what is necessary for the successful implementation and scale-up of HIVST in Rwanda included the creation of awareness, training those involved in the implementation process, regulation of the selling of the self-test kits, reduction of the costs of acquiring the self-test kits, and ensuring consistent availability of the self-test kits. Health education programs, community mobilization, development of HIVST country guidelines and provision of subsidies to cushion the cost of test kits would strengthen implementation and scale-up efforts for HIVST in Rwanda.

In the current study, key stakeholders' perceived HIVST as an auspicious intervention with the potential to bridge the current gap in uptake of HTS in Rwanda. This corroborates well with findings from a similar study conducted in South Africa, which demonstrated key stakeholders' confidence in HIVST improving uptake of HTS in underserved population [31]. A recently published systematic review and meta-synthesis on men's perspectives on HIV self-testing in sub-Saharan Africa recommended presented evidence of poor awareness but high acceptability of HIVST among men [32]. While this is the case; stakeholders expressed concerns that need to be addressed before effective implementation and scale-up can be achieved. Concerns cited by key stakeholders in the current study on the regulation of the sale of test-kits and results interpretation have been reported elsewhere [7,33–35]. Healthcare workers in Kenya concerned with challenges with test results interpretation recommended proper regulatory measures to be put in place prior to scale of HIVST intervention [7]. Similarly, healthcare providers in Kwa-Zulu Natal province in South Africa perceived issues with results interpretation as a potential challenge with HIVST implementation in South Africa [33]. A cross-sectional study on participants without prior experience with the HIV self-test revealed the most common interpretation error was incorrectly identifying a negative result as invalid [10]. Raising public awareness levels emerged as a key strategy to effective implementation and overall success of scale-up efforts in this study. Similar recommendations have been made elsewhere [31,32,36,37]. Key stakeholders in South Africa [31], researchers, academics, journalists, community advocates, policy makers and other key stakeholders in Nigeria [36] and reviews by Hlongwa et al. [32], Johnson et al. [37] all recommend need the improve public awareness on HIVST. Similar to the findings of the current study, ensuring that kits are affordable has been recommended by key stakeholders in South Africa [31,33] and potential users in Singapore [10].

The study has demonstrated the feasibility of HIVST implementation and scale-up in Rwanda from a key stakeholder's perspective. There is a need to document HIVST guidelines and policies that define the supply chain, stakeholder roles and responsibilities, implementation strategy, quality assurance measures, and monitoring and evaluation strategies. Policymakers need to ensure that effective mobilization programs are designed to raise public awareness. Training for those involved in the implementation and subsequent step-down training will be key in the implementation and scale-up efforts of HIVST in Rwanda.

A notable strength of the current study is that the majority of the key stakeholders were men. The current study is part of a larger study aimed at adaptation of a health education program for improving the uptake of HIVST among men. A limitation of this study was that only national-level key stakeholders residing in Kigali were enrolled, thus limiting the generalization of study findings to other settings. There is a need for further research incorporating lower-level stakeholders and to fully understand the challenges and inform policy. However, our sample was drawn from individuals who are responsible for the implementation of HIVST in the country, with knowledge on the status of the current implementation and scale-up efforts across the country. Finally, qualitative findings are highly subjective [38]. However, we used prolonged engagement [39,40] to ensure credibility and pilot testing of the interview guide [41,42] to ensure dependability. We enhanced the credibility and dependability of the study findings by following a rigorous inductive analysis and interpretation of the data. We also employed note-taking [42] and participant validation of transcripts [43] to enhance the credibility of the reported findings.

#### **6. Conclusions**

The current study findings demonstrate the confidence of key stakeholders in the Rwanda health system to effectively sustain the HIVST intervention. The concerns raised over factors with the potential to impede smooth implementation and scale-up should be addressed.

**Supplementary Materials:** The following are available online at http://www.mdpi.com/2075-4418/10/4/194/s1, Supplementary File 1: Interview guides for the in-depth interviews with different key stakeholders and health care providers. Supplemental File 2: Detailed analysis of the interview transcripts against the Consolidated Framework for Implementation Research.

**Author Contributions:** T.D. and T.P.M.-T. conceptualized the study. T.D. and C.K. carried out the first analysis of the study. T.D. produced the first draft of the manuscript. T.P.M.-T. reviewed the draft and contributed to the final version. All authors have read and agreed to the published version of the manuscript.

**Funding:** The University of KwaZulu-Natal, College of Health Sciences PhD Scholarship supported this study (Grant number 641581). The funder had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

**Acknowledgments:** The authors would like to thank all stakeholders that participated in this study by sharing their valuable input. Tafadzwa Dzinamarira is supported by The University of KwaZulu-Natal, College of Health Sciences PhD Scholarship. This study was supported by the CIHR Canadian HIV Trials Network (CTN 222). Tivani P. Mashamba-Thompson is supported by CTN Postdoctoral Fellowship Award. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Further, the authors acknowledge Claude Mambo Muvunyi and Gashema Pierre for their contribution in the data analysis.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

*Case Report*
