*Antibiotics* **2020**, *9*, 315 *Antibiotics* **2020**, *9*, x 8 of 16 directly from the intervention, where antibiotics in red denote project-related increases and those in blue denote decreases.

have failed.


outcomes. The following section describes the supply chain system at FPRRH and presents data on

1. The hospital may only order against a budget prescribed by the Ministry of Finance and held

2. The hospital can only order antibiotics from a prescribed catalogue which excludes many of the antibiotics indicated as necessary from laboratory results and present on the 'Essential

3. There are major and unpredictable discrepancies in what is ordered and what is delivered

4. As a result of the above, most IPC supplies and antibiotics run out half-way through the bi-

In Uganda, the funds for procurement of drugs and supplies in the public sector are highly centralised and inadequate. NMS procures and distributes supplies to health facilities based on a centrally allocated Annual Supplies Budget. Each hospital is required to produce an Annual Procurement Plan. Once agreed, this Plan is fixed and cannot be varied over the year reducing the opportunity for flexibility and responsiveness to the hospital laboratory results and any changes indicated by a future antibiogram. This budget is held by NMS. With the exception of private wards, it is not possible for a RRH to source supplies from elsewhere. NMS deliveries take place bi-monthly. During the annual procurement process, the hospital may only order those items authorised by the 'Essential Medicines and Health Supplies List for Uganda'. However, not all essential drugs

*As much as we may desire a certain antibiotic, we can't plan for it if it is not present in the catalogue. A* 

Out of the nine antibiotics tested against *Acinetobacter* samples in the laboratory, only two doxycycline and amikacin—showed greater levels of susceptibility than resistance (for details of these test results, see Supplement 1). Given the much higher success rate of amikacin, it is paramount that the antibiotic can be obtained for cases of severe *Acinetobacter* infections where other avenues

Within the constraints described above, the MSI has influenced procurement planning for 2020/21. Figure 1 evidences significant changes in antibiotic ordering and consumption arising

feature in the NMS catalogue. A hospital pharmacist describes the situation as follows:

consumption. Critical problems include:

('Order Fill Rates').

*case in point is Amikacin and Moxifloxacin.* 

monthly supply cycle (Stock-Outs).

Medicines and Health Supplies List for Uganda' [20].

*3.5. The Impact of the MSI Project on the 2020/2021 Procurement Plan* 

by NMS.

**Figure 1.** Extract from the 2020/2021 Procurement Plan (with bi-monthly figures) focusing on key Antibiotics used on PNG Wards. In this table, the column 'unit' shows the number of doses per unit as sold. 'Price' is the price per unit in Ugandan Shillings (1 USD = 3780 UGX, May 2020). The column '2019/2020 plan' shows the number of units ordered for delivery every other month in 2019, with 'past **Figure 1.** Extract from the 2020/2021 Procurement Plan (with bi-monthly figures) focusing on key Antibiotics used on PNG Wards. In this table, the column 'unit' shows the number of doses per unit as sold. 'Price' is the price per unit in Ugandan Shillings (1 USD = 3780 UGX, May 2020). The column '2019/2020 plan' shows the number of units ordered for delivery every other month in 2019, with 'past av consumption' detailing the average bi-monthly consumption of the past year. The column '2020/2021' details the set number of units ordered every other month for this year, with the corresponding bi-monthly cost reported in the final column.

The pharmacy team involved in procurement planning pointed out the severe budgetary constraints they faced when attempting to order new antibiotics in response to laboratory results. In practice, this meant making difficult 'trade-offs', especially when the new antibiotics are so much more expensive than those they were able to reduce. The reduction in supply of amoxicillin for example is explained as follows:

*We realised that the majority of patients using amoxicillin were mothers discharged after giving birth. They are usually given amoxicillin as prophylaxis to prevent infection. Some were being given for a longer duration than necessary. As pharmacy sta*ff*, we intervened so that the duration of treatment would be reflective of the nature of risk. This led to a reduction in use. We had to increase certain antibiotics or include new antibiotics as well. Due to budget constraints, it was agreed during the planning stage that we cut on the quantity of Amoxicillin to free up some budget to cater for other needed antibiotics.*

The laboratory results indicated very high levels of resistance to both amoxicillin and ampicillin, both of which are derived from penicillin (Supplement 1). Based on these assumptions, ampicillin and amoxicillin will have minimal effects on the three primary bacterial causes of infection.

Significant changes in planned use of Meropenem can also be directly attributed to the MSI. The pharmacist explains that consumption of this drug over the past year has been reliant entirely on donated supplies (it was not ordered in 2019):

*[The increased order] can be supported by evidence generated by the laboratory. Due to the increased culture and sensitivity reporting, we noticed that there was improved sensitivity to meropenem. This ensured that we were able to convince members involved in planning to include it on the 2020*/*2021 plan. We were able to get some donations last year and that's why it shows that we consumed it. What's more, we wrote to NMS to allow us procure it, even though it's not in the current plan.*

This action, of communicating directly with NMS on procurement, represents one example of a scenario where the pharmacy team have attempted to advocate as a result of the MSI. The impact of this procurement may be to the benefit of other hospitals if NMS are influenced to place it on their catalogue in future.

*Antibiotics* **2020**, *9*, 315

The decision to increase orders of Meropenem required the team to make stark choices, which led to the reduction in orders of cefotaxime:

*Just like Meropenem, this particular consignment of Cefotaxime (used in 2019) was a donation. It wasn't in the procurement plan. While working on the 2020*/*2021 plan, we had to prioritise between Cefotaxime and Meropenem. We had to go with Meropenem. We did factor in the cost and resistance profile per the lab reports.*

Cefotaxime has shown high levels of resistance in the laboratory tests (Supplement 1). The marked rise in procurement of Ciprofloxacin is also directly attributable to the MSI, although the pharmacy team were concerned about the volume needed:

*What we require is actually a lot more. Again, [the increase] can be explained by the results of culture and sensitivity. There seems to be less resistance to ciprofloxacin.*

The pharmacist sums up the impact that the project has had on procurement planning and the constraints the team had to work with:

*We had to reduce the quantity of Ceftriaxone by a significant margin. This again was supported by laboratory data which showed a lot of resistance to ceftriaxone. Some of the monies freed up were used to plan for chloramphenicol and meropenem, drugs which are showing less resistance as per lab reports. There is no significant increase in this current budget and the incoming budget for drugs and medical sundries. It's therefore painstaking to reallocate priorities in terms of drugs while maintaining the same budget. Our [MSI] e*ff*orts to encourage and support Culture and Sensitivity testing and sharing this with the procurement planning team lead us to include some much-needed antibiotics (Meropenem and Chloramphenicol) on next year's plan and reduce the procurement of antibiotics with a lot of resistance (Ceftriaxone).*

Figure 1 also provides an indication of the cost implications of the changes in antibiotic procurement as a result of the MSI. Most of the increases in procurement involve more expensive antibiotics. The procurement plan reflects the negotiations the pharmacy team have engaged in, to balance the need for rational prescribing against the cost implications of buying more expensive antibiotics. Unfortunately, the constraints of the NMS budget-line are not the end of the story.

#### *3.6. Discrepancies between Order and Supply (Order Fill Rates)*

In practice, not all that is ordered by the hospital from NMS is supplied. The 'Order Fill Rate' gauges the delivery performance of total number of items ordered against the total number of items delivered. As clearly seen in Table 2, NMS supplies about 75% of orders. More specific discrepancies may also arise. Unusually, NMS failed to deliver Ceftriaxone in September 2019, for example.


**Table 2.** The Order Fill Rate at Fort Portal Regional Referral Hospital (FPRRH) (2019/2020).

Source: Rx on-line medicines management system (National Medical Stores (NMS) do not provide data on fill rates for specific medications)

#### *3.7. Key Challenges to Sustained Behaviour Change: The impact of Stock-Outs on AMS*

Stock-outs (the exhaustion of supplies) are a feature of Ugandan public health facilities at all levels, and are a major factor contributing to sepsis deaths in maternal and new-born health [21]. Inevitably, the bi-monthly deliveries tend to be exhausted quite rapidly and often by the end of the first month, when 'stock-outs' become a major feature of life and cause of morbidity and mortality at FPRRH. A pharmacist describes the situation as follows:

*For example, when we get 6000 vials of Ceftriaxone, we consume all of it in maybe 4 weeks, then we stay without for another 3–4 weeks. And the following cycle, we get the same quantity. Therefore [consumption data] are merely an average of what is not enough.*

Stock-outs are caused by a combination of misuse and overall shortfalls. The pharmacists noted that the project had improved antibiotic use on the wards:

*When these antibiotics are received [from NMS] they tend to run out quickly. Again, this is attributed to the small budget and probably misuse*/*irrational prescribing. However [MSI's] endeavour to link the ward, pharmacy and the Lab has to a great extent solved the issue of irrational antibiotic prescribing.*

By way of illustration this shortfall, on 18th February 2020 the PNG ward contacted K4C to request support in the purchase of gauze. Without this, they would not be able to continue with the wound dressing established on the ward. This would have resulted in increased infection and sepsis (and antibiotic consumption). We were aware during the project visit in January 2020 that the hospital had also run out of disinfectants, iodine and spinal needles (amongst many other things). In such circumstances the only option is for staff to ask patients to pay for the necessary items, and if they are unable to pay then operations will not happen, and major delays occur in treatment. On 19th February 2020 we established that 13 key items for use on the PNG ward were out of stock and had been for over a month:


The next supplies were expected on 25th February.

#### *3.8. Antibiotic Consumption Patterns at Ward Level in FPRRH*

When supplies arrive at FPRRH from NMS, they are located at the Main Stores. At this point, supply data is recorded electronically in the on-line ordering system (Rx). The process of distributing supplies within the hospital is, unfortunately, not covered by this electronic system. Instead, the in-patient pharmacy (located a short distance from the Stores) orders from the Stores. Individual wards then visit the in-patient pharmacy to requisition supplies, and this is recorded manually on forms and in a records ledger book (the HMIS Dispensing Log). Figure 2 presents data on the distribution of oral antibiotics between the main hospital wards in January and February 2020. As can be seen, the level of antibiotic consumption on the PNG wards as a proportion of overall consumption is high and indicates the importance of this to overall AMS. The data presented in this figure illustrates three important trends, which were also seen in the use of intravenous antibiotics:


13. Povidone Iodine;

The next supplies were expected on February 25th.

*3.8. Antibiotic Consumption Patterns at Ward Level in FPRRH* 

• The profound impact of stock-outs on access to antibiotics with a reduction in oral antibiotic use in February, showing a reduction in consumption to between 25% and 30% compared to the January figures. • The significant contribution that PNG ward makes to overall antibiotic consumption. • The profound impact of stock-outs on access to antibiotics with a reduction in oral antibiotic use in February, showing a reduction in consumption to between 25% and 30% compared to the January figures.

• The dominance of Amoxicillin and Metronidazole in antibiotic consumption.

*Antibiotics* **2020**, *9*, x 11 of 16

When supplies arrive at FPRRH from NMS, they are located at the Main Stores. At this point, supply data is recorded electronically in the on-line ordering system (Rx). The process of distributing supplies within the hospital is, unfortunately, not covered by this electronic system. Instead, the inpatient pharmacy (located a short distance from the Stores) orders from the Stores. Individual wards then visit the in-patient pharmacy to requisition supplies, and this is recorded manually on forms and in a records ledger book (the HMIS Dispensing Log). Figure 2 presents data on the distribution of oral antibiotics between the main hospital wards in January and February 2020. As can be seen, the level of antibiotic consumption on the PNG wards as a proportion of overall consumption is high and indicates the importance of this to overall AMS. The data presented in this figure illustrates three

**Figure 2.** Supply of Oral Antibiotics to All Wards in January and February 2020. **Figure 2.** Supply of Oral Antibiotics to All Wards in January and February 2020.

 Figure 3; Figure 4 present data collected from the in-patient pharmacy on key antibiotics used in the PNG wards only. NMS deliveries were made on 13th December 2019 and February 24th, 2020. The dispensing patterns show stark evidence of stock-outs. Figures 3 and 4 present data collected from the in-patient pharmacy on key antibiotics used in the PNG wards only. NMS deliveries were made on 13th December 2019 and 24th February 2020. The dispensing patterns show stark evidence of stock-outs. *Antibiotics* **2020**, *9*, x 12 of 16

**Figure 3.** Dispensing of IV Ceftriaxone and Metronidazole from In-Patient Pharmacy to Post-Natal and Gynaecology (PNG) Wards (1/12/2019-29/03/2020). **Figure 3.** Dispensing of IV Ceftriaxone and Metronidazole from In-Patient Pharmacy to Post-Natal and Gynaecology (PNG) Wards (1/12/2019–29/03/2020).

The higher consumption of metronidazole can be attributed to the high dosing regimen (three times daily), plus its empirical indication as a broad-spectrum therapy for prophylaxis against anaerobes. IV Ceftriaxone (dosed once daily) is also being used empirically and for prophylaxis, especially in surgical cases. Dispensing of IV metronidazole showed evidence of stock-outs but for shorter periods than Ceftriaxone. In the period between February 15th and 23rd, neither IV Metronidazole or IV Ceftriaxone was available to the PNG wards. Similar falls can be seen at

**Figure 4.** Dispensing of Oral Amoxicillin and Metronidazole from In-Patient Pharmacy to PNG Wards

Christmas and New Year.

(1/12/2019-29/03/2020).

**Figure 3.** Dispensing of IV Ceftriaxone and Metronidazole from In-Patient Pharmacy to Post-Natal

The higher consumption of metronidazole can be attributed to the high dosing regimen (three times daily), plus its empirical indication as a broad-spectrum therapy for prophylaxis against anaerobes. IV Ceftriaxone (dosed once daily) is also being used empirically and for prophylaxis, especially in surgical cases. Dispensing of IV metronidazole showed evidence of stock-outs but for shorter periods than Ceftriaxone. In the period between February 15th and 23rd, neither IV

and Gynaecology (PNG) Wards (1/12/2019-29/03/2020).

**Figure 4.** Dispensing of Oral Amoxicillin and Metronidazole from In-Patient Pharmacy to PNG Wards (1/12/2019-29/03/2020). **Figure 4.** Dispensing of Oral Amoxicillin and Metronidazole from In-Patient Pharmacy to PNG Wards (1/12/2019–29/03/2020).

The higher consumption of metronidazole can be attributed to the high dosing regimen (three times daily), plus its empirical indication as a broad-spectrum therapy for prophylaxis against anaerobes. IV Ceftriaxone (dosed once daily) is also being used empirically and for prophylaxis, especially in surgical cases. Dispensing of IV metronidazole showed evidence of stock-outs but for shorter periods than Ceftriaxone. In the period between 15th and 23rd February, neither IV Metronidazole or IV Ceftriaxone was available to the PNG wards. Similar falls can be seen at Christmas and New Year.

Dispensing patterns for oral metronidazole show higher utilisation and longer periods of stock-outs than oral amoxicillin, with an extended stock-out from 26th January to 23rd February coinciding exactly with the stock out of IV Metronidazole.

#### *3.9. Interpreting In-Patient Pharmacy Data—A Note*

We discussed some of the challenges of collating and interpreting facility consumption data (above). The emphasis in instruments such as the WHO's Practical Toolkit for AMS programmes in health-care facilities in low- and middle-income countries [22], which prescribe 'outcome measures' aligned to Western consumption indicators (such as Defined Daily Dose and Direct Observed Therapy) fail to capture the reality of many LMIC contexts. Hantrais' work on comparative methods [23] discusses the idea of 'conceptual equivalence', arguing that 'concepts cannot be separated from contexts' [23] (p. 73). Operationalising the concept of direct observed therapy presents insurmountable challenges in Ugandan RRHs. Hospital pharmacists were very aware of these limitations:

#### *We cannot measure performance by zeroing on antibiotic [consumption] only.*

Collecting and analysing the data on antibiotic 'consumption' patterns at FPPRH has emphasised the dangers of empiricist approaches to data analysis and presentation, and the importance of interrogating data rigorously. In most cases, data presents a myriad of questions and very few obvious and immediate answers. This is especially the case when attempting to collate data from public health facilities in Uganda and many other LMICs. Collecting data has been a continuous process of trial and error, merging with the underlying ethnographic journey. As described, in the case of in-patient pharmacy, the data is not yet managed electronically via the Rx system. Rather, individual wards come to a window in the in-patient pharmacy with paper forms requesting supplies for that day. The pharmacy then maintains a hand-written record of dispensing in a records book and this data is compiled into forms for the Ministry of Health. In the first instance, we believed that all wards behaved in this manner, but the initial data suggested otherwise. The TB ward, for example, appeared to collect very few drugs. We found that most of the drugs used on the TB ward are in fact provided by a donor and follow a different track. These drugs pass through the main stores and are directly requisitioned by the TB ward. As such, they do not pass through in-patient pharmacy and neither are they recorded as received from NMS on the Rx system, leaving a gap in overall supply and consumption data.

Our observational work on the wards, supplemented by qualitative interviews and many emails led us to question the relationship between this 'consumption' data and overall consumption patterns. We know, for example, that since laboratory results have been available, many women are getting higher-end antibiotics and that this is contributing to shortened stays and improved patient outcomes [10]. However, the supplies of these drugs were not visible in the data. It seems that the pharmacists have played a critical role in supporting access to these antibiotics through a combination of 'borrowing' from other hospital supplies. The following excerpt explains this process:

*There is a TB focal person who handles all TB related logistics, including the ordering of TB drugs. These drugs are stored in main stores. Rx only focuses on drugs from NMS. In most cases, we don't have the changed antibiotics in stock or in our procurement plan or we have limited quantities. Take Amikacin for example. We usually borrow from the TB program and give to the patients. The same applies to Moxifloxacin. These medicines are not available in the inpatient pharmacy. In fact, one time you had to use K4C money to purchase Amikacin. Because at times we have septic patients who are only responsive to these drugs, we 'beg'*/*borrow from the TB drugs. Apparently, this has caused audit queries.*

This 'borrowing' behaviour clearly saves women's lives; it also compromises the pharmacists under pressure to assist, but potentially contravening donor conditionalities. We can anticipate similar situations in relation to anti-retroviral therapies (for HIV patients) and also, in the case of FPRRH given its proximity to Congo, some (necessary) stockpiling in the event of Ebola spread.

Other apparent 'discrepancies' in the data, including very sporadic and low use of antibiotics in the neo-natal intensive care unit (NICU) and the paediatric ward uncovered other variances in practice, which are undocumented and apparently do not comply with the published protocol. This was explained by one respondent as follows:

#### *The paediatric ward gets injectable drugs (directly) from main stores, including antibiotics, but oral antibiotics and other oral drugs from in-patient pharmacy.*

When drugs are prescribed but not in-stock, patients are asked to buy them privately. Current recording systems cannot capture the consumption of privately purchased drugs. These examples underline the need to exercise caution when interpreting data on antibiotic consumption. The team is currently completing a more in-depth SSI follow-up on patients which will capture some of these processes.

#### **4. Summary and Conclusions**

Dyar's paper [24] reviews the use of the term 'antimicrobial stewardship' and concludes that there has been an overemphasis on conceptualisations focused on 'individual prescriptions', and insufficient emphasis on the societal implications of antimicrobial use. Furthermore, and of particular relevance to the MSI project, there has been insufficient translation of the concepts of 'responsible use' into context and time-specific actions. The authors conclude that AMS is not so much a concept, as it is a tool to assess whether organisations are identifying actions to improve responsible use in the specific context within which they are functioning. This idea fits very well with the action-research approach used in our intervention, and the results arising from that.

Sadly, the changes described above are essential to achieving 'responsible use' in a Ugandan RRH; but they are not sufficient. They create the opportunity for active pharmacy engagement in multi-disciplinary decision-making. This achievement could have a major impact on AMS at the hospital, as these wards consume by far the largest volume of antibiotics at the hospital. The cost-effectiveness of this intervention underlines the sustainability potential and the immediate opportunity for scale-up across the hospital as a whole, but also to other public health facilities in Uganda and beyond. Cox et al. make the important point that, 'delayed or no access to antibiotics kills more people than antibiotic resistant bacteria. . . . AMS is not only about reducing inappropriate use, but also assuring access to effective treatment' [1] (p. 813).

The findings evidence significant and impactful behaviour change on the PNG wards, with genuine multi-disciplinary team-working contributing to changes in prescribing behaviour and AMS. Wound care and laboratory testing lie at the heart of these changes centre staging nurses, midwives and laboratory scientists in AMS processes. The results of the microbiology testing then provide a platform for genuine multi-disciplinarity and, specifically, the first opportunity for clinical pharmacy engagement. This is true both at the level of rational (evidence-based) prescribing for individual patients and in improving the evidence base behind empirical prescribing (through an understanding on patterns of AMR). If access to a full range of antibiotics were available, this platform of behaviour change would transform antimicrobial use patterns, reduce the overuse and inappropriate use of antimicrobials and improve patient outcomes and deliver significant cost-benefits.

The second part of the paper has elaborated the complexity and opacity of the supply chain system in a RRH setting in Uganda. In the absence of effective supplies not only will these cost-savings elude facilities, but the patients involved will fail to thrive, and the motivation of health workers to apply the skills and knowledge they have demonstrated will inevitably decline. The paper has explained, in detail, the complex dynamics of supply chain management in a Ugandan public hospital. Understanding and piecing together these processes has required painstaking ethnographic research to unpick major errors in record-keeping and interpret the trends observed. In the first instance, the very centralised system creates huge dependency on the functionality of NMS and the adequacy of Ministry budgets. Centralisation may be seen as necessary in systems so damaged by corruption but where this undermines flexibility and responsiveness and generates extended and predictable stock-outs, the systems put in place to improve AMS will, inevitably, fail.

The MSI has demonstrated the potential for change and the efficiencies associated with this. We hope that publication of this evidence will stimulate discussion at national level amongst all key stakeholders, and generate a momentum for change. The COVID-19 pandemic has thrown a light on supply-chain effectiveness and the impact of weak supply chains on global and national inequalities. Although the poorest in societies will suffer disproportionately, the tentacles of AMR, as with all global pandemics will reverberate across the globe.

At a local level, after years of ongoing engagement, the Kabarole Health Partnership has recently signed a Public-Private-Partnership (PPP) agreement, which was triggered by the current project. The PPP will generate a more sustainable and integrated mechanism for supply-chain augmentation with an emphasis on IPC and antimicrobials. This will enable foreign organisations to cooperate on a co-decision and co-funding basis, guided by the hospital's Medicine Therapeutic Committee, and supported by a not for profit supplier, Joint Medical Stores. The objective will be to move away from dependency-generating donations to a more integrated approach with the agility to respond to local needs.

**Supplementary Materials:** The following are available online at http://www.mdpi.com/2079-6382/9/6/315/s1, File 1: Antimicrobial Resistance patterns at FPRRH. Figure 1: Extract from the 2020/2021 Procurement Plan (with bi-monthly figures) focusing on key Antibiotics used on Post Natal and Gynae Wards, Figure 2: Supply of Oral Antibiotics to All Wards in January and February 2020, Figure 3: Dispensing of IV Ceftriaxone and Metronidazole from In-Patient Pharmacy to PNG Wards (1/12/2019–29/03/2020), Figure 4: Dispensing of Oral Amoxicillin and Metronidazole from In-Patient Pharmacy to PNG Wards (1/12/2019–29/03/2020).

**Author Contributions:** L.A. was the Principal Investigator for the MSI and was involved in project conceptualisation; funding acquisition, research design; data collection, analysis and original draft application. G.A.-J. was involved in project design, data collection, formal analysis and draft preparation. M.S. was involved in investigation, formal analysis and writing–original draft preparation. J.O. was involved in investigation and writing–original draft preparation. S.O. was co-applicant and funding acquisition and review and editing. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was funded by the Commonwealth Partnership for Antimicrobial Stewardship (CwPAMS) Project Reference AMSB03.

**Acknowledgments:** The authors would like to acknowledge the support of Richard Walwema and Daniel Kibombo (Infectious Disease Institute); Louis Muhindo (Hospital Administrator); the nursing and midwifery team on Post Natal and Gynae ward; Alan Muwaza and Simon Sseguya (Hospital Pharmacists); the K4C team and Beatrice Waddingham, Project Manager at the Tropical Health and Education Trust.

**Conflicts of Interest:** The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

#### **References**


© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

*Article*

#### **Developing a Sustainable Antimicrobial Stewardship (AMS) Programme in Ghana: Replicating the Scottish Triad Model of Information, Education and Quality Improvement**

**Jacqueline Sneddon 1,\*, Daniel Afriyie <sup>2</sup> , Israel Sefah <sup>3</sup> , Alison Cockburn <sup>4</sup> , Frances Kerr 5,6 , Lucie Byrne-Davis <sup>7</sup> and Elaine Cameron 7,8**


Received: 24 August 2020; Accepted: 22 September 2020; Published: 23 September 2020

**Abstract:** (1) Background: Our aim was to develop robust and reliable systems for antimicrobial stewardship (AMS) in Keta Municipal Hospital and Ghana Police Hospital. Objectives were to build capacity through training staff in each hospital, establish AMS teams, collect data on antibiotic use and support local quality improvement initiatives. (2) Methods: The Scottish team visited Ghana hospitals on three occasions and the Ghanaian partners paid one visit to Scotland. Regular virtual meetings and email communication were used between visits to review progress and agree on actions. (3) Results: Multi-professional AMS teams established and met monthly with formal minutes and action plans; point prevalence surveys (PPS) carried out and data collected informed a training session; 60 staff participated in training delivered by the Scottish team and Ghanaian team cascaded training to over 100 staff; evaluation of training impact demonstrated significant positive change in knowledge of antimicrobial resistance (AMR) and appropriate antibiotic use as well as improved participant attitudes and behaviours towards AMR, their role in AMS, and confidence in using the Ghana Standard Treatment Guidelines and antimicrobial app. (4) Conclusions: Key objectives were achieved and a sustainable model for AMS established in both hospitals.

**Keywords:** antimicrobial stewardship; training; antibiotics use; behavior change

#### **1. Introduction**

The Scottish Antimicrobial Prescribing Group (SAPG) has established a comprehensive and robust national antimicrobial stewardship (AMS) programme coordinated by a national group working with regional antimicrobial multi-professional teams [1]. The national group is chaired by an Infection Specialist (Infectious Diseases Consultant or Microbiology Consultant) but the lead for the programme is an Antimicrobial Pharmacist. The regional AMS teams, in common with those in the rest of the UK

and other European countries, are generally led by an Infection Specialist but the majority of their stewardship interventions are delivered by Antimicrobial Pharmacists and increasingly supported by specialist nurses. Close multi-professional working has been critical to the success of the Scottish AMS programme. This has been successful in changing prescribing practice, providing rich data on antimicrobial use and resistance, providing education for health and social care staff across all settings and applying quality improvement methodology at scale to tackle areas of poor practice [2]. The approach in Scotland is aligned with and informed by the United Kingdom (UK) Antimicrobial Resistance (AMR) National Action Plan [3] and supports the ambitions for stewardship within Europe [4] and those of the World Health Organisation (WHO) [5] as one of several important actions for tackling AMR.

The model for SAPG was adopted from the Swedish Strategic Programme Against Antibiotic Resistance (Strama) programme [6] following visits by key personnel. The Scottish triad approach utilises Information, Education and Quality Improvement as the three key elements required for effective stewardship. The SAPG model has informed approaches in several other countries including Wales [7], Kenya [8], South Africa [9] and Brazil [10].

In 2019 the SAPG secured a global volunteering grant from the Fleming Fund's Commonwealth Partnerships for Antimicrobial Stewardship (CwPAMS) [11] to work with two hospitals in Ghana. This was the first such grant that required partnership leads to be pharmacists, reflecting their major role in delivery of AMS. The Ghanaian Ministry of Health had developed national Standard Treatment Guidelines (STG) for the management of common infections and had a 5-year National Action Plan (NAP) for AMR (2017–2021) [12]. The NAP covers improving knowledge of AMR, establishing surveillance of antimicrobial consumption, optimising antimicrobial use, establishment of a functional antimicrobial stewardship (AMS) team in all health facilities in Ghana and supporting sustainable investment in AMR reduction. The implementation of the NAP included, among others, the establishment of a functional AMS team in all health facilities in Ghana, which was lacking [12]. At the time of this study few hospitals in Ghana had progressed with establishing an AMS team or programme. The two hospitals involved in this partnership were keen to progress AMS, management support had been agreed and AMS team members identified.

The SAPG team (antimicrobial pharmacists, antimicrobial nurses, Infectious Disease Consultants and researchers from the University of Strathclyde) created a partnership with lead pharmacists in Ghana Police Hospital (GPH), Accra, and Keta Municipal Hospital (KMH), Volta Region, to support the development of antimicrobial stewardship. These lead pharmacists were supported by medical and nursing managers within their hospitals to provide leadership for a multi-professional AMS team. The project was also supported by health psychologists from The Change Exchange, who provided behavioural science strategies in assessing and changing influences on AMS behaviours [13].

The aim of the project was to develop and implement robust and reliable systems (accountability) and processes (practical tools) for antimicrobial stewardship in GPH and KMH by April 2020. This was to include establishing a local AMS team for each hospital, building capacity through provision of training sessions for a total of up to 25 professionals (medical, pharmacy, nursing and laboratory staff) in each hospital to deliver a local stewardship programme and a supported point prevalence survey (PPS) across each hospital to provide baseline surveillance data on antibiotic use to inform improvements. A simplified behaviour change wheel approach was taken to supplement the SAPG model. In this approach, behaviours are specified, the influences on behaviour are studied and these influences targeted in the intervention [14]. The SAPG triad approach to stewardship (Information, Education and Quality Improvement) was applied with behaviour change concepts incorporated throughout with the aim of developing a robust and crucially sustainable antimicrobial stewardship programme in each hospital.

#### **2. Results**

#### *2.1. Hospital AMS Teams*

In advance of the initial visit by the SAPG team the Ghanaian lead pharmacists with their hospital management team each convened a local multi-professional antimicrobial stewardship team to support the project and to ensure long term sustainability in antimicrobial stewardship. A standardised assessment of current stewardship was undertaken in both hospital using a tool developed by the Commonwealth Pharmacy Association (CPA). This identified gaps and informed discussions with the SAPG team during the initial visit. The local AMS teams (specialist doctors, pharmacists and nurses) established regular meetings and acted as champions to promote and engage all professional staff in antimicrobial stewardship. These teams also supported the lead pharmacists on all three elements of the project.

#### *2.2. Information*

For the initial PPS in May 2019 data were collected from prescription charts and patient notes by the Scotland/Ghana teams from all wards on a single day in each hospital utilising paper-based Global Point Prevalence Survey [15] methodology. Prescriptions were compared for compliance with available STG prior to data entry into the online Global PPS system. The overall prevalence of antibiotic use was 65.0% in GPH and 82.0% in KMH. Prevalence rates ranged from 46.7% to 100.0%, depending on the clinical specialty and patient population (Table 1). Penicillins and other beta-lactam antibiotics were the most prescribed antibiotics in both hospitals, with amoxicillin/clavulanic acid being the most commonly prescribed antibiotic.

**Table 1.** Prevalence of antibiotic use in Ghana hospitals compared with Africa data from Global PPS.


Some differences were observed in the quality indicators between the two hospitals (Table 2) however in both hospitals there was good documentation of the indication for antibiotic treatment compared with the benchmark level for African hospitals in the Global PPS. For some indications, guideline compliance could not be assessed especially for antibiotic use for surgical procedures as they were not included in the STG. Where a guideline was available, compliance with the choice of agent was ≥50% in both hospitals for both medical and surgical patients.

**Table 2.** Quality indicators for antibiotic use in Ghana hospitals compared with Africa data from Global PPS.


No treatment was observed to be based on microbiology data in GPH and were only used for one patient in KMH on the day of survey. Duration of surgical prophylaxis was typically more than one day (GPH 69.0%, KMH 77.0%) with no single dose prophylaxis in either hospital.

Data collection for a follow up PPS was carried out in February 2020 by the Ghanaian teams and results were discussed with the SAPG team. Online data entry and reporting was paused due to the COVID-19 pandemic and will be completed in due course.

#### *2.3. Education*

#### 2.3.1. Engagement

A total of 60 staff participated in a one day training session held across two days, delivered twice in each hospital by the SAPG team. Nurses made up the majority of participants (22, 36.7%) followed by medical doctors (10, 16.7%) and pharmacists (10, 16.7%). Laboratory scientists, hospital managers, midwives and a public health practitioner made up the remaining 30%.

Feedback forms on the SAPG training were completed by 48 of the 60 participants. Responses were positive with 39 participants rating the session as very good and 9 participants as good.

#### 2.3.2. Knowledge Evaluation

For the knowledge quiz in GPH the participant mean scores were: pre-training 9.2 (SD2.2, range 5–13) and post-training 11.1 (SD1.8) (range 8–13), and in KMH the mean scores were: pre-training 9.4 (SD1.8, range 5–13) and post-training 10.9 (SD1.4) (range 8–13). The mean difference between pre and post-training participant scores in GPH was 1.88 (95% CI 0.753 to 3.008) (*p* = 0.00002) and in KMH the mean difference between the scores was 1.57 (95% CI 0.93 to 2.21) (*p* = 0.00001).

In GPH, training was cascaded by the local AMS team to a total of 25 staff across one session. A total of 18 participants completed the knowledge quiz before a session and 8 participants fully completed it post-training. The mean pre-training score was 8.5/13 (range 6–12) and the mean post-training score was 9.3/13 (range 8–11). During the final visit by the SAPG team, 2 of the original training participants completed a further knowledge quiz (4 months after the training session), scoring 10 and 13 points and total of 8 staff (additional 6 people trained by GPH team) completed the knowledge quiz and scored a mean of 10/13 (range 8–13).

In KMH, training was cascaded by the local AMS team to a total of 144 staff over two training sessions. During the final visit by the SAPG team, 2 of the original training participants completed a further knowledge quiz, scoring 9 and 13 points and a total of 12 staff (who completed SAPG or KMH team training) completed the knowledge quiz and scored a mean of 10.5/13 (range 6–13).

#### 2.3.3. Attitudes and Behaviours Evaluation

Participants from both hospitals demonstrated improved attitudes and behaviours around use of antibiotics after the training session as shown in Tables 3 and 4. Attitudes and behaviours were similar across professional groups based on comments from the training sessions.

Sustained change in attitudes and behaviours were assessed during the final visit with the following findings: in GPH, staff agreed or strongly agreed with all but two of them having positive stewardship behaviours. Areas where some staff did not agree were the ease of adhering to guidelines and the need for peer support for adherence to guidelines. In KMH, a larger number of staff did not agree with these attitudes towards the guidelines and more staff said they could not access the guidelines.


**Table 3.** Pre and post education responses to survey questions by staff at GPH (Ghana Police Hospital).

GSTG—Ghana Standard Treatment Guidelines.



GSTG—Ghana Standard Treatment Guidelines.

#### *2.4. Quality Improvement*

In both hospitals access to guidelines and gaps in local guidance were identified by AMS teams as a key target for improvement. Local guidelines in poster format were developed in collaboration with clinical teams for display in wards and departments to ensure staff were aware of which antibiotics should be used for common infections seen among inpatients. Colour laminated copies of these posters were provided by the SAPG team during the final visit.

In GPH, the AMS team agreed a local action plan with a focus on introducing interns (doctors, pharmacists, nurses) to AMS and developing local guidelines for antibiotic prescribing for wound management, as well as obstetric pre- and post-delivery (Figure S2). Other actions included addressing the need for surveillance and analysis of laboratory antimicrobial data for common infections such as urinary tract infections, initiating routine collection and analysis of antimicrobial prescribing at the outpatient department. All findings were to be shared periodically at clinical meetings and with the drug and therapeutic committee, as well as publishing findings as appropriate.

In KMH the AMS team agreed a local action plan that focused on: rollout of AMS education to all staff; improving the adherence to the local treatment guideline on empirical management of pneumonia for ambulatory patients; and increasing patient awareness (Figure S3). Their long term goal was to create and locally adapt antibiotic policies for KMH. Progress has already been made towards these goals with over 144 staff trained in antimicrobial stewardship locally, patient education initiated in some pharmacy led clinics and an ongoing Quality Improvement project in the out patients department which to date has increased compliance with policy and reduced amoxicillin/clavulanic acid prescribing.

#### **3. Discussion**

Immense progress has been made with the establishment of a robust and sustainable stewardship in GPH and KMH as a result of this project. Through the expert team from SAPG and The Change Exchange providing practical support and guidance, the Ghanaian lead pharmacists have been able to lead their local AMS teams to gain experience and knowledge of the requirements for a successful AMS programme. Building successful relationships has been key to the success of the project and having a single profession, pharmacists, as leaders has been helpful to demonstrate behaviours and capabilities amongst peers [16]. This will support long-term engagement beyond the project to provide continued advice and guidance as the Ghana AMS programmes mature, as well as potentially supporting the spread of AMS to other hospitals in Ghana. Using a multi-professional approach along with behavior change techniques has also been crucial as stewardship needs to be owned by clinical teams and practiced by all staff to be reliable and sustainable [17].

Regarding the Information element of the project, we demonstrated that the PPS assessment was feasible in both hospitals and can be achieved with limited resources and minimal training of a multi-disciplinary team. Now that staff are familiar with the process, further repeats of PPS will take less time and we are hopeful that eventually direct electronic data collection may be possible to reduce data entry time. The use of repeated PPS is a well-recognised method for measuring both the quantity and quality of antibiotic prescribing where electronic medicine management systems are not available [14]. This will allow progress with improvement work to be tracked and smaller bespoke PPS can also be used to investigate prescribing practice in specific clinical areas or of specific antibiotics.

Our approach to the Education element of the project involved developing training collaboratively to ensure the content met the needs of local clinicians. Delivery of the education by a multi-professional team was successful in imparting knowledge, skills and positive behaviours to support improved use of antibiotics. Key behaviours identified by the psychologists during the first visit around supporting access to guidelines and responsibilities of all staff groups for querying prescriptions that do not follow the guidelines featured in the role play scenarios, giving staff a chance to practice promoted behaviours in a non-threatening way. Participants rated the training highly and the use of lectures and interactive sessions supported good engagement and involvement of everyone in discussion of the issues. The 'train the trainer' approach has been successful in building local capacity for provision of ongoing training in both hospitals and potentially beyond to other hospitals in these regions of Ghana. This was demonstrated by the capacity of the local team to train more staff as means of cascading the knowledge of the principles of AMS to untrained staff. The training materials used for the project have been made available via the Commonwealth Pharmacist Association (CPA) website and can be used by others to support similar work. Key learning from the one day sessions was that participants would

prefer lectures and interactive sessions to be interspersed rather than have all the lectures at the start. This would also help educators and participants to relax and get to know each other to make the most of the sessions.

The Quality Improvement element of the project was tailored to each hospital's priorities and ambitions based on the action plans agreed by the AMS teams. Both hospitals identified a need for improved access to guidelines so that staff without a smartphone to access the MicroGuide STG app could easily find the information required when prescribing or administering antibiotics. Locally designed posters proved a useful format and the SAPG team were able to produce a quantity of these for each hospital to support compliance with the guidelines across all wards and departments.

In GPH, the AMS team with obstetrics and gynaecology (OBG) and the surgical unit have developed their antibiotic guidelines for pre- and post-delivery and wound management, respectively. Furthermore, the guidelines for common infections seen at the OBG were developed with guidance from the STG. Currently, routine microbial antibiotic sensitivity data from the laboratory, as well as prescribing of antibiotics audits by the pharmacy department, are being done.

In KMH, weekly prescription analysis of compliance to empirical management of pneumonia of ambulatory patients by pharmacists showed an increasing change in behaviour towards the use of first line antibiotics, and work is ongoing.

Limitations of this project included the limited time spent in Ghana by the SAPG team and by the Ghana team in Scotland. With a large team of 10 experts from SAPG (5 for each hospital) and a limited budget, an intensive schedule was necessary to ensure all three elements of the work were delivered in each hospital. Reliance on email communication and some Skype/WhatsApp calls for discussion of the project was not ideal but in the current climate of virtual meetings and global collaboration that may be the way forward. Time for staff to work as volunteers on the project was also at times difficult to manage as all were full time employees with busy work schedules. A further limitation for the training element is potential bias in data collection as a clear protocol for mandatory participant completion of knowledge and behaviour surveys was not employed.

In the current context of the COVID-19 pandemic, future financial support for exchange visits to support development of AMS in low and middle income countries is unlikely and innovative virtual solutions will be a more feasible approach. There may also be merit in supporting the train the trainer approach employed in this study to spread local expertise for AMS to other Ghana hospitals.

#### **4. Materials and Methods**

#### *4.1. Study Design*

The study design was developed in late 2018 and detailed plans were progressed during March and April 2019 following the grant award. Implementation of the three elements of AMS was facilitated by exchange visits during a 9-month period from May 2019 to February 2020. There were three visits by the Scottish team and The Change Exchange to Ghana to support AMS implementation and one visit by the Ghanaian partners to Scotland to observe how AMS has been embedded at local and national level. Regular virtual meetings and email communication were used between visits to review progress, plan training sessions and agree actions. The study did not require ethics approval.

At the initial visit in May 2019 a small multi-professional group from SAPG visited both hospitals and supported data collection on antibiotic use for a baseline PPS using the Global PPS system. At this visit, in both hospitals, the health psychologists interviewed a variety of staff whose behaviours would impact on the use of antimicrobials. This included prescribers and dispensers. These discussions probed the behaviours that would support prescribers and dispensers to improve AMS and the barriers and facilitators to those behaviours.

On the second visit in September 2019, two separate multi-professional groups worked with Ghanaian Partners to provide 2 × 1-day 'train the trainer' education in each hospital following a training plan (Figure S1) informed by findings from the initial visit. Interactive training activities were developed using behaviour change principles, including a fun Antibiotic Guardian session where trainees pledged their commitment to AMS actions; an activity identifying barriers to changing practice and problem-solving potential solutions; generating action plans to initiate and maintain changes; role playing potentially difficult conversations with prescribers, patients and families; and practicing using the CwPAMS MicroGuide app to access antimicrobial guidelines. Local pharmacist-led antimicrobial teams agreed an action plan and a Quality Improvement (QI) project.

Local Ghana teams cascaded training to other staff and conducted a second PPS between October 2019 and February 2020.

On the 3rd visit by the SAPG team in February 2020, laminated guideline posters for each hospital were provided to increase access for all staff and progress with local action plans was discussed with the AMS teams to agree next steps. Health psychologists and nurses from the visiting team interviewed a range of healthcare staff at both hospitals to identify changes in AMS behaviours since the trainings and ongoing barriers.

#### *4.2. Practical Delivery of the Project*

The Global Point Prevalence Survey system [15] was used to collect, submit and generate reports on antibiotic use in each hospital.

Training sessions utilised Microsoft PowerPoint presentations and both plenary and small group workshop discussions. Some elements of the training were filmed using a smartphone camera as a record of AMS pledges made by staff. Staff who attended the training session received a signed certificate of participation. Training was evaluated to assess the change in knowledge and behaviours of participants before and after the session using paper forms. Participants were also asked to complete a paper-based feedback form about their perception of the training session. Participants were not asked for formal consent to use information they provided in the evaluation and feedback forms but consent was presumed from their participation in the training session.

Each facility was encouraged to identify a QI project to address the shortfalls identified in key quality indicators identified by the PPS.

#### **5. Conclusions**

Key objectives were achieved and a sustainable model for AMS was established in both hospitals. Support for spread of AMS at national level was discussed through partnership meetings with academics in the Medical and Pharmacy Schools, the Ministry of Health AMR lead and Pharmaceutical Society staff with commitment to ongoing collaboration. Overall, members of the SAPG team and the Ghanaian lead pharmacists learned much about each other's professional practice and countries' cultures which will remain important memories for all.

**Supplementary Materials:** The following are available online at http://www.mdpi.com/2079-6382/9/10/636/s1, Figure S1: Antimicrobial stewardship training plan—Ghana 2019, Figure S2: Ghana Police Hospital AMS Action Plan—2019/2020, Figure S3: Updated Action Plan Keta Hospital (February 2020).

**Author Contributions:** Conceptualization, J.S., D.A. and I.S.; methodology, J.S., A.C., F.K.; writing—original draft preparation, J.S.; writing—review and editing, D.A., I.S., A.C., F.K., L.B.-D., E.C.; funding acquisition, J.S., D.A., I.S. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was funded via a grant from the Commonwealth Partnerships on Antimicrobial Stewardship (CwPAMS) supported by Tropical Health and Education Trust (THET) and Commonwealth Pharmacists Association (CPA) using Official Development Assistance (ODA) funding, through the Department of Health and Social Care's Fleming Fund. The views expressed in this publication are those of the author(s) and not necessarily those of the NHS, the Fleming Fund, the Department of Health and Social Care, THET or CPA. UK partners involved were volunteers supported by their NHS host organisations.

**Acknowledgments:** We thank the members of the Healthcare Improvement Scotland and Ghana hospitals partnership for supporting development and delivery of the training sessions, and staff at Keta Municipal Hospital and Ghana Police Hospital for engaging with the sessions. Thanks to Jo Hart and Joanna Goldethorpe from the Change Exchange for their contributions to the project. Special thanks to Marion Pirie, SAPG Project Officer, for administrative support, organising all travel and budget management and to Lesley Cooper, SAPG Health Service Researcher, for help with literature evaluation and analysis of data.

**Conflicts of Interest:** The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

#### **References**


*Antibiotics* **2020**, *9*, 636

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

#### *Review*

#### **Knowledge, Attitudes and Perceptions of Medical Students on Antimicrobial Stewardship**

#### **Panagiotis Efthymiou , Despoina Gkentzi \* and Gabriel Dimitriou**

Department of Paediatrics, Patras Medical School, University of Patras, 26504 Rio Achaia, Greece; panosefth@upatras.gr (P.E.); gdim@upatras.gr (G.D.)

**\*** Correspondence: gkentzid@upatras.gr

Received: 14 October 2020; Accepted: 16 November 2020; Published: 17 November 2020

**Abstract:** Antimicrobial Resistance (AMR) is an ongoing threat to modern medicine throughout the world. The World Health Organisation has emphasized the importance of adequate and effective training of medical students in wise prescribing of antibiotics Furthermore, Antimicrobial Stewardship (AMS) has been recognized as a rapidly growing field in medicine that sets a goal of rational use of antibiotics in terms of dosing, duration of therapy and route of administration. We undertook the current review to systematically summarize and present the published data on the knowledge, attitudes and perceptions of medical students on AMS. We reviewed all studies published in English from 2007 to 2020. We found that although medical students recognize the problem of AMR, they lack basic knowledge regarding AMR. Incorporating novel and effective training methods on all aspects of AMS and AMR in the Medical Curricula worldwide is of paramount importance.

**Keywords:** knowledge; attitudes; medical students; antimicrobial; stewardship; prescribing; antibiotics

#### **1. Introduction**

Antimicrobial Resistance (AMR) is an ongoing threat to modern medicine throughout the world with a negative effect on patient treatment outcome. Pathogens are developing mechanisms of resistance, making it difficult to treat common infectious diseases like pneumonia, tuberculosis and foodborne diseases [1–4]. Antibiotic prescribing is determined by various factors, including the socio-cultural and socio-economic factors of each country and the beliefs of patients and professionals regarding antibiotic use [5,6]. In many developing countries, there is shortage of appropriate diagnostic tools, resulting in the unnecessary administration of antibiotics [7,8]. It has also been observed that the insufficient regulatory policies of each country can cause an increase in over-the-counter antibiotics [9]. The World Health Organisation (WHO) has clearly emphasized the importance of adequate and effective training of medical students in the wise prescribing of antibiotics [10]. In 2015, the WHO endorsed the Global Action Plan on Antimicrobial Resistance, which highlights the importance of training all healthcare professionals regarding AMS [11]. It is vital that healthcare students are aware of the challenges posed by AMR, and that there are investments in training them on topics relevant to responsible antibiotic use in their chosen specialties [11].

Thus, future medical professionals have to be prepared appropriately in order to face the challenges of antimicrobial use in everyday clinical practice [12]. Nowadays, medical education incorporates thorough knowledge of infectious diseases and diagnosis, as well as antibiotic utilization and pathogen resistance mechanisms. All the above-mentioned fields of knowledge are highly required for medical students [13]. Furthermore, Antimicrobial Stewardship (AMS) has been recognized as a rapidly growing field in medicine that sets a goal of rational use of antibiotics in terms of dosing, duration of therapy and route of administration [13–15].

Taking into consideration the importance of medical education on AMS and AMR, we undertook the current review to systematically summarize and present the published data on knowledge, attitudes and perceptions of medical students on AMS.

#### **2. Results**

In the present review, we included 25 studies. Fifteen of them were focused exclusively on medical students (Table 1), whereas ten were conducted on healthcare professional students with the inclusion of medical students amongst them (Table 2). All studies included final year medical students, three of them also included prefinal students, and three studies included medical students from all years of medical school. We found studies from all over the world in countries of Europe, America, Africa, Asia and Oceania. As for the studies focusing exclusively on medical students, five were conducted in Asia [16–20], three in Africa [21–23], one in Oceania [24], five in Europe [10,25–28] and finally one study in the USA [11]. All studies used questionnaires as research tools so they could collect data regarding the knowledge, attitudes, and perceptions of medical students about AMS. More precisely, questionnaires included questions on self-perceived preparedness to prescribe antibiotics and the importance of AMS. Knowledge regarding antibiotics was, in most cases, tested by clinical scenarios or general questions. The oldest study was conducted in 2012 and the latest in 2020. We also included 10 multidisciplinary studies (Table 2) that incorporated medical, pharmacy, dental, veterinary students, medical interns and physicians [29–38].



**Table 1.** *Cont*.

*Antibiotics* **2020**, *9*, 821


*Antibiotics* **2020**, *9*, 821

A study in a Japan University in Japan showed that 92.6% of their medical students had sufficient knowledge of the mechanism of actions of antibiotics. However, 30% of the respondents worryingly answered that antibiotics could be used to treat viral infections and around half of them believed that they could be administered for the common cold. We should point out that wrong answers were significantly lower among final year medical students compared to the first-year students. Lastly, only 6.5% of the students were aware of the AMR plan that was promoted by the Japanese government [18].

In 2015, a survey in eight universities from Australia showed that 70% of medical students felt more confident regarding their knowledge in cardiology compared to infectious diseases, where 54% were confident of their knowledge. As for clinical knowledge, students scored better in cardiovascular disease questions (64%) contrary to antibiotic prescribing questions (45%). In addition, nearly all students were aware of guidelines referring to antimicrobial prescribing. A negative aspect of this study was the small number of participants, which may not give an accurate perception of the country's undergraduate students [24].

Our research revealed two studies from China, a country with the second largest antibiotic consumption in the world and high rates of dispensable use of antibiotics. The majority (92%) of medical students from Central China agree that misuse of antibiotic treatment increases AMR, where 67% found their training useful in antimicrobial management. Once again, the percentage of correct answers in a broad spectrum of questions around the treatment of different infections was low, with the total correct percentage being 34% [16]. Another study from China showed that 27% of medical students reported self-medication with antibiotics. Additionally, 64% of the respondents reported that they were stocking antibiotics for personal use, whilst 97% have bought antibiotics without medical prescription in the past [17].

The studies that we found from Europe were conducted under the European Society of Clinical Microbiology and Infectious Diseases Study Group of Antimicrobial Policies (ESGAP/ESCMID). In 2012, seven European medical schools competed an online survey which reported that medical students felt more confident about infection diagnosis in comparison to treatment, e.g., administration route, duration, necessity of antimicrobial use. A high percentage of them (83%) believed that incidence of methicillin-resistant *Staphylococcus aureus* (MRSA) bacteraemias has increased in their country, which was not true. Moreover, medical students overestimated possible death rate resulting from resistant bacteria compared to lung cancer. Nearly all medical students (98%) believed that AMR would be a major issue in the foreseeable future [10]. In another study, we have more detailed results from France, where 64% of the respondents were aware of the implementation of antibiotic guidelines in their hospital and 62% have used them in practice. Furthermore, 94% believed that AMR is a national problem. It should be noted at this point that this study was limited due to the small number of participants [27]. A Spanish study showed that 40.4% of medical students claimed that they would prescribe an antibiotic without consulting guidelines. On the other hand, only 24.3% believed they had adequate training regarding rational use of antibiotics [26]. In 2015, 29 European countries participated in an online survey of ESGAP to assess the preparedness of medical students using antibiotics in a judicious way. Countries that incorporate guidelines in clinical practice, like the United Kingdom, reported a higher rate of preparedness. This conclusion can also be related to quality of education on the use of antibiotics. Hence, there was some variability between different European countries [25]. For example, a study comparing France and Sweden showed that French medical students were less likely to feel prepared compared to the Swedish students and they suggest a need for more focused education in the field. A hypothesis that originated from this study claims that this concept may come from the fact that Swedish medical students tend to have more clinical exposure to the approach of antibiotic management [28]. In general, 37.3% of medical students requested more education on this subject. This is one of the greatest studies considering the preparedness of medical students on a topic [25].

Another study from India revealed that medical students have insufficient knowledge about AMR and AMS (39.7% of prefinal and 54.8 of final year medical students answered correct). Most of the students knew that viral infections cannot be treated with antibiotics. However, many of them had did not know how to treat and prevent MRSA infections effectively and could not recall the acronym *Enterococcus faecium*, *Staphylococcus aureus*, *Klebsiella pneumoniae*, *Acinetobacter baumannii*, *Pseudomonas aeruginosa*, and *Enterobacter* species (ESKAPE) pathogens [19]. Additionally, a study from a medical school in Nigeria showed that 64.7% of the students had a good understanding of antibiotic resistance and use, although a mere 39% of them would treat common cold with antibiotics. Furthermore, only 8.2% of the students took medical consultation before taking antibiotics [21]. In a multi-centre, cross-sectional study in Egypt, 43% of the respondents considered that skipping doses of antibiotic treatment does not affect AMR, which is a common misconception. Around 40% of the students would use antimicrobials for a sore throat. Furthermore, students in final years performed better in the knowledge section of the questionnaire [23].

On a survey taking place in the USA in 2012, 90% of medical students from 3 universities (University of Miami, John Hopkins University, University of Washington) reported that they would like more education on antibiotics. More specifically, mean correct knowledge score was 51% which has been significantly affected by study sources that students used as learning methods. For instance, medical students scored high on questions regarding the use of antibiotics and management of community-acquired pneumonia. In contrast, they scored low in urinary tract infections and in questions for recognition of Clostridium difficile infection. Students who referred to physicians, pharmacists and those who used guidelines had better scores. Except from that only 15% of the students had followed a rotation in Infectious Disease department during their studies [12]. In addition, in a study from three medical schools in Thailand, 90% of students affirmed that the misuse of antibiotics is a major problem in their hospital and their country. The majority (98%) believed that they were capable of prescribing antibiotics, while 71.4% would feel stressed when prescribing. Over 10% of medical students reported that they have never been taught the principles of prudent use of antibiotics and AMR [20].

Last but not least, a study including three medical Schools in Africa showed that one out of three medical students did not feel confident enough on antibiotic prescribing [22]. On the other hand, students claimed that antibiotic overuse (63%) and resistance (61%) is an essential problem in their hospital, while 92% believed that antibiotics are overused and that the AMR is a crucial issue in the South African region. They also had the perception that hand hygiene is not a large contributing factor to AMR [22].

#### **3. Discussion**

In the current review, our main goal was to reveal if and how medical students were taught the basic principles of AMS. Nowadays, we are living in an era of excessive and often irrational usage of antibiotics in some settings, resulting in a significant increase in AMR. As this is an intercontinental issue, several studies conducted around the world give us valuable data for the preparation of future medical professionals. Our research has shown noteworthy findings regarding the knowledge, attitudes and perception of medical students.

#### *3.1. Knowledge*

It is clearly shown that medical students today lack adequate basic and clinical knowledge on the principal concepts of infectious diseases. In all parts of the world, medical students score relatively low in the relevant knowledge questionnaires as well as in the included clinical scenarios, making it clear that there is a room for improvement in medical education in both developed and developing countries [12,20,23,33]. Notwithstanding that medical students know that inappropriate use of antibiotics increases bacterial resistance, there are some common misperceptions [22]. For example, many of them claim that antibiotics can be used for treating viral infections and common flu [23,33]. It is widely known that those practices can increase inappropriate antimicrobial usage, resulting in high rates of AMR. In contrast to the above, diagnosis seems to be an easier task for medical students [10]. These knowledge gaps can occur due to the ineffective curriculums of each Medical School, which do not thoroughly cover the fundamentals of antibiotic usage, management, and duration of treatment, although the situation varies among different countries [39]. However, students who were in their final years of their studies in Medicine or had completed a clinical rotation in the Infectious Diseases Department of their hospital scored better in the knowledge section [16,18,33]. On the other hand, students who followed guidelines and reliable sources for learning tend to have more structured knowledge [34]. This finding implicates the importance of guidelines in undergraduate medical education in the early stage of the curricula. In any case, in all studies, medical students would appreciate more education in both basic science as well as clinical grounds so that they feel better prepared in their future tasks of everyday clinical practice [10,12,16,18,19,22,31,35,36,39]. It is of interest to note that, overall, following the WHO Global Action Plan on Antimicrobial Resistance [11], medical education about AMS might be essentially different in individual countries. Although we observed a trend of better knowledge in the field from 2015 onwards, we cannot accurately assess this difference as we have no country-specific data on educational changes and their potential impact. In addition, none of the included studies were designed to address this particular question, ideally comparing knowledge and attitudes on AMR with the same research tool and in the same setting before and after the WHO Global Action Plan on AMR.

#### *3.2. Attitudes and Perceptions*

Medical students do believe that rational use of antimicrobials is an essential aspect of their career to avoid the spread of AMR among pathogens [10,12,19,20,31]. These findings indicate that medical students and tomorrow's doctors have a positive moral attitude towards this issue. They also believe that antibiotics should not be sold and administered without a medical prescription [17,19]. The recognition of AMR's severity is the major factor that will guide and assist professionals in searching for efficacious ways of fighting mild as well as more severe infectious diseases. Although most students assess that antimicrobial overuse and resistance are worldwide issues, they tend to underestimate that this problem also exists in their hospital environment [12,16,40]. Such behaviors can lead to improper prescribing. Another interesting point is that a large percentage of medical students lack confidence and preparedness referred to antibiotic prescription [20,34,35]. This can correlate with each country's AMR status, where students in countries with low resistance rates tend to feel more prepared, possibly because they are less exposed to severe infectious clinical challenges [36]. On the other hand, evidence exists that overconfidence in the field had a negative impact on antibiotic prescription [22,35].

#### *3.3. Limitations of the Study*

To the best of our knowledge, this is the first review that summarizes the relevant studies worldwide on AMS and medical students. However, our review has some limitations regarding data evaluation. First of all, medical students in the reviewed studies were questioned around different aspects of AMS, making it difficult to proceed to an equal assessment of the results. In addition to that, no international validated questionnaire exists in the field, which would obviously make the results of the studies more comparable and could be used for future studies. Moreover, different antibiotic policies and guidelines are in place in each country, and hence different behaviors and attitudes can be described. We should also take into consideration that learning and training resources do vary around the globe due to social and economic conditions. This can obviously have an impact on medical education strategies on AMS. Finally, depending on countries, the status of AMR is different, which may be related to medical education, but we do not have the data to proceed to such a comparison between countries.

#### *3.4. Implications for Future Approach*

With all above-mentioned issues, we understand the need to enhance medical education towards AMS. The majority of students worldwide consider traditional lectures and passive learning tools ineffective methods or an unsuccessful way of promoting knowledge on antibiotics and AMS [11,21,36,41]. Therefore, a more practical approach such as discussion of clinical scenarios and presence in clinical practice (e.g., clinical placements, clinical rotations [12,16,38]) seems to have a positive effect on knowledge of and attitude toward antibiotic usage and administration [20,35]. In addition, a more detailed teaching of basic microbiology knowledge could reinforce medical students with important information, which is necessary for a better understanding of antibiotics [42]. Another useful intervention would be cooperation between medical universities in order to exchange educational approaches, and also between medical schools and faculties like pharmacy schools (e.g., interprofessional workshops and simulations between medical and pharmacy students), so they can learn more about the uses and specific features of antibiotics, by introducing principles and the importance of AMR [43–45]. As far as time organization is concerned, curricula studies have shown that early introduction of AMR teaching as well as repetitive and enriched training upon antibiotic resistance, diagnosis, management, prescribing and communication skills would lead to a more comprehensive understanding of the challenges and complexities of infectious diseases [43,46–48]. Moreover, e-learning and online education about AMS is a desirable and effective method according to medical professionals and students [41,49], although a European survey questions the successfulness of this means [35,50]. Furthermore, a study based on a seminar for medical students included real patients and their advocates. Post this seminar, students believed that hearing patients' stories is an effective way of learning more about AMR and the importance of stewardship [51]. There should also be a change regarding the learning tools and resources which medical students study during their years of medical school. Of note, students who follow the updated guidelines [52] and those who referred to medical and pharmacy specialists tended to have a more completed and updated knowledge on AMS [12]. Another helpful implication would be to encourage medical students to get involved in undergraduate research to acquire new academic skills and be aware of both AMR and AMS [53]. However, there is need for further future assessment of current medical training methods so we can make clear assumptions about their effectiveness [54].

#### **4. Materials and Methods**

We reviewed all studies published in English from 2007 to 2020. The studies were included if they contained original results or had exceptional content with particular emphasis on studies Knowledge, Attitude, Perceptions (KAP) studies.

The initial search was conducted in the PUBMED and Scopus databases and the last search was performed on 1 September 2020. The following key words and their combinations were used for the search: antimicrobial stewardship\*, medical students\*, knowledge\*, attitudes\*, perceptions\*. Duplicate publications were identified and removed.

We identified 160 potentially relevant articles through database searches. Of these, there were 48 duplicates and 70 were excluded on the basis of title and abstract screening. We also excluded studies that investigated training methods for promoting AMS. Hence, only studies with quantitative results regarding the knowledge and attitudes of medical students regarding AMS were included. We focused on medical students and no other health-related undergraduate students (e.g., pharmacy or dental students). Studies on the knowledge and attitudes of other healthcare professionals or students (i.e., non-medical students) are listed as Supplementary Materials (Table S1). Moreover, protocols and editorials were excluded. Figure 1 describes the details of the methodology and excluded studies.

**Figure 1.** Flowchart of methodology and included studies.

#### **5. Conclusions**

Education on AMS is an emerging fundamental value for medicine around the world due to rapidly increasing AMR. Today's medical professionals will hand over the baton to medical students and hope for a greater improvement in AMR and antibiotic usage. Although medical students recognize the imminent issue of excessive resistance, they also lack basic knowledge regarding AMR. Consequently, at this time we should provide knowledge and confidence to medical students so that they will be able to face ongoing daily clinical challenges in the future. This could be achieved by incorporating novel and effective training methods on all aspects of AMS and AMR in the medical curricula worldwide.

**Supplementary Materials:** The following are available online at http://www.mdpi.com/2079-6382/9/11/821/s1, Table S1: Studies on knowledge and attitudes of other healthcare professionals and non-medical students.

**Author Contributions:** Conceptualization, D.G. and G.D.; methodology, D.G.; software, D.G., P.E.; validation, D.G., P.E. and G.D.; formal analysis, P.E. and D.G.; investigation, D.G. and P.E.; resources, D.G. and P.E.; data curation, D.G. and P.E.; writing—original draft preparation, P.E.; writing—review and editing, D.G. and G.D.; visualization, P.E., D.G., and G.D.; supervision, D.G. and G.D.; project administration, D.G. and G.D. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

*Article*

#### **Improving Access to Antimicrobial Prescribing Guidelines in 4 African Countries: Development and Pilot Implementation of an App and Cross-Sectional Assessment of Attitudes and Behaviour Survey of Healthcare Workers and Patients**

**Omotayo Olaoye <sup>1</sup> , Chloe Tuck <sup>1</sup> , Wei Ping Khor <sup>1</sup> , Roisin McMenamin <sup>1</sup> , Luke Hudson <sup>2</sup> , Mike Northall <sup>2</sup> , Edwin Panford-Quainoo <sup>3</sup> , Derrick Mawuena Asima <sup>4</sup> and Diane Ashiru-Oredope 1,\***


Received: 4 August 2020; Accepted: 26 August 2020; Published: 29 August 2020

**Abstract:** Smartphone apps have proven to be an effective and acceptable resource for accessing information on antimicrobial prescribing. The purpose of the study is to highlight the development and implementation of a smartphone/mobile app (app) for antimicrobial prescribing guidelines (the Commonwealth Partnerships for Antimicrobial Stewardship—CwPAMS App) in Ghana, Tanzania, Uganda and Zambia and to evaluate patients' and healthcare providers' perspectives on the use of the App in one of the participating institutions. Two structured cross-sectional questionnaires containing Likert scale, multiple-choice, and open-ended questions were issued to patients and healthcare workers six months after the introduction of the app at one of the hospital sites. Metrics of the use of the app for a one-year period were also obtained. Download and use of the app peaked between September and November 2019 with pharmacists accounting for the profession that the most frequently accessed the app. More than half of the responding patients had a positive attitude to the use of the app by health professionals. Results also revealed that more than 80% of health care workers who had used the CwPAMS App were comfortable using a smartphone/mobile device on a ward round, considered the app very useful, and found it to improve their awareness of antimicrobial stewardship, including documentation of the indication and duration for antimicrobials on the drug chart. It also encouraged pharmacists and nurses to challenge inappropriate antimicrobial prescribing. Overall, our findings suggest that its use as a guide to antimicrobial prescribing sparked positive responses from patients and health professionals. Further studies will be useful in identifying the long-term consequences of the use of the CwPAMS App and scope to implement in other settings, in order to guide future innovations and wider use.

**Keywords:** CwPAMS App; smartphone apps; antimicrobial prescribing; pharmacy

#### **1. Introduction**

Antimicrobial stewardship programs in hospitals are focused on optimising antimicrobial prescribing to improve individual patient care, decrease healthcare costs and combat antimicrobial resistance [1]. The availability of accurate and up-to-date information is important to guide the right diagnosis and prescription of antimicrobials. Healthcare providers' attempts to access this information are influenced by previous training, availability of the information, ability to access and leverage technology [2]. There has been a recent rise in the use of smartphones generally across global population and it is predicted to be rising fastest in Africa. There has been increased development of smartphone apps designed for use in healthcare, including in the area of antimicrobial stewardship [3–7]. Current research in medicine has shown that the use of mobile phones and devices in medical settings is more popular and is increasingly being brought to the fore of international research [1]. For instance, recent studies have shown that 52% of smartphone users access medical information through their devices [8]. A study by Kamerow, Chief scientist and Associate editor for the British Medical Journal, revealed that there are approximately 100,000 health-oriented smartphone apps and, by the year 2015, over 500 million smartphone owners worldwide will use these apps [9]. The study also highlighted that, although designed for health professionals, around 15% of health apps are now marketed to patients to help them monitor, evaluate, and transmit medical data such as blood pressure and body weight among other health checks [9]. The author also stated that the use of these apps was higher amongst the younger population, females, and people who earned a higher income. Similarly, results from a longitudinal study of 206 medical doctors working at Hannover Medical School, Germany in the summer of 2012 and spring of 2014 also revealed a rapid increase in the use of mobile devices in medical settings during patient interaction and professional collaboration [10]. This significant increase was observed in both the frequency of use and the expansion of the areas of application of these devices. Smartphones have specific features that support their increasing use in healthcare delivery and behavioural interventions. They are highly portable, more convenient, cost-effective and interconnected compared to reference books and computers, thus promoting improved communication and the sharing of knowledge, data and resources among health professionals and as well as facilitating regular updates as new data becomes available [11–13]. Furthermore, the ability of smartphones to use internal sensors to deduce context including emotions, location and activity has greatly increased their relevance in the consistent monitoring and tracking of health-related behaviours and healthcare delivery [14–19]. In the early days of their use, there was a significant paucity of academic research on users' viewpoints and experiences with the use of these apps. The recent literature has provided positive feedback on the acceptability and workability of smartphone apps although it has also been recognized that this evolving technology may raise concerns regarding privacy and security [17,20]. In the past decade, there has been a rapid increase in the use of mobile phones in Africa [18]. There has also been a rapid integration of mobile health technologies and telecommunication into the healthcare system, especially in low and middle-income countries. In addition to this there has been an increased investment in mobile healthcare interventions including the use of these technologies for behavioural change communication [19]. With the increasing burden of communicable and non-communicable diseases in Africa, low-cost mobile health technology has the potential to make healthcare more accessible to disadvantaged communities [21]. For example, in Zambia and Ghana adverse event reporting apps were developed by medical regulatory authorities in 2019 [22]. The Zambia Medicines Regulatory Authority—ZAMRA also launched Adverse Drug Reaction Application (ADRA), a new mobile application for android phone users for reporting adverse medicines reactions in 2017 [23]. Furthermore, apps have been used to identify falsified and substandard medicines in Kenya [24]. These technologies also offer great solutions aimed at improving the speed, safety and quality of healthcare provision in resource-constrained settings by providing easy access to local and international guidelines and resources. The purpose of the study is to highlight the development and implementation of an app to support prudent antimicrobial prescribing and improved antimicrobial stewardship practice; as part of the Commonwealth Partnerships for Antimicrobial Stewardship (CwPAMS) programme in Ghana, Tanzania, Uganda and Zambia and to conduct a pilot

study assessing patients and healthcare providers' perspectives on the use of the app in one of the hospitals in Ghana.

#### **2. Results**

#### *2.1. App Metrics 1 Year from Launch (April 2019–May 2020)*

The Commonwealth Partnerships for Antimicrobial Stewardship App was developed to improve antimicrobial prescribing and stewardship practices among health professionals in Ghana, Tanzania, Uganda and Zambia. The app provides, for the first time in the four countries, easy access to infection management resources to improve appropriate use of antimicrobials in line with national and international guidelines. Following the launch of the app in four countries, there were 530 downloads of the app and 2,795 guide opens within 12 months. Ghana had more page hits (50.3%) than Uganda (31%), Tanzania (13%), Zambia (1.9%) and others (3.8%) (Table 1). The most visited section of the app was the National Prescribing Guidelines, accounting for 66.1% of the total number of page hits while the section for Updates on antimicrobial resistance (AMR) (coming soon) was the least visited (0.7%). Pharmacists (51.1%) and nurses (20.4%) accounted for the highest number of registered users while pharmacists (64.1%) and medical doctors (20.3%) had the highest frequency of downloads and guide opens (Table 2).




348



#### *2.2. Cross-Sectional Survey Studies*

A cross-sectional attitude and behaviour survey was carried out on patients and healthcare professionals to determine their attitudes/views on the use of antimicrobial prescribing guidelines by health professionals. A total of 47 patients and 38 health professionals participated in the survey; response rates were 51% and 38%, respectively.

#### 2.2.1. Demographics

Demographics presented in Table 3 shows that respondents comprise various age groups and educational qualifications and professions.


**Table 3.** Demographics of patients and healthcare professionals.

#### 2.2.2. Patients

Patients' Responses to the Use of Smartphone Mobile Apps in Healthcare Delivery

Patients' views on the use of the app by health professionals obtained using a Likert scale of five options (Strongly agree, Agree, Neutral, Disagree and Strongly disagree) are presented in Table 4. More than 50% of patients had a positive attitude to the use of smartphone apps by health professionals and the fact that it increases the quality of healthcare offered by health professionals and quickens access to healthcare. Patients' greatest concern was that the use of smart phone mobile apps in healthcare delivery could be a distraction to healthcare provision. This was followed by concerns that their data may not be protected/secure and that mobile devices may not be technically reliable enough. Patients' least concern was that the health professional "may not be competent enough".


**Table 4.** Patients' responses to the use of smart phone mobile apps in healthcare delivery—5-point Likert scale from Strongly disagree (1) to Strongly agree (5) (n = 47).

Patients' Concerns with Their Health Professionals' Use of Smartphone Apps by Age and Education

The highest proportion of patients who had no concerns with their use of smartphone apps by health professionals were aged 26–35 (71.4%). This was followed by patients aged 68 and above (66.7%), 18–25 (64.0%), 56–67 (60.0%) and 46–55 (33.3%) in descending order. Patients aged 36–45 had concerns with health professionals' use of smartphone apps. Patients with the most concern with health professionals' use of smartphones were aged 46-55. With respect to patients' highest level of education, patients with tertiary education (63.2% had the least concern with health professionals' use of smartphone apps while patients with basic primary education (25%) had the most concern.

#### Patients' Preferences for Health Professionals' Use to Access Medicines Information

The highest proportion of patients wanted health professionals to use a computer or laptop (38.3%). This was followed by smartphone mobile apps (23.4%), reference books (6.4%) and tablets (6.4%) in descending order. A computer/laptop/reference book was preferred by 6.4% of patients while 2.1% preferred any of a smartphone, computer/laptop or tablet, a smartphone, computer/laptop or reference book, a smartphone or tablet, and a computer/laptop, reference book or tablet. Additionally, 10.6% of patients had no preference (n = 47).

#### 2.2.3. Healthcare Workers

#### Use of the CwPAMS App and Other Sources of Information

Thirty-eight healthcare workers (HCWs) comprising of four doctors, eighteen nurses, six pharmacists and ten other healthcare workers participated in the survey. On a daily basis, mobile phones (28.9%) and printed posters (13.2%) were most predominantly used by the HCWs, while tablets and computers (7.9% each) were the least used devices (Table 5). Mobile phones were used more than once a day by 60.5% of healthcare workers. Percentages of healthcare workers who had never used a tablet, pocketbook, printed posters and computers were 47.4%, 28.9%, 26.3% and 21.0%, respectively. Healthcare workers' responses showed that many respondents had not consulted the CwPAMS App for antimicrobial prescribing information. The British National Formulary

(BNF)/National guidelines, a printed copy of standard treatment guidelines, senior colleagues and junior doctors were mostly consulted daily. In descending order, internet search engines, senior colleagues and pharmacists were consulted more than once a day. No additional source of information on antimicrobial prescribing was mentioned.


**Table 5.** Use of the Commonwealth Partnerships for Antimicrobial Stewardship (CwPAMS) App and other sources of information by healthcare workers (n = 38).

Use of the CwPAMS App and Other Sources of Information on Antimicrobial Prescribing by Profession

An assessment of the various sources of information on antimicrobial prescribing used by healthcare workers showed that the CwPAMS App was mostly used by nurses and other health workers. BNF and National guidelines were mostly used by doctors (100%) and pharmacists (66.7%) and least used by nurses (33.3%). Internet search engines were mostly used by pharmacists (100%) and least used by doctors (25%) (See Figure 1). Pharmacists were seen to refer to their senior colleagues for antibiotic information more than doctors, nurses and other health professionals. More doctors and other healthcare workers (midwives, dispensing technicians and medication counter assistants) sought information from pharmacists than nurses. Printed copies of the standard treatment guidelines were mostly used by pharmacists and least used by nurses.

treatment guidelines were mostly used by pharmacists and least used by nurses.

**Figure 1.** Use of the CwPAMS App and other sources of information on Antimicrobial prescribing by profession. **Figure 1.** Use of the CwPAMS App and other sources of information on Antimicrobial prescribing by profession.

Assessment of Standard Treatment Guidelines and Drug Resistant Infections Assessment of Standard Treatment Guidelines and Drug Resistant Infections

All responding healthcare practitioners admitted being concerned about the emergence of drug resistant infections while 79.0% agreed or strongly agreed that these guidelines are easy to access. A total of 44.7% stated that they preferred their senior's preferences over standard treatment guidelines. Only 18.5% preferred to use non-standard treatment guidelines for antimicrobial prescribing while 13.2% felt the standard treatment guidelines did not apply to their patients (Table 6). All responding healthcare practitioners admitted being concerned about the emergence of drug resistant infections while 79.0% agreed or strongly agreed that these guidelines are easy to access. A total of 44.7% stated that they preferred their senior's preferences over standard treatment guidelines. Only 18.5% preferred to use non-standard treatment guidelines for antimicrobial prescribing while 13.2% felt the standard treatment guidelines did not apply to their patients (Table 6).

**Table 6.** Assessment of Standard treatment guidelines and drug resistant Infections on a 5-point Likert scale from Strongly Agree (1) to Strongly Disagree (5), median response category for each question marked in bold. **Table 6.** Assessment of Standard treatment guidelines and drug resistant Infections on a 5-point Likert scale from Strongly Agree (1) to Strongly Disagree (5), median response category for each question marked in bold.


guidelines to guide my antimicrobial 5.3 13.2 7.9 **36.8** 28.9 Perception and Assessment of the CwPAMS Smartphone App

prescribing I am concerned about the emergence of drug-resistant infections **50** 44.7 0 0 0 All healthcare workers who had used the App agreed that the app was very useful, relevant to their patient population and considered it the best way to access standard antimicrobial treatment guidelines. In addition, they all felt comfortable using a smartphone on a ward round, admitting that the app increased their awareness of antimicrobial stewardship and encouraged them to challenge inappropriate prescribing and to document the indication and duration for antimicrobials on the drug chart. Furthermore, participants found the country-specific standard treatment guidelines most useful. This was followed by the WHO Essential Medicines list section and the Antimicrobial Stewardship (AMS) resource section.

#### **3. Discussion**

#### *3.1. CwPAMS App Metrics*

Analysis of the CwPAMS App metrics revealed that the months with the highest downloads and page hits were September, October and November. The increase in September and October can be largely attributed to partnership project visits and antimicrobial stewardship interventions in all four countries. The spike in the month of November can most likely be linked to events during the World Antibiotic Awareness Week in all four countries as well as the app promotion by the Commonwealth Pharmacists Association during the World Antibiotic Awareness Week. Pharmacists accounted for the highest number of registered users and had more page hits and downloads than other health care professionals and workers. While this could mean that the app is more common among pharmacy teams, it calls for increased app promotion among doctors and other health professionals, who have also begun to use the app. The variations in the number of page hits and app downloads in each country can be explained by the number of partnerships in per country as Ghana and Uganda had the highest number of partnerships while Tanzania and Zambia had the lowest number of partnerships.

#### *3.2. Cross-Sectional Survey*

The use of smartphone mobile apps in healthcare delivery has gained acceptance over the years among patients and health professionals in sub-Saharan Africa and worldwide [19]. The CwPAMS App was developed by the Commonwealth Pharmacists Association to provide easy access to medicine management information for health professionals across Ghana, Tanzania, Uganda and Zambia. In addition to providing health professionals with relevant national and international guidelines, notable advantages of the app are its usability without internet access, a feature which suits low and middle-income countries, and its easy adaptability. Most recently, the app was updated to provide health care professionals across the commonwealth with links to relevant country-specific and international resources on COVID-19 from the World Health Organization (WHO), International Pharmaceutical Federation (FIP) and the Africa Centres for Disease Control and Prevention, among other relevant sources. The pilot study showed that more than 50% of patients were content with their health professional's use of smartphone apps while attending to them. Age and education level had an impact on the patient's acceptance of smartphone mobile technology as middle-aged patients had the least acceptance while the young and the most elderly had the greatest acceptance. Patients with tertiary education had the highest acceptance for these technologies while those with basic primary education had the least acceptance. These results correlate with a study carried out in 2014 on the acceptance and use of health technology by community-dwelling elders which revealed that income, education and age were found to significantly affect the acceptance of technology in healthcare. Patients with higher education and income used the internet at rates close to or exceeding the general population [25]. Another study also revealed that the acceptance of mobile phone technology among the older population was on the increase as they were found to constitute the fastest-growing group using the internet and computers [26]. Regarding patients' preferences, our survey reveals that more patients preferred their health professionals using a computer/laptop to access information over a smartphone or reference book. This can be explained by the fact that the patients' greatest concern was that smartphones could be a distraction to healthcare provision. This concern corroborates findings from a study by Wu et al. which revealed that on an average, physicians' smartphones received 21.9 emails and 6.4 telephone calls, sent out 6.9 emails and initiated 8.3 telephone calls within 24 h. The study also revealed that 55.6% of 439 perfusionists admitted that they had used a

cellular phone for purposes other than healthcare delivery while performing their duties [27]. On the contrary, a cross-sectional survey of adult patients in metropolitan academic and private dermatological clinics carried out in 2015 revealed that most patients (69.7%) considered personal smartphones an acceptable reference tool to provide information in patient care [28]. To access medical information more than once a day, health care workers mostly use mobile phones (60.5%) and printed posters (15.8%). These sources were also the most predominantly used daily (28.9% and 13.2%), respectively. This supports previous studies which have highlighted an increase in the use of smartphone mobile apps by health professionals [3–5]. Healthcare workers were also found to mostly consult internet search engines (50%), senior colleagues (36.8%) and pharmacists (31.6%) to access antibiotic prescribing information more than once a day. This demonstrates the need to involve these groups in promoting the app as they have a significant influence on antibiotic prescribing behaviours and healthcare workers' decisions. Furthermore, healthcare professionals' responses to the use of the CwPAMS App was found to correspond with results obtained from a similar study by Panesar et al. involving 146 healthcare professionals. Both studies show that the health professionals found apps useful and relevant to their patient population. They also agreed that apps encouraged them to challenge inappropriate prescribing [6]. The concern displayed by healthcare workers for the emergence of drug-resistant infections and the use of the standard treatment guidelines as seen in Table 6 was highly impressive. Healthcare workers also found the country-specific section of the CwPAMS App most useful. This correlates with the app metrics from all four countries which revealed that the National prescribing guidelines had the highest number of page hits from May 2019 to May 2020. The study highlights the need for more healthcare workers, especially doctors, to use the CwPAMS App as app metrics and the pilot cross-sectional survey both reveal that more nurses and pharmacists than doctors had used the app. There is also the need for more focused implementation as well as app promotion at all partnership sites and among all health professionals, especially doctors who are prescribers. Furthermore, there may be a need for subsequent studies to be carried out within the hospital when a higher number of healthcare professionals have used the app, in order to have a broader perspective from patients and health professionals. It would also be important to incorporate regular reminders about the app into the implementation strategy. A recently published study by Lester et al. [29] highlighted that implementing a locally appropriate, pragmatic antibiotic guideline through an app, supported by a simple educational strategy of weekly 'reminders', led to a significant reduction in third generation cephalosporin usage as well as an increase in the proportion of 48-h antibiotic reviews.

#### *3.3. Strengths and Limitations*

The CwPAMS Microguide antimicrobial prescribing app is the first of its kind to combine country-specific and international guidelines and information on antimicrobial prescribing for Ghana, Tanzania, Uganda and Zambia. Hence, based on our knowledge, this study on the development, implementation and use of the app in these four countries is novel. One of the limitations is the low sample size for the surveys, which was due to the time constraint in carrying out the survey, limited time spent by patients at the waiting room of a single hospital site and health care workers' busy schedules. However, it is important to note that this section of the full study was intended to be a pilot in one setting and to provide initial descriptive findings. Extensive surveying across other sites would enable a test of significance and to confirm trends. In addition, the survey encompassed a wide range of health care workers, including doctors, pharmacists, nurses, midwives and other health care workers. Patients' who participated where across a broad range with respect to age and education, providing a wide perspective. The response rate was greater for patients than health professionals, most likely because patients were available to fill questionnaires whilst in waiting rooms compared to health professionals. The proportion of healthcare workers groups that responded to the survey were not comparable. This is due to more nurses and other health care workers being available in the hospital compared to doctors and pharmacists. Though not all healthcare workers had used the app, there was an 85.7% response rate from those who had used the app to questions on the use of the

App. Frequent updates and increased use of the app by health care workers highlight the need for further studies.

#### **4. Materials and Methods**

#### *4.1. Development of the App*

The CwPAMS App was developed by the Commonwealth Pharmacists Association using the MicroGuide platform (http://www.microguide.eu). The platform provides a cloud-based service that allows local pharmacists to develop, manage, update and publish clinical guidelines to various apps for any mobile operating system including iOS (Apple, Cupertino, CA, USA), Android (Google, Mountain View, CA, USA), Windows devices (Microsoft, Redmond, WA, USA) among other operating systems. It offers healthcare professionals offline access to clinical guidelines and content autonomously managed by pharmacy teams. It is also available online via https://viewer.microguide.global/CPA/CWPAMS. The CwPAMS App contains national and international guidelines listed into various sections including the WHO Essential Medicines List, surveillance tools, antimicrobial stewardship training, Infection Prevention and Control (IPC) resources, and country-specific Standard Treatment Guidelines. The App metrics and statistics were derived from routine data collection by Horizon Strategic Partners.

#### *4.2. Study Site*

The CwPAMS App was developed for use by 14 secondary care institutions that were part of the CwPAMS programme in four countries Ghana, Tanzania, Uganda and Zambia (S1–S3, Video S1). One of the hospitals in the partnership was used as the pilot study site. The hospital is a secondary health facility with a 100-bed capacity.

CwPAMS is a health partnership programme funded by the UK Department of Health and Social Care's Fleming Fund to tackle antimicrobial resistance (AMR) globally. CwPAMS will support partnerships between the UK NHS and institutions in Ghana, Tanzania, Uganda and Zambia to work together on AMS initiatives. This aims to enhance implementation of protocols and evidenced based decision making to support antimicrobial prescribing, as well as capacity for antimicrobial surveillance. Further information about CwPAMS is available via https://commonwealthpharmacy. org/commonwealth-partnerships-for-antimicrobial-stewardship/. CwPAMS is being run by the Commonwealth Pharmacists Association (CPA) and Tropical Health Education Trust (THET).

#### *4.3. Study Design*

The CwPAMS App metrics were obtained from data collected by the Horizon Strategic Partners. These assessed the frequency of page hits, guide opens and the number of registered users and downloads. The pilot study was a cross-sectional survey with patients and healthcare workers in one of the hospital sites, six months after the introduction of the App using questionnaires adapted from Panesar et al. [6]. Patients' questionnaires comprised of four sections with eight questions using a Likert scale and multiple-choice questions. The first section comprised of demographics including age, gender, highest education qualification and occupation. The second section assessed patients' attitudes to health professionals' use of smart phone mobile apps in healthcare delivery. The third section was designed to obtain patients' concerns about the use of these smart phone apps, while the last section requested patients' preferences for health professionals reference ranging from a smart phone mobile app to a tablet, computer/laptop and a reference book. The health care workers' questionnaires comprised of nine sections with 15 questions designed as a Likert scale and open-ended questions. The first section obtained healthcare workers' demographics including country, specialty, year of graduation, grade, type of institution and profession and role. The eight sections following comprised of health professionals' attitudes to the use of the CwPAMS App and current practices.

#### *4.4. Sample Size Determination*/*Sample Size and Sampling Technique*

A convenience sample size determination of maximum 100 each was used for the cross-sectional study.

#### *4.5. Procedure for Data Collection*

#### 4.5.1. CwPAMS App metrics

App metrics for user engagement evaluating the number of registered users, downloads, guide opens and page hits for various sections of the App from April 2019 to May 2020 were obtained through the MicroGuide platform. (http://www.microguide.eu).

#### 4.5.2. Pilot Cross-Sectional Survey

Health Professionals Survey: Questionnaires were distributed among healthcare workers comprising of doctors, pharmacists, nurses and other healthcare workers at various points of care in the hospital including consulting rooms, nurses' station, pharmacy sections and wards. A total of 100 questionnaires were distributed to health professionals with 38 returned questionnaires completed anonymously.

Patients Survey: Patients' questionnaires were distributed to patients in the waiting room within the consulting area. Patients' questionnaires comprised of demographic data and questions regarding attitude to the use of smartphone apps among health professionals over a one-week period. Patients' consent was sought for before administration of the questionnaires. A total of 93 questionnaires were distributed to patients based on patients available in hospital during the study period. All 47 questionnaires (S4: Questionnaires) were completed anonymously with no personally identifiable information documented.

#### 4.5.3. Study Approval

Study was conducted under service improvement as part of the CwPAMS project therefore no ethical approval was required but the Ghana Health Service and the Ghana AMR Platform were made aware of the pilot project.

#### *4.6. Procedure for Data Analysis*

Microsoft Excel 2013 was used to analyse the data obtained from the pilot study using descriptive statistics.

#### **5. Conclusions**

Our study provides insight into the overall perception of the use of mobile apps as a means to improve antimicrobial stewardship, demonstrating general acceptance among patients and healthcare professionals. In general, the patients and healthcare workers surveyed had a positive attitude following the introduction of the CwPAMS App as a fundamental resource for accessing information on antimicrobial prescribing. Hence, increased and more comprehensive use of all sections of the App could contribute to improved antimicrobial stewardship practices among healthcare workers and increased acceptance of the use of smartphone apps among patients. App downloads and utilization were found to be highest during partnership visits and App promotion, highlighting the need for more focused implementation and promotion of the App among all health professionals, especially doctors. Further studies will be useful in evaluating the impact of the App on antimicrobial prescribing as well as guide future Antimicrobial Stewardship interventions.

**Supplementary Materials:** The following are available online at http://www.mdpi.com/2079-6382/9/9/555/s1: S1: Launch Communications presentation, S2: AMS App–Commonwealth Pharmacists Association (CPA) Press Release https://commonwealthpharmacy.org/ams-app-cpa-press-release/, Video S1: Commonwealth Partnerships for Antimicrobial Stewardship App https://www.youtube.com/watch?v=MJ7fa\_aLgCI, S3: App Launch Posters, S4: Questionnaires Healthcare workers and patients.

**Author Contributions:** Conceptualization, C.T. and D.A.-O.; Data curation, O.O., C.T., W.P.K., R.M., L.H., E.P.-Q., D.M.A. and D.A.-O.; Formal analysis, O.O.; Funding acquisition, D.A.-O.; Methodology, O.O., C.T., W.P.K., R.M., E.P.-Q. and D.A.-O.; Project administration, O.O.; Resources, D.A.-O.; Supervision, D.A.-O.; Writing—original draft, O.O.; Writing—review and editing, C.T., W.P.K., L.H., M.N., E.P.-Q., D.M.A. and D.A.-O. All authors have read and agreed to the published version of the manuscript.

**Funding:** This project was funded as part of the Commonwealth Partnerships on Antimicrobial Stewardship (CwPAMS) supported by Tropical Health and Education Trust (THET) and Commonwealth Pharmacists Association (CPA) using Official Development Assistance (ODA) funding, through the Department of Health and Social Care's Fleming Fund. The views expressed in this publication are those of the author(s) and not necessarily those of the NHS, the Fleming Fund, the Department of Health and Social Care, THET or CPA.

**Acknowledgments:** Charlotte Ashton, Communications and External Engagement Manager THET for design of launch materials and Comms for CwPAMS App launch. Victoria Rutter, Sarah Cavanagh, (Commonwealth Pharmacists Association), Richard Skone-James, Beatrice Waddingham, William Townsend (THET) for CwPAMS contributions.

**Conflicts of Interest:** The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results. MK and LH work at Horizon Strategic Partners who own and manage the Microguide app.

#### **References**


© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

MDPI St. Alban-Anlage 66 4052 Basel Switzerland Tel. +41 61 683 77 34 Fax +41 61 302 89 18 www.mdpi.com

*Antibiotics* Editorial Office E-mail: antibiotics@mdpi.com www.mdpi.com/journal/antibiotics

MDPI St. Alban-Anlage 66 4052 Basel Switzerland

Tel: +41 61 683 77 34

www.mdpi.com ISBN 978-3-0365-7254-3