**How Can National Antimicrobial Stewardship Interventions in Primary Care Be Improved? A Stakeholder Consultation**

**Aleksandra J. Borek 1,\*, Marta Wanat 1, Anna Sallis 2, Diane Ashiru-Oredope 3, Lou Atkins 4, Elizabeth Beech 5, Susan Hopkins 3,6, Leah Jones 3, Cliodna McNulty 3, Karen Shaw 3,7, Esther Taborn 5,8, Christopher Butler 1, Tim Chadborn <sup>2</sup> and Sarah Tonkin-Crine 1,6**


Received: 1 October 2019; Accepted: 22 October 2019; Published: 31 October 2019

**Abstract:** Many antimicrobial stewardship (AMS) interventions have been implemented in England, facilitating decreases in antibiotic prescribing. Nevertheless, there is substantial variation in antibiotic prescribing across England and some healthcare organizations remain high prescribers of antibiotics. This study aimed to identify ways to improve AMS interventions to further optimize antibiotic prescribing in primary care in England. Stakeholders representing different primary care settings were invited to, and 15 participated in, a focus group or telephone interview to identify ways to improve existing AMS interventions. Forty-five intervention suggestions were generated and 31 were prioritized for inclusion in an online survey. Fifteen stakeholders completed the survey appraising each proposed intervention using the pre-defined APEASE (i.e., Affordability, Practicability, Effectiveness, Acceptability, Safety, and Equity) criteria. The highest-rated nine interventions were prioritized as most promising and feasible, including: quality improvement, multidisciplinary peer learning, appointing AMS leads, auditing individual-level prescribing, developing tools for prescribing audits, improving inductions for new prescribers, ensuring consistent local approaches to antibiotic prescribing, providing online AMS training to all patient-facing staff, and increasing staff time available for AMS work with standardizing AMS-related roles. These prioritized interventions could be incorporated into existing national interventions or developed as stand-alone interventions to help further optimize antibiotic prescribing in primary care in England.

**Keywords:** antimicrobial stewardship; antibiotic prescribing; primary care; implementation; behavior change; stakeholder consultation

#### **1. Introduction**

Conserving antibiotics by optimizing antibiotic prescribing to reduce the spread of antimicrobial resistance is a key public health priority both globally and nationally in the UK [1–3]. In England, 81% of antibiotics were prescribed in primary care in 2017 [4], and up to 23% of these are estimated to be prescribed inappropriately, mostly (unnecessarily) for self-limiting respiratory tract infections (RTIs) [5]. While antibiotic prescribing in primary care in England reduced by 13.2% between 2013 and 2017 [4], antibiotic use in the community is still higher than in several other European countries [6]. There is also a considerable variation in antibiotic prescribing between general practices, with many practices remaining high prescribers [7], and between practices and other types of healthcare providers in the community (e.g., out-of-hours, urgent care) [4]. The variations in antibiotic use are not (fully) explained by differences in patient characteristics, such as clinical presentation or prevalence of comorbidities [7,8].

Changing healthcare professionals' (HCP) prescribing behaviors can help reduce antibiotic use and many factors influencing antibiotic prescribing for RTIs in primary care have been identified [9–13]. A range of antimicrobial stewardship (AMS) interventions targeting HCPs have been developed, with many shown effective in trials [14–16]. However, despite the recent decrease in antibiotic prescribing and availability of AMS interventions, further optimizing and reducing inappropriate antibiotic use in English primary care remains critical, especially among the higher prescribers. Further progress has been included in the recent National Health Service (NHS) long-term plan [17] and is required to meet the UK five-year target to reduce antibiotic prescribing in community by 25% by 2024 [2]. Behavioral science evidence shows that to be effective, behavior change interventions need to target relevant determinants of behavior and needs of the targeted population, and fit within the contexts where they are implemented [18,19]. Thus, further improving antibiotic prescribing might involve adapting and implementing effective AMS interventions that have not been yet widely used in England [14], and/or addressing contextual and implementation-specific influences experienced by those using AMS interventions [11].

A recent study aimed to explore nationally implemented AMS interventions in the UK and the extent to which they target behaviors related to antibiotic use. Twenty-two interventions for primary care prescribers and eight for community pharmacy staff were identified, targeting on average 5.8 HCPs' behaviors [10]. A follow-up study identified barriers and facilitators to appropriate antibiotic prescribing in primary care and found nine interventions evaluated in the UK and shown effective at reducing antibiotic prescribing [11]; these included five research-only interventions [20–24] and four nationally available interventions: communication skills training [25], FeverPAIN clinical score [26], the TARGET toolkit [27], and the Chief Medical Officer's letters with prescribing feedback to the highest-prescribing practices [28]. Analyzing the behavioral content of the identified AMS interventions and comparing the extent to which they address relevant behaviors and key barriers and facilitators led to identification of potential changes to, or gaps to be addressed by, AMS interventions. However, such theoretical analysis lacks the insight from the targeted population and intervention users, and does not address factors related to context and implementation of interventions. Therefore, we aimed to build on this recent research by consulting stakeholders (i.e., HCPs from general practices, out-of-hours, community pharmacies and commissioning organizations in England) to: (a) identify barriers and facilitators to optimizing antibiotic prescribing and implementing AMS interventions specific to their settings, (b) generate suggestions for improvements of AMS interventions in their specific, primary care settings in England, and (c) prioritize interventions (using pre-specified feasibility and acceptability criteria). This paper reports the findings of this stakeholder consultation.

#### **2. Results**

#### *2.1. Stakeholder Focus Group and Telephone Interviews*

Twelve stakeholders attended the focus group and three participated individually by telephone. Seven were representatives from Clinical Commissioning Groups (CCGs, i.e., organizations responsible

for planning and commissioning of health care services for their local areas in England), three were from NHS England, two from out-of-hours (OOH) organizations, one from a chain of community pharmacies, and two were general practitioners (GPs).

In the first part, the stakeholders discussed barriers and facilitators to optimizing antibiotic prescribing in primary care settings. These are summarized in Table 1 and reported in more detail in Supplementary Materials (Boxes S1–S4). In brief, as key facilitators to optimizing antibiotic prescribing, the stakeholders reported the availability of many AMS interventions, and awareness of healthcare professionals of the need for appropriate and prudent antibiotic prescribing. As one of the key challenges they reported a variation in use (and sometimes low uptake) of interventions between organizations and HCPs. This was exacerbated by barriers including: limited dissemination of information about specific interventions; insufficient time to engage with interventions (related to large workloads and multiple competing priorities); lack of clarity on which interventions to engage with (influenced by a perceived large number of interventions); insufficient initiatives with professionals collaborating across networks (e.g., involving GPs, pharmacists, nurses) which fueled perceptions of 'working in silos'.

In the second part of the consultation, the stakeholders identified challenges to implementing specific current AMS interventions and made suggestions for improvements. These suggestions were compiled, separately for each primary care setting, and are summarized in Table 1 (and reported in more detail in Supplementary Materials, Boxes S1–S4). Key suggestions included: offering financial incentives; mandating certain target behaviors (e.g., making AMS training a mandatory part of professional development or appraisal); regularly auditing prescribing in all practices and of individual prescribers and, based on this, providing interventions tailored to local contexts and individual needs and addressing specific reasons for suboptimal prescribing; developing multi-professional networks and learning groups to promote communication, collaboration and learning between different professions (e.g., GPs, nurses, pharmacists); incorporating interventions nationally within existing clinical systems; and using point-of-care (POC) diagnostics, such as C-Reactive Protein (CRP) tests or throat swabs (although stakeholders expressed ambivalent views on these). No suggestions were identified for walk-in/urgent care centers as no stakeholders were from this specific setting. However, it was agreed that some of the suggested interventions may be relevant to this setting.


**Table 1.** Summary findings from stakeholder focus group and interviews.


**Table 1.** *Cont.*

#### *2.2. Revising and Selecting Intervention Suggestions*

Additional intervention components were identified based on available evidence on AMS interventions shown to be effective in the UK [11]. In addition, steering group members and the research team provided additional suggestions based on their experience and knowledge of current national AMS policy. These were added to the list of suggestions made by the stakeholders. Altogether, 45 intervention suggestions were identified. Some involved modifications of existing interventions or their implementation (e.g., relating to dissemination of information), whereas others involved new intervention components (e.g., that could form a part of an existing intervention or be implemented as a stand-alone intervention).

In order to identify which influences on antibiotic-related prescribing behaviors the suggested interventions aimed to address, the interventions were mapped onto barriers and facilitators to appropriate antibiotic prescribing. These influences were identified in the literature review [11] and by the stakeholders. After revising and selecting intervention suggestions, 31 interventions were included in an online survey comprising: seven suggestions potentially applicable to all settings, ten suggestions specifically for general practice, nine for OOH, and five for community pharmacy. The full list of the 45 identified interventions is available in Supplementary Materials (Table S1), together with barriers and facilitators that they addressed, source of how each intervention was identified, and indicating which interventions were included in or excluded (with reasons) from the survey.

#### *2.3. Stakeholder Survey and Prioritized Intervention Suggestions*

Out of 40 stakeholders invited to complete the survey, 15 (38%) completed it. Seven respondents indicated that they worked (or had expertise) in general practice, five in CCGs, four in OOH, three in walk-in/urgent care centers, one in community pharmacy, and four in other settings (i.e., two working across settings; one in community hospital; one in e-learning for healthcare professionals). The respondents reported between 4 and 23 (mean 10.7) years of relevant experience. The APEASE scores for each intervention and setting are reported in full in Supplementary Materials (Tables S2–S5).

Nine unique interventions were prioritized (Table 2). Three interventions were prioritized for OOH and community pharmacy, and four were prioritized for general practice and walk-in/urgent care centers (as two of the highest-scoring interventions for these two settings had even scores). As some interventions were assessed for multiple settings, four interventions were prioritized for multiple settings: '(2) Multi-disciplinary small group learning' was prioritized for general practice, walk-in/urgent care centers, and community pharmacy; '(3) Appointing AMS leaders' for general practice and OOH; '(7) Agreeing on a consistent local approach to antibiotics' for walk-in/urgent care

centers and community pharmacy; '(8) Providing online AMS training to all patient-facing staff' for walk-in/urgent care centers and community pharmacy.

The lowest scoring intervention (with 22.7% of the maximum APEASE score) was 'providing diagnostic point-of-care CRP testing, including training in using it, interpreting the results and maintaining the equipment' in community pharmacy setting, which was assessed by only five out of 11 respondents as relevant to the setting, by two as practical and acceptable and by none as affordable. This intervention was also rated as the second lowest for general practice (44% of the maximum APEASE score), walk-in/urgent care centers (48.7%) and OOH (50%). Participants' comments provided as free text in the survey indicated that cost and funding, time to do the tests, and concern about over-use of the tests by patients and clinicians were considered the main barriers to using this intervention; for example:

"*Would need to have clear guidance and uptake would depend on who was funding [POC CRP tests]. Barriers to GP practices are cost of equipment and cost of tests, as well as time it takes to perform the test when only have 5–10 min consultation and test takes a few minutes to perform so practical and a*ff*ordability issues are the main barriers*."

*"Concerns [that POC CRP tests] may increase attendance to 'get a test'. May involve clinicians overly relying on a test which is not always accurate or there may be a time lag in the increase in CRP. Time taken in consultation to administer test is a barrier and test strips are costly."*

The two second lowest scoring (with 33.3% of maximum APEASE score) interventions were also in community pharmacy setting; although both were assessed by only three participants. One was 'providing training and resources to structure the way(s) of asking patients the right questions about self-limiting infections and identifying red flags to help decide what to advise patients'. In comments, the participants suggested that: "*this sort of training is available* via *CPPE, however uptake is voluntary*" and "*this should already be done as part of the core community pharmacy contract*." The other intervention was 'promoting the use of patient records by pharmacists (e.g., by digital prompts) to review whether antibiotics were prescribed appropriately'. Participants' comments suggested that:

"*At present community pharmacists do not have access to enough information to be able to do this e*ff*ectively. There may need to be specialist clinical training for community pharmacists to do this*."

"*The relevance will depend on where the community pharmacist is in the patient pathway. If contractual levers remain as is the community pharmacist may require remuneration*."

Other lowest scoring interventions (see a full list in Supplementary Materials) were: 'providing information on opening hours of all local healthcare services' for general practice (31.8% of maximum APEASE score) which was scored particularly low on 'effectiveness', and 'co-organizing national AMS events together with different professional networks' for OOH and walk-in/urgent care centers (50% and 38.5%, respectively) which was rated low on 'affordability'.



#### **3. Discussion**

The stakeholder consultation identified setting-specific barriers and facilitators to current antibiotic optimization, and generated and prioritized suggestions for improvements of AMS interventions in primary care in England. Stakeholders' appraisal of relevance, feasibility and acceptability of 31 intervention suggestions led to nine interventions being prioritized across settings. These prioritized interventions address some of the identified influences on antibiotic prescribing. They could be incorporated as part of existing AMS interventions or further developed and implemented as stand-alone interventions.

#### *3.1. Implications within the Context of Current AMS Research and Practice*

The interventions assessed and prioritized (or not) by the stakeholders build on AMS interventions currently implemented in England [10] and effective interventions tested in UK-based research studies [11]. How the 31 interventions fit with current research and practice, and how those prioritized may be implemented, is discussed below and summarized in Table 3. Interventions are grouped by intervention 'type', for ease of reference.

*Improving engagement with AMS training and resources:* There are many AMS interventions available, including AMS training and resources tested and shown effective in trials [21,25,27]. For example, the TARGET antibiotic toolkit (with training and resources primarily targeted at GPs) is available online [29] and practice workshops promoting the TARGET resources were shown effective [27]. Similarly, the STAR online communication skills training (with a practice seminar) was shown effective in a trial [25] and is now available online on the clinical professional development website [30]. CCG professionals report that HCPs are aware of, and promote or engage with, different AMS training and resources (such as the TARGET toolkit), but time, reaching the correct people and lack of clarity on which online training to promote were reported as the main issues with AMS education [31]. Similarly, the stakeholders in our study identified improving engagement with AMS interventions as a challenge, influenced by lack of time and priority, clarity on which interventions to use, incentives, and opportunities (e.g., protected time, training for HCPs in organizations other than general practice). Current AMS interventions may, at least in theory and research trials, facilitate change but improving the uptake of and engagement with AMS interventions in the real world is critical to further optimizing antibiotic use. This may be facilitated by the 'train the trainers' opportunities provided by the TARGET team [29] and by increasing numbers of pharmacists appointed in primary care settings with AMS as part of their roles. The importance of improving engagement with AMS training and resources was reflected by suggestions prioritized by the stakeholders, such as organizing '(2) multi-disciplinary small group learning' (that could be delivered face-to-face, in addition to online resources, and focus on identifying challenges and solutions specific to local contexts), '(8) providing online AMS training to all patient-facing staff' (rather than, as currently, targeting primarily GPs), and '(9) increasing staff time available for AMS work and standardizing AMS roles'. Another suggestion to improve engagement with AMS training that was made by the stakeholders but received medium APEASE score was to make the AMS training mandatory in general practices.





Notes: 1 Nine UK-based studies of effective AMS interventions [8,20–24,26–28] were identified and are reported elsewhere [11]. 2 Twenty six nationally implemented AMS interventions were identified previously and are reported elsewhere [10]. 3 The nine prioritized interventions are numbered as in Table 2 and include the highest-scoring interventions (3–4 per setting) (green rows). Lowest-scoring interventions (3 per setting) are in orange rows; the remaining interventions with the APEASE scores in the middle are in white rows. All APEASE scores for each intervention and setting are reported in the Supplementary Materials (Tables S2–S5).

*Enhancing prescribing data monitoring and feedback:* Antibiotic prescribing data has been publicly available and fed back to prescribers by a vast majority of CCGs for many years and interventions involving prescribing feedback have been shown effective [24,28]. In contrast, detailed action planning is rarely used (reported for only 16% of CCGs [31], often due to insufficient time for it [32]), as is feedback on individual prescribing—both of these strategies were prioritized. Including them may enhance the impact of monitoring of and feedback on prescribing by specifying and tailoring actions (setting goals and/or 'if-then' plans) to address specific reasons for inappropriate antibiotic prescribing and by activating individual accountability for prescribing. Based on available evidence from lifestyle-related interventions, such behavioral regulation strategies (i.e., self-monitoring, especially when combined with other 'regulatory' techniques such as goal-setting, problem solving or 'if-then' plans) and individual tailoring can be effective behavior change techniques (e.g., [33,34]). Moreover, while monitoring/auditing of, and feedback on, prescribing have likely contributed to reduced antibiotic prescribing in general practice, the stakeholders reported barriers to using these strategies in OOH, such as lack of stable patient population and different computer systems. Developing tools to enable and automate prescribing audit and provision of individualized feedback might further optimize antibiotic prescribing in OOH.

*Ensuring consistency in AMS approaches:* The suggestion to '(7) agree on a consistent local approach to antibiotics' (prioritized for walk-in/urgent care centers and community pharmacies and also highly scored for general practice and OOH) highlights the importance of consistency in managing infections and reinforcing consistent messages to patients by different HCPs and across organizations. It could be implemented by developing and agreeing within-and between-organization action plans or protocols, for example, on using patient leaflets and other resources promoting messages about infection prevention and self-care. The importance of consistency between HCPs in antibiotic prescribing and the messages given to patients was also reflected in the following interventions prioritized by the stakeholders: '(2) multidisciplinary small group learning' and '(8) providing online AMS training to all patient-facing staff'. Moreover, four interventions were prioritized for multiple settings (i.e., '(7) agreeing on a consistent local approach to antibiotics'; '(2) multi-disciplinary small group learning'; '(8) providing online AMS training to all patient-facing staff'; '(3) appointing AMS leaders') and could be considered for implementation across settings and by involving HCPs from different professional networks. This could help promote a more integrated, system-wide approach to AMS [17]. Respondents in a recent survey representing 99% of Clinical Commissioning Groups (CCGs) reported that AMS training was targeted primarily at GPs, compared to 67% of CCG professionals reporting focus on other prescribers, 42% on all practice staff and only 28% on OOH staff; consequently, a system and practice-wide approach was one of the top suggestions for AMS training by the CCG professionals [31].

Finally, it may also be important to consider interventions that have not been prioritized by stakeholders. Providing POC CRP tests was among the lowest-scoring suggestions in all settings. The stakeholders considered it not affordable or practical to deliver. While POC CRP testing is supported by examples of countries with low prescribing rates that routinely use it (e.g., Netherlands) and trial evidence showing it as an effective and safe strategy to reduce antibiotic prescribing for RTIs in general practice [14,35], it may not have sustained effects on prescribing behavior [36] and is often met with mixed views on its usefulness and feasibility from HCPs [37,38]. Moreover, 'co-organizing national AMS events for participants from different professional networks to facilitate multi-disciplinary AMS work' and 'promoting the use of patient records by pharmacists to review whether antibiotics were prescribed appropriately' seemed to be considered by stakeholders as unaffordable; 'providing information on opening hours of local healthcare services' (to reduce prescribing when concerned about limited access to healthcare) was considered unlikely to be effective; and 'providing training and resources to structure the ways of asking patients the right questions about self-limiting infections and identifying red flags to help decide what to advise patients' in community pharmacy was seen as of low relevance. Our findings suggest that these interventions might be less promising ways to optimize antibiotic use for RTIs, at least as seen by the small number of stakeholders consulted in this study. Further research may need to explore and identify ways to address barriers related to these interventions with a larger

group of stakeholders and intervention users. In addition to the nine prioritized interventions and five lowest-scoring interventions, there were also 17 other interventions included in the survey and another 14 suggestions that were not prioritized for inclusion in the survey (all available in the Supplementary Materials) that could potentially be considered and refined.

#### *3.2. Strengths and Limitations*

This study was based on expert input from stakeholders and the study steering group, who had practical experience and knowledge about national AMS interventions and policy. Views of stakeholders and intervention users are critical, yet at times under-represented, in theoretical approaches to developing and refining AMS interventions. Our study, focused on views of commissioners and other key stakeholders, may help address this gap and provide some indication for future research on implementation of AMS interventions. However, our findings need to be interpreted with caution. The stakeholders were identified by the steering group members from their own professional networks, and out of 40 stakeholders invited to the focus group and survey, only 15 responded. It is possible that a larger number of stakeholders or intervention users might have generated suggestions for, and prioritized, different interventions. The stakeholders had varied levels of experience and knowledge of AMS interventions and different settings (with less experience specific to OOH, walk-in centers and community pharmacies). This made it difficult to identify very specific improvements to current AMS interventions and to capture a wider range of views from different organizations and settings. As the study focused on stakeholders and interventions in England, the generalizability of the findings may be limited beyond England.

#### **4. Materials and Methods**

The following steps were taken: (1) a stakeholder focus group and telephone interviews to identify barriers and facilitators to appropriate antibiotic prescribing and to generate intervention suggestions; (2) revision and selection of intervention suggestions; (3) a stakeholder survey to assess and prioritize interventions according to pre-specified criteria of relevance, feasibility and acceptability. We focused on interventions targeting HCPs' antibiotic prescribing for RTIs in the following settings: general practice, out-of-hours (OOH), walk-in/urgent care center, and community pharmacy. The study was reviewed by the University of Oxford Clinical Trials and Research Governance team and classified as service development, and as such it did not require research ethics review. Participants were free to participate in any stage of the consultation and withdraw at any point without any consequences. They were offered reimbursement of travel expenses but no payment for participation.

#### *4.1. Stakeholder Focus Group and Telephone Interviews*

Relevant stakeholders (including HCPs with expertise in and/or experience of antibiotic prescribing in relevant settings) were identified by the study steering group members. The steering group included experts in AMS with expertise in, and experience of, designing and implementing AMS interventions and influencing national AMS policy in the UK. Identified stakeholders were invited by email to attend a 3-hour face-to-face focus group held at Public Health England premises in London. They were followed up once in cases of non-response. Those who could not attend the focus group in person were invited to a telephone interview.

The aims of the focus group were (a) to identify barriers and facilitators to appropriate antibiotic prescribing and to implementing AMS interventions in relevant settings; and (b) to generate suggestions for improvements of current AMS interventions and/or their implementation or for new interventions addressing the identified barriers and facilitators. In the first part of the focus group, the stakeholders were presented with barriers and facilitators identified from a literature review [11]. This was followed by a discussion of stakeholders' experiences and examples of these, and of any other influences on antibiotic prescribing (especially in settings under-represented in the literature, such as community pharmacy, OOH and walk-in centers). In the second part of the focus group, the stakeholders were

presented with examples of nationally available AMS interventions and/or interventions trialed in research studies in the UK. This was followed by a discussion about stakeholders' experiences of using these interventions, any challenges to their use or implementation, and suggestions for improvements and/or any new interventions.

The focus group discussions were facilitated by two researchers in a semi-structured way, which included short presentations and general discussion, followed by more specific questions about participants' views, experiences and examples. A third researcher took detailed notes. The focus group was audio-recorded and the recording was subsequently used to check the notes and add more detail. Additionally, participants were provided with handouts including the information on barriers and facilitators and AMS interventions identified in the literature, and with questions for discussion. They could provide additional comments on the handouts and return them to the researchers. The stakeholders interviewed by telephone were given information and asked questions in a similar format to the focus group. Detailed notes were made on each telephone interview. All suggestions made during the focus group, telephone interviews and on the handouts were collated and summarized. The summary of notes was used to generate a list of barriers and facilitators and a list of intervention suggestions.

#### *4.2. Revision and Selection of Intervention Suggestions for a Survey*

Barriers and facilitators to appropriate antibiotic prescribing and effective research interventions were identified in a literature review [11]); these were used to generate additional suggestions which were added to the list of interventions identified by the stakeholders. The intervention suggestions were divided into those potentially applicable to all settings or specific to each relevant setting. They were mapped onto barriers and facilitators identified by the stakeholders and in a preceding literature review [11].

The list of suggested interventions was reviewed by the researchers (AB, MW, STC, AS) and steering group members (remaining authors) who provided feedback on and suggestions for rephrasing the suggested interventions, and made additional suggestions that were added to the list of interventions. To reduce a relatively large number of identified suggestions, the comments from the steering group members (who have knowledge of existing AMS interventions and those currently in development) were used to prioritize selection of interventions for the survey; suggestions were excluded if they were considered unfeasible, already implemented, already under development, or insufficiently specific.

#### *4.3. Stakeholder Survey and Prioritization of Interventions*

An online survey was used for the stakeholders to assess intervention suggestions using the APEASE criteria [39]: Affordability (is an intervention affordable?), Practicability (can it be delivered easily?), Effectiveness (is it likely to be effective?), Acceptability (is it acceptable to staff?), Side effects and safety (is it safe to implement?), and Equity (can it avoid inequalities in patient care?). The survey was designed and delivered using online Survey Monkey software (www.surveymonkey.com). Stakeholders identified by the steering group (those invited to the focus group) were invited by email, including a brief description about the survey and a link to complete it online. Participants were sent one reminder; responses were anonymous.

The survey asked about participants' roles, work setting or expertise, and years of relevant experience. Participants were then presented with interventions that could be potentially applicable to all settings (i.e., general practice, OOH, walk-in/urgent care center, community pharmacy) and suggestions specific to each setting: general practice, OOH, and community pharmacy. For each suggestion, participants were asked to: (a) assess whether the intervention was relevant to the setting, and, if yes, then (b) to assess it, for that setting, using the APEASE criteria.

Participants could skip questions and whole sections so the numbers of respondents that assessed each intervention for each setting varied. We calculated the number of responses for each intervention in each setting, the maximum possible APEASE score for each intervention/setting, and the actual APEASE score for each intervention/setting. The percentage of the maximum possible APEASE score obtained for each intervention/setting was calculated to allow comparison between interventions.

In order to identify interventions which were rated highly, the following criteria were used to prioritize interventions in each setting: (a) at least 50% of respondents for that question assessed the intervention as relevant to that setting; and (b) intervention was scored as one of the top three (or four in cases of equal scores) based on the percentage of the maximum APEASE score.

#### **5. Conclusions**

The study identified a number of barriers and facilitators to optimizing antibiotics and engaging with AMS interventions, and possible ways in which current AMS interventions in primary care in England could be optimized. Nine interventions were prioritized by stakeholders as relevant, acceptable, affordable and feasible to implement in English primary care. They involve suggestions to help improve engagement with existing AMS training and resources (e.g., by face-to-face small group learning, tailoring AMS training to all patient-facing staff rather than prescribers only), enhancing data monitoring and feedback (e.g., audit and feedback on individual's prescribing, or developing tools to automate prescribing audits), and promoting consistency in AMS approaches across healthcare professionals and services (e.g., by agreeing consistent local approaches, upskilling all patient-facing staff). These can be adapted and further developed as part of current AMS initiatives. Additionally, stakeholders prioritized suggestions to incorporate AMS interventions into clinical systems at a national level and enable sharing of patient information between general practice and OOH clinical systems. Future work needs to also focus more on addressing particular barriers to engagement with specific AMS interventions.

**Supplementary Materials:** The following Supplementary Materials and Data are available online at http://www. mdpi.com/2079-6382/8/4/207/s1: Box S1. Summary findings from stakeholder consultation: relevant to all settings. Box S2. Summary findings from stakeholder consultation: relevant to general practice. Box S3. Summary findings from stakeholder consultation: relevant to out-of-hours. Box S4. Summary findings from stakeholder consultation: relevant to community pharmacy. Table S1. All identified intervention suggestions. Table S2. Stakeholder ratings of interventions for general practice. Table S3. Stakeholder ratings of interventions for out-of-hours. Table S4. Stakeholder ratings of interventions for walk-in/urgent care centers. Table S5. Stakeholder ratings of interventions for community pharmacy.

**Author Contributions:** Conceptualization: S.T.-C., A.J.B., M.W., A.S., T.C.; methodology: all authors; formal analysis: A.J.B., M.W., S.T.-C.; interpretation of results: all authors; writing—original draft preparation: A.J.B.; writing—review and editing: all authors; study administration: A.J.B.; funding acquisition: S.T.-C., T.C. The study steering group members included: D.A.-O., E.B., S.H., L.J., C.M., K.S., E.T., C.B., and T.C.

**Funding:** This study was a part of a consultation commissioned and funded by Public Health England Behavioural Insights, and was carried out as a collaboration between the University of Oxford and Public Health England. S.T.-C. was supported by funding from the National Institute for Health Research (NIHR) Health Protection Research Unit in Healthcare Associated Infections and Antimicrobial Resistance at University of Oxford in partnership with Public Health England [HPRU-2012-10041]. The views and opinions expressed in this paper are those of the authors and not necessarily those of the NHS, NIHR, the Department of Health and Social Care or Public Health England.

**Acknowledgments:** We thank the participants in this study for their helpful contributions.

**Conflicts of Interest:** C.B. reports receiving advisory board fees from Pfizer and Roche Molecular Systems, and grant support from Roche Molecular Diagnostics. Other authors declare no conflict of interest.

#### **Abbreviations**

AMS: Antimicrobial Stewardship; CCG: Clinical Commissioning Group; CRP: C-Reactive Protein; GP: General Practitioner; HCP: Healthcare professional; NHS: National Health Service; NIHR: National Institute of Health Research; OOH: Out-of-hours; POC: Point-of-care; RTI: Respiratory tract infection.

#### **References**


© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

*Article*
