**2. Methods**

#### *2.1. Study Design*

We ran a multicenter pre-post study to assess the impact of a stewardship program in ICUs. The program was based on a plenary meeting with representatives from ICUs and on-site audits. The impact of the program was measured by comparison of a panel of indicators computed before and after the intervention.

#### *2.2. Participating Units*

Participation in the study was voluntary, but limited to units working with the software M3, integrated with the laboratory information system. M3 is an EHR developed by a multidisciplinary team involving IT specialists, researchers, physicians, and nurses from the GiViTI network. It was designed to support clinical practice in ICUs and ensure high-quality data for research purposes [18]. M3 is property of Istituto di Ricerche Farmacologiche Mario Negri IRCCS (Milano, Italy) and GiViTI (Ranica, Italy).

#### *2.3. Study Population*

The study population comprised all patients admitted to seven general Italian ICUs of different sizes and case-mix. The study took place between January 2017 and February 2020 and had 12 months of data collection (see the Supplementary Materials for the list of ICUs and their characteristics).

#### *2.4. Data Collection and Management*

All data (clinical and microbiological diagnoses, laboratory tests, and treatments) were automatically acquired from the M3 EHR, without further intervention of the ICU physicians, limiting the risk of biases.

Information in M3 is primarily stored in structured or partially structured form to facilitate data analysis. Automatic services import patients' parameters and results of chemical and microbiological tests from monitors, ventilators, blood–gas analyser devices, and from the hospital information systems. M3 stores patients' data in a local PostgreSQL database in each hospital. Data are then encrypted and transferred in pseudonymised form to a server at the GiViTI coordinating center at the Mario Negri Institute for Pharmacological Research.

For this project we extracted the following variables from M3 databases: presentat-admission or ICU-acquired infection, site of infection, microbiological diagnosis and sensitivity pattern, where available, antimicrobials employed (drug, start and end dates of treatment, drug combinations), the rationale for antibiotic prescription (prophylaxis, targeted or empirical therapy), and length of ICU stay.

#### *2.5. Phases of the Project*

The study was coordinated and monitored by a study board nominated by the GiViTI steering committee. The members of the board were chosen for their expertise in critical care medicine, infectious diseases, clinical microbiology, and data science. The board defined the project's specific objectives and designed all the phases of the intervention.

Definition of the indicators: The study board designed all indicators to measure several dimensions related to the management of infections in ICUs (resistance patterns of the isolated microorganisms to drug classes, appropriateness of drug use, clinical decisions). When needed, the EHR M3 was modified to collect the variables employed to calculate those indicators.

Plenary session: A kick-off meeting was organised with representatives of the ICUs (nurses, intensivists, microbiologists, infectious disease specialists, pharmacists/pharmacologists) to share the objectives of the project, to describe its phases, and to recall and discuss standard strategies for infection control in the ICU and what is known to limit the emergence of antimicrobial resistance. This course was structured with plenary lectures and workgroups based on case records extracted from the EHR of the ICUs. The indicators to describe and quantify the measured data were discussed in this meeting.

To build a common multidisciplinary background, the topics discussed in the meeting aimed to update knowledge about risk stratification, diagnosis of infection, antibiotic prescription for community- and hospital-acquired infections, PK/PD optimisation, interpretation of antibiotic sensitivity tests for classical and novel diagnostic technologies, communication strategies with the laboratories, and information from biomarkers. The importance of environmental cleanliness and prevention of transmission were stressed as fundamental issues

On-site visits and follow up: All ICUs were visited between October 2018 and February 2019 by experienced members of the study board. The multidisciplinary visiting team involved an intensivist, a clinical microbiologist, an infectious disease specialist, and a data scientist from the coordinating team. Each visit lasted a whole day. The morning was dedicated to visiting the ICU and the microbiology laboratory to study the organisation of clinical activities and the decision-making. In the afternoon, pre-intervention data were evaluated, and critical aspects were identified and discussed. In a final de-briefing, the ICU members and the peers agreed on and fixed the goals to be achieved in one year. During this year each ICU could consult the clinical experts.

Final evaluation of the results: One year after the visit data from each center were processed and the indicators computed, each center received a report comparing its own performance to all the other ICUs.

The results were presented in a GiViTI meeting organised in online format due to COVID-19 restrictions in Italy.

#### *2.6. Outcomes*

The success of the stewardship program was evaluated through the following indicators. Mortality and ICU length of stay were used as safety parameters to make sure that the intervention did not harm patients.

#### 2.6.1. Frequency of Patients with MDR Infections

Ratio of patients with at least one infection due to MDR bacteria according to the definition of Ref. [15] to the total number of infected patients. This endpoint was stratified by infections present at ICU admission or acquired during the ICU stay. Infections whose symptoms appeared during the first 48 h in the ICU were considered infections at admission.

#### 2.6.2. Median Duration of Empirical Therapy and Prophylaxis

Kaplan–Meier curves were built to assess the duration of empirical therapies and prophylaxis (antimicrobial treatments aiming to avoid infections, including perioperative prophylaxis), censoring patients with ongoing therapies at discharge (see Supplementary Materials).

#### 2.6.3. Inappropriateness of Antibiotics by Penetration into the Site of Infection

Ratio of inappropriate antibiotic therapies regarding tissue penetration to the number of antibiotics prescribed, based on a recent systematic review [19]. An antibiotic is considered inappropriate when it cannot reach the site of the infection.

#### 2.6.4. Inappropriateness of Antibiotics by Microorganism Resistance Pattern

Ratio of inappropriate antibiotic therapies to the number of antibiotics prescribed. An antibiotic is considered as inappropriate if the bacteria causing the infection are intrinsically resistant [20] or resistant according to susceptibility tests [21–24].

#### 2.6.5. Use of Fluoroquinolone Antibiotics

Proportion of patients who received at least one fluoroquinolone.

#### 2.6.6. Inappropriate Prescriptions of Carbapenems

Ratio of inappropriate prescriptions of carbapenems to the total number of treatments with these drugs. Treatment with carbapenems is considered inappropriate when the microorganism causing the infection was responsive to other molecules with a more limited spectrum or anti-MDR specificity such as penicillin or cephalosporins.

#### 2.6.7. Inappropriate Prescriptions of Colistin

Ratio of inappropriate prescriptions of colistin to the total number of treatments with these drugs. Treatment with colistin is considered inappropriate when the microorganism causing the infection was responsive to penicillin, cephalosporins, and carbapenems.

#### 2.6.8. Inappropriate Prescriptions of Linezolid

Ratio of inappropriate prescriptions of linezolid to the total number of linezolid therapies. Empirical therapies in patients with acute renal failure were considered appropriate. Therapies in patients with SNC infection by Gram + bacteria or any infection due to MRSA or VRE were deemed appropriate.

The board of experts used three additional indicators to condense the results (before and after) in each ICU concerning patients' outcomes and drugs used.


#### *2.7. Statistical Analysis*

Categorical variables are reported as frequency and percentage, continuous variables as mean and standard deviation (SD) or median and interquartile range (IQR), as appropriate.

Chi-squared and Wilcoxon rank-sum tests were applied to compare proportions and distributions of continuous variables, respectively, with a significance level of 0.05.

To take into account stratification by ICU, the results of the indicators before and after the intervention were compared using Cochran–Mantel–Haenszel, stratified Mann–Whitney, and stratified log-rank tests for proportions, distribution of continuous variables, and Kaplan–Meier curves, as appropriate, with a significance level of 0.05.

All analyses were done with R, version 3.6 (R Core Team, R Foundation for Statistical Computing, Vienna, Austria).

#### **3. Results**

The indicators were evaluated for data collected in 2018 on 2901 patients to assess the performance of the seven ICUs before the ASP intervention. The program's efficacy was assessed by comparing the same indicators on data collected for a whole year after the site visits for 3389 patients. The patients' main characteristics are reported in Table 1.

The indicators computed in the pre- and post-intervention phases are compared in Table 2. Improvement was obtained on the frequency of infections caused by MDR bacteria (39.5% post-intervention vs 44.9% pre-intervention), especially for ICU-acquired infections (48.8% vs. 57.7%). The frequency of MDR in infections on admission and acquired in ICU for each center are plotted in Figure 1a,b, before (dashed) and after (solid) the ASP intervention. The horizontal lines indicate the overall average. Although the Cochran–Mantel–Haenszel tests are not significant, the changes are substantial and the percentage of MDR in ICUacquired infection decreased in all but one of the participating ICUs.

**Table 1.** Descriptive table (pre-/post-) main demographics, comorbidities, infections present at ICU admission and infections acquired during ICU stay. Significant levels are indicated as \* *p* < 0.05, \*\* *p* < 0.01, \*\*\* *p* < 0.001.



#### **Table 1.** *Cont.*

**Table 2.** Endpoints with % pre-/post- (aggregated) and *p*-values for all indicators. Significant levels are indicated as \* *p* < 0.05, \*\*\* *p* < 0.001.


**Figure 1.** %MDR on admission (panel (**a**)) and %MDR in ICU-acquired infections (>48 h, panel (**b**)) for the participating centers, pre- (dashed line) and post-intervention (solid line). The horizontal line indicates the average.

The median duration of empirical therapy and prophylaxis was reduced from 5.6 to 4.6 days (*p* < 0.001) and from 2.3 to 2.0 days (*p* = 0.06), respectively. The median duration of empirical therapy before the intervention ranged from about 4 to 8 days in the seven ICUs. This decreased in all the ICUs, significantly in four of them (Figure 2a). Regarding prophylaxis, the behaviour of the ICUs differed widely (Figure 2b). The two ICUs with the longest durations before the intervention improved their performance, coming close to the

average of all the centers. The duration of prophylaxis significantly increased only in one ICU, nonetheless remaining well below the average.

**Figure 2.** Median duration of empirical therapy (**a**) and prophylaxis (**b**) for the participating centers, pre- (dashed line) and post-intervention (solid line). The horizontal line indicates the average. The use of quinolones more than halved. Before the intervention 15.3% of patients needing antibiotics received quinolones. This decreased to 6.0% after the intervention (*p* < 0.001). Quinolones were used for about 10% to 30% of patients in the seven ICUs. Its usage in all the units decreased in both value and variability, ranging from about 3% to 10% (Figure 3a).

The only indicator that significantly increased was the use of linezolid, though with a limited number of prescriptions. After our ASP, 69.8% of linezolid prescriptions were inappropriate (as defined in Section 2), while 54.9% were considered inappropriate before the intervention. This worsened in more than half of the centers (Figure 3b), but the confidence intervals are quite wide since only a few patients received linezolid.

**Figure 3.** Use of quinolones (**a**) and inappropriate prescriptions of linezolid (**b**) for the participating centers, pre- (dashed line) and post-intervention (solid line). The horizontal line indicates the average.

How far appropriateness is concerned, 16,2% and 17,3% of empirical, and 3,8% and 4.8% of targeted treatments were considered inappropriate according to our definitions and no pre/post change could be found (Table 2).

The average length of stay increased (not significantly) from 5.4 to 5.5 days (*p* = 0.07) and mortality remained unchanged (from 16.2% to 15.9%).

The other indicators did not change significantly. They are plotted in the Supplementary Materials.

#### **4. Discussion**

Continuous education and monitoring and improvement of the quality of care in ICUs are the primary missions of the GiViTI group. Given the lasting interest in the epidemiology and reduction of infectious complications in critically ill patients [25], an ASP study was mandatory. The objectives of an ASP are the containment of infections, better use of antimicrobials, and reduction of the emergence and spread of MDR bacteria. Although these goals are universally recognised, standardised methods for their implementation and monitoring are far from being defined yet.

Here, we report a pilot ASP that was education- and culture-based, with no additional workload or formal protocols for healthcare workers. Its implementation was adapted to the different operating conditions of each ICU. The indicators used to monitor the ICU performance are simple, easy to understand and offer a possible tool for continuous surveillance.

Monitoring was made easier by taking data directly from the EHR M3, thus minimising the risk of bias due to the manual input into an ad hoc case report form. Standardised indicators addressing several items in the ASP were automatically computed from M3 data: admission of infected and MDR-infected patients, ICU-related acquisition of MDR infection, duration of antimicrobial treatments (targeted, empirical, or prophylactic), and number of treatments with specific antimicrobials (carbapenems, colistin, quinolones, and linezolid).

Outcomes such as the length of stay and mortality cannot be seen as indicators of efficacy but as an attempt to monitor safety. The possibility of benchmarking results in time with a before/after analysis and among units stimulates them to improve their performance and shows that improvements are possible in clinical practice.

Seven units participated in our study on a voluntary basis. The kick-off meeting of the project gave the opportunity to update clinical knowledge and governance policies. The site visits established personal relationships with the experts and from the discussion of data the specific weak points of each unit could be identified to set individual goals.

Data collected before the intervention from 2901 patients (Table 1) showed large baseline differences among centers. This testifies to the wide diversity in patients' case mix and clinical behaviour as reported in Ref. [2].

The results of the project were positive for the majority of indicators, apparently without causing patients any harm. As in other ASPs [5,8–11,13,26,27], there were reductions in antibiotic prescriptions (especially quinolones), treatment duration, and MDR emergence.

As quinolones are considered as facilitators of MDR [28–32], the drastic reduction of their prescriptions confirms the willingness to improve therapeutic strategies based on scientific knowledge and compliance to protocols. Shorter durations of empirical treatments suggest more efficient management of microbiological samples, from withdrawal to reporting of sensitivity tests. The marked reduction of ICU-acquired MDR infections, although globally non-significant and with quite large differences between centers, illustrates a general improvement in the management of infected patients, regarding either antibiotic prescriptions or infection control.

No before/after changes could be found in the appropriateness issues (Table 2). Our expectations were probably too ambitious and the methodology and definitions not able to detect differences in the prescribing behaviour.

Non-significant changes were observed in the use of carbapenems, and a specific study may be necessary to understand this result more in fully.

The original plan of the study included one more year of observation in 2020 to test the "survival" of improved clinical practice, but unfortunately the COVID pandemic changed the case mix and the ICU work so deeply that comparisons would be meaningless.

Nonetheless, what have we learned from this experience? The enthusiastic acceptance and collaboration of clinicians delegated by each ICU as project contact persons underline the intensivists' interest in improving clinical practice.

In view of the voluntary nature of the project, it was hard to engage colleagues not directly involved in the ASP. In a few ICUs, local site visits were limited because of work shifts, holidays, or lack of interest. The results of the project are more effective and enduring when the ASP message and the need for its implementation are shared among the whole ICU staff.

The ICU is a key node in the complex hospital network of players involved in the management of infections. However, an ASP would not be effective if devoted only to ICU physicians and nurses. For this reason, we also invited on-site microbiologists, pharmacists, and infectious disease specialists to participate at the site visits and encouraged the creation of multidisciplinary teams.

Our pilot project was very resource-consuming: we could never offer it to the approximately 200 units associated with GiViTI. To extend the program to other ICUs, we would have to identify which parts of our project were essential and which could be resized, saving workforce and time.

Furthermore, the medical community has to take account of the terrible impact of the SARS-CoV-2 pandemic on the use of antibiotics in the population, inside and outside hospitals and ICUs [33]. ASPs will be urgently and widely necessary, at least to return to the basic concepts of proper antibiotic prescription of the "pre-COVID" era. Similarly, the reduction in MDR infections needs to be rapidly transferred into real COVID-19 life, since in Italy there has been a significant increase of these pathogens. Hopefully our experience will be helpful.

#### *Limitations of the Study*

The main limitation of the study is the lack of the second year of monitoring the ASP indicators to see if the positive effects were just a "study-related" benefit or it really changed the use of antimicrobial drugs. Most of our participating units are in northern Italy, and most of them became COVID ICUs with a completely different case mix and organisation.

Moreover, the number of participating ICUs was limited. Unfortunately, at that time, few ICUs met the necessary conditions for participation: interest in the study, use of M3 as the EHR, and integration of M3 with the laboratory. For these reasons, we downgraded our study to a pilot study, which, however, gave a considerable amount of important information.

#### **5. Conclusions**

Our ASP adopted a multidisciplinary approach involving clinicians, microbiologists, pharmacologists, infectious disease specialists, and data scientists. It successfully reduced antibiotic consumption and MDR, without risking patient safety. Simple indicators, which can easily be updated to the newer drugs and different patient populations, were automatically computed from common EHR, helping to monitor ASP data.

The feasibility and the success of this multicenter ASP should now encourage healthcare policy makers to consider that "where there's a will, there's a way".

**Supplementary Materials:** The following supporting information can be downloaded at: https:// www.mdpi.com/article/10.3390/jcm11154409/s1. 1. List of ICUs; 2. ICU characteristics; 3. Survival analysis for median duration of antibiotics treatments; 4. Indicators—comparison among ICUs. (a) Frequency of patients with MDR infections. (b) Inappropriateness of antibiotics by penetration into the site of infection. (c) Inappropriateness of antibiotics by microorganism resistance pattern in empirical therapy. (d) Inappropriateness of antibiotics by microorganism resistance pattern in targeted therapy. (e) Use of carbapenems. (f) Average ICU Length of stay. (g) ICU Mortality.

**Author Contributions:** Conceptualization, G.M., S.F., G.B. and B.V.; methodology, S.F. and G.M.; statistical analysis, S.F. and G.M.; investigation, definition of indicators and on-site visits, S.F., G.M., G.B., M.L., B.V., A.D.P., A.G. (Andrea Gori), C.O., A.P., G.M.R., D.S., T.G., L.A., M.R. and A.M.; implementation of stewardship intervention, D.S., R.C., A.G. (Aimone Giugni), U.L., E.R., A.B. and M.P.; data curation, G.M.; writing—draft preparation, S.F., G.M., M.L. and F.D.; review & editing, S.F., G.M., F.D., G.B., M.L., B.V., A.D.P., A.G. (Andrea Gori), C.O., A.P., G.M.R., D.S., T.G., L.A., M.R. and A.M.; project administration, G.M. and E.G. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was funded by MSD, ALIFAXe Thermo Fisher Scientific through an unconditional contribution.

**Institutional Review Board Statement:** The project was conducted as an educational program to improve the quality of care. Data collection with the electronic health record, *MargheritaTre* received the approval from the Ethics Committee of the Coordinating Center, Ospedale Maggiore di Bologna (ethic code 17164).

**Data Availability Statement:** Data can be accessed upon request and under appropriate data sharing agreement.

**Conflicts of Interest:** The authors declare no conflict of interest influencing the representation or interpretation of reported research results.
