*3.1. CHOC*

The multidisciplinary champion team at CHOC was comprised of a neonatologist, nurse educator and respiratory therapist. A total of 10 HCPs (four physicians, three nurses, three respiratory therapists) completed the online video training. Video-recorded simulations and debriefings were started in July 2018. The first 10 recordings served as a baseline to inform the creation of the first simulation improvement bundle which included (1) use of standardized briefings prior to simulation; (2) NRP education classes and skills workshops; (3) consistent use of the debriefer rating tool from CAPE; (4) use of debriefing the debriefer; and (5) addition of a simulation specialist to help conduct these team exercises. This bundle was implemented in January 2019 with modifications during multiple PDSA cycles. A total of 38 simulation exercises were completed in situ on Labor and Delivery and the NICU or in a simulation lab.

### 3.1.1. SMART Aim #1

Increase sta ff participation. A total of 73% of physicians, 48% of nurses, and 100% of respiratory therapists were exposed to at least one simulation exercise through June 2019.

### 3.1.2. SMART Aim #2

Decrease LSTs. After implementation of the simulation improvement bundle, LSTs decreased and there was a shift in the median to two LSTs per simulation (Figure 2). Of the LSTs identified, 57% were found to involve technical (e.g., lack of knowledge on usage of laryngeal mask) and 31%, behavioral issues (e.g., lack of role assignment) while 6.4% were attributable to cognitive issues (e.g., knowledge about delayed cord clamping) and 5.4% to system errors (e.g., failed pages).

**Figure 2.** LSTs identified during CHOC simulations over time. Of note, there was a significant shift in the median (>8 consecutive data points below the median) during PDSA cycle 2 towards the goal of ≤1.

### 3.1.3. Other Notable Results

DART scoring revealed improvement over time towards the goal ratio of 3:1 [trainee responses: instructor questions + statements] (Figure 3). Of the 61 post-simulation surveys sent, 56 were completed; 90% of participants strongly agreed or agreed with the objectives of the program. These objectives were to provide a realistic simulated multidisciplinary team training experience in a constructive and psychologically safe learning environment and with ongoing feedback for improvement from participants. Notably, 89% of the participants believed the debriefing was constructive, 92% felt safe participating in the debrief and 90% wanted to experience more simulation sessions. In response to early qualitative feedback on sessions sometimes being overly long, subsequent sessions were adjusted by having a set time for debriefing.

**Figure 3.** CAPE Real-Time Debriefing Evaluation (DART) scores for CHOC simulations over time. Of note, median shows improvement over time with a shift in median towards a goal ratio of >3:1.

### 3.1.4. Lessons Learned

The collaborative process of CPQCC's Simulating Success Program provided many benefits including the opportunity to: (1) learn from national experts; (2) share challenges and successes; (3) learn from and adapt to different settings; (4) share tools such as confidentiality agreement, surveys and clinical scenarios amongs<sup>t</sup> the sites. This enabled participating sites to appreciate the power of learning from one another. The collaborative approach also helped the team to develop an urgency for change at the institutional level, encouraged friendly competition and fostered accountability. Monthly review of video recordings of the simulation and debriefings with our mentors at CAPE gave the team several opportunities to improve. Systems issues identified during these exercises led to process changes in how codes were called overhead in the NICU and replacing pagers to phones for Labor and Delivery to eliminate missed calls from failed pages.
