Next Article in Journal
Foot Reflexology: Recent Research Trends and Prospects
Next Article in Special Issue
Factors Influencing Food Safety Education Practices among Obstetricians
Previous Article in Journal
Diagnostic Performances of Ultrasound-Based Models for Predicting Malignancy in Patients with Adnexal Masses
Previous Article in Special Issue
Fetal Movement Counting in Prolonged Pregnancies: The COMPTAMAF Prospective Randomized Trial
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Healthcare
Volume 11
Issue 1
10.3390/healthcare11010007
healthcare-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Commentary

Vignette Research Methodology: An Essential Tool for Quality Improvement Collaboratives

by
Kurlen S. E. Payton
1,2,* and
Jeffrey B. Gould
2,3
1
Cedars-Sinai Medical Center, Department of Pediatrics, Division of Neonatology, Los Angeles, CA 90048, USA
2
California Perinatal Quality Care Collaborative, Stanford, CA 94305, USA
3
Department of Pediatrics, Division of Neonatology, Stanford University, Stanford, CA 94305, USA
*
Author to whom correspondence should be addressed.
Healthcare 2023, 11(1), 7; https://doi.org/10.3390/healthcare11010007
Submission received: 16 November 2022 / Revised: 11 December 2022 / Accepted: 14 December 2022 / Published: 20 December 2022
(This article belongs to the Special Issue Safety and Quality in Maternal and Neonatal Care)
Browse Figures
Review Reports Versions Notes

Abstract

:
Variation in patient outcomes among institutions and within institutions is a major problem in healthcare. Some of this variation is due to differences in practice, termed practice variation. Some practice variation is expected due to appropriately personalized care for a given patient. However, some practice variation is due to the individual preference or style of the clinicians. Quality improvement collaboratives are commonly used to disseminate quality care on a wide scale. Practice variation is a notable barrier to any quality improvement effort. A detailed and accurate understanding of practice variation can help optimize the quality improvement efforts. The traditional survey methods do not capture the complex nuances of practice variation. Vignette methods have been shown to accurately measure the actual practice variation and quality of care delivered by clinicians. Vignette methods are cost-effective relative to other methods of measuring quality of care. This review describes our experience and lessons from implementing vignette research methods in quality improvement collaboratives in California neonatal intensive care units. Vignette methodology is an ideal tool to address practice variation in quality improvement collaboratives, actively engage a large number of participants, and support more evidence-based practice to improve outcomes.

1. Introduction: How Can Vignette Research Methods Help Address Practice Variation and Support Quality Improvement?

Variable patient outcomes within institutions and among institutions is a major problem in healthcare. Outcomes in some hospitals or units may be excellent, while outcomes in others are below expectations. Unwarranted practice variation in healthcare contributes to variable patient outcomes, increases the risk of harm, and increases healthcare costs [1,2,3,4,5,6,7,8]. Some practice variation may be due to patient mix and other contextual factors. However, a notable proportion of variation in outcomes is due to differences in how care is delivered. At a global level, this is said to be due to differences in practice, termed practice variation. Operationally, practice variation for any given situation depends upon the extent to which clinicians make the best possible decisions for a given patient and circumstance. Some practice variation is expected when appropriately individualizing care for a given patient. However, some practice variation is due to the provider preference or style. Addressing this dichotomy between appropriate and inappropriate variation is vital for quality improvement [8]. An essential task for any quality improvement (QI) initiative is to identify situations where sub-optimal decisions are being made and then implement strategies to improve the decisions. However, a key challenge in planning and implementing any quality improvement initiative is truly understanding the variation and the drivers that influence the variation. Without this information, stakeholders conducting QI projects may be misinformed about the ideal approach to improvement. If a team has accurate and objective information regarding practice variation, it can develop interventions to encourage and sustain optimal decisions that are likely to lead to better outcomes. The California Perinatal Quality Care Collaborative (CPQCC) recently explored the use of the research vignette methodology as a strategy to describe variation and optimize improvement within the neonatal intensive care unit (NICU)-based quality improvement collaboratives (QICs). This review describes our experience and lessons from implementing vignette methods in the context of NICU QICs, but these lessons are relevant to improvement efforts in any field of healthcare.

2. What Is Vignette Research Methodology?

Vignette research methodology uses narratives with pragmatic manipulation of case characteristics/variables to explore decisions, beliefs, and/or attitudes of the respondents [9,10,11]. The vignette methods are underutilized in healthcare [10,12]. They are commonly used in the social sciences to describe decisions and behaviors that respondents may exhibit in real-life scenarios [13]. In the field of marketing, vignette methods are called conjoint analysis. Conjoint analysis, for example, aims to describe combinations of preferences for certain characteristics of a product considered for purchase by consumers [14,15]. The analysis describes which set of factors are most desired or most important to the consumer. In healthcare, the aim would be to objectively describe the health care practitioner’s decision preferences. The drivers of these decisions can also be explored with vignette methods. The drivers of the decisions may be vital targets of interventions to reduce variation and sustain improvements over time.
Factorial vignette methods involve adding single or multiple characteristics to the base vignette narrative, while keeping other characteristics the same to isolate the impact of the characteristics of interest. This method distinguishes factorial vignette methods from a traditional survey [16]. This strategy capitalizes on the strengths of experimental methods and traditional survey methods, while minimizing the weaknesses of each approach. Figure 1 displays the structure of a vignette with three variables and a question testing the response to each of the three variables. Figure 2 shows an example of the same vignette structure with details of a preterm infant antibiotic stewardship case vignette. Figure 3 shows a term newborn antibiotic stewardship vignette with results showing the impact of two sepsis risk factors on the proportion of prescribers choosing to start antibiotics. The figure displays how the experimental aspect of isolating the impact of the two sepsis risk factors quantifies the impact of these risk factors.

3. What Evidence Supports the Validity and Utility of Vignette Methods in Healthcare?

Vignette methods are considered the most efficient and effective methods of identifying and describing healthcare clinician decisions [10,11,12,17,18]. Peabody et al. studied the relative accuracy of vignettes and chart review. They compared the vignette results to the gold standard of standardized patients in an outpatient adult medicine clinic. They found that the vignettes more closely identified the quality of care compared to the chart review [19,20]. Claims data have also been used to evaluate the quality of care. Converse et al. described how vignettes performed better in assessing quality of care, when compared to claims analysis [10]. Vignettes have been used to assess the quality of care between and within several countries to contrast quality on an international scale [21]. Vignette research methods describe variation and also may support reducing variation, improving care, and reducing costs [22]. Computerized vignette case simulations and gamified practitioner engagement platforms with audit and feedback have been shown to improve the quality of care and reduce costs [23]. Overall, these data suggest that vignettes are a valid and effective method of describing clinician decisions to support optimal healthcare outcomes.

4. How Does Vignette Design Impact the Accuracy and Validity of Results?

The validity of a vignette survey is dependent on the quality of the writing, design, structure, and methods used to present the questions in the vignettes [11,24]. The aim is to reduce bias and maximize internal and external validity. The specific methods and strategies used to minimize bias and maximize the validity of vignette methods are beyond the scope of this review. However, this is a vital aspect of vignette development. The thoughtful implementation of a range of vignette design elements can optimize the validity of vignette results [10,11,12,17,24,25,26,27]. Conversely, poorly constructed vignettes may be susceptible to bias and misleading interpretation.

5. How Can Vignette Methods Support QICs?

QICs are widely used to improve patient outcomes on a large scale. Overall, QICs tend to yield positive results [28,29]. However, evidence supports many opportunities to improve the effectiveness of QICs [30,31,32]. CPQCC has conducted several statewide multi-center NICU quality improvement collaboratives. Overall, CPQCCs collaboratives have led to notable improvements in infant outcomes on a wide scale in California [33,34,35,36,37,38]. The collaboratives are 18 months long and include California NICUs that voluntarily join to participate. These collaboratives have used the Institute for Health Care Improvement (IHI) model for improvement (www.ihi.org, accessed on 14 December 2022) [39].
Individual participating sites have variable success in meeting the improvement goals during the collaboratives. This is common also in other QICs. The reasons for this are multifactorial. Access to resources, QI culture, QI experience, leadership, staffing, and staff engagement, all may impact effectiveness. Sustained improvements are more likely when a strong evidence base for practices was available, with acceptable guidelines and relatively straightforward aspects of care. Additionally, contextual factors that vary between local participants are expected to be an important barrier or driver of improvement. One important contextual factor is practice variation among providers in a given site. Practice variation can also occur at the site level. We noted that practice variation among individuals within a NICU was repeatedly reported as a primary barrier by the participating site QI teams in our recent antibiotic stewardship collaborative and our very low birth weight infant quality improvement collaboratives. Innovative strategies are needed to improve and optimize the effectiveness of QICs [40].
There are several traditional QI tools that were developed to aid improvement. Statistical process control analysis of time series data helps detect improvement while distinguishing signal from noise. The fishbone, or Ishikawa diagram, helps display and organize cause and effect theories. A Pareto chart helps prioritize the targets of interventions of the causes of the problems to be addressed. Plan-Do-Study-Act (PDSA) cycles help organize the approach to improvement by planning, executing a plan to improve, studying this plan, and then capturing the lessons learned to repeat this process in another cycle. The A3 report provides an initial framework for beginning to examine the problem and organizes all of the QI work in a single document. These strategies provide a helpful structure and address specific problems in QI efforts. QICs frequently are aimed at improving care around areas with a wide variation in practice or outcomes. However, the systematic assessment of individual practice variation is not a commonly used strategy in published QIC studies.

6. How and Why Did We Integrate Vignette Methods into the Antibiotic Stewardship QIC?

There is a widespread unwarranted use of antibiotics, and the rates of antibiotic use do not correlate with the rates of proven infection [41,42,43]. Historically, antibiotic use in NICUs evolved with a minimal consideration of the adverse effects. The accepted dogma was “better safe than sorry”. In 2013, some CA NICUs had antibiotic utilization rates (AURs) >50%, reflecting that antibiotics were given in half or more of the patient NICU days [42].
Obstetric antenatal group B streptococcal antibiotic prophylaxis has led to markedly reduced rates of early-onset sepsis in newborns [44,45]. As the rates of early-onset sepsis (EOS) decreased, the evidence and awareness of short- and long-term adverse effects of antibiotics increased. These two inversely changing trends and mandates for antibiotic stewardship implementation led many NICUs to prioritize antibiotic stewardship. However, the optimal strategies to improve and sustain improvement have not been clearly defined.
In response to the high rates of antibiotic use and the wide variation among NICUs in California, CPQCC planned and conducted its first antibiotic stewardship collaborative from 2015 to 2017. This study was the largest multi-center (28 NICUs) NICU antibiotic stewardship collaborative reporting the comprehensive monthly AURs among the participating sites. This stewardship collaborative targeted antibiotic use for suspected early-onset sepsis. The collaborative included 28 California NICUs following the IHI model engaged in monthly webinars working together to improve appropriate antibiotic use based on a set of optimal approaches. However, even among these highly motivated NICUs, many sites struggled to improve or sustain reductions in AUR. Individual prescriber practice variation was noted as an important barrier again. After months of work in the collaborative, many sites were having notable challenges with reducing AURs. The sites reported difficulty obtaining consensus among prescribers, and many sites had not finalized EOS guidelines. Statistical control charts for all sites were analyzed individually. A few sites had evidence of improvement. However, just as many showed brief reductions in AUR and lost the gains. Many did not show any notable improvement. Although practice variation was noted to be widespread, we did not have helpful information about this variation. Towards the end of the collaborative, we decided to implement the vignette study as a supplemental strategy to identify and describe practice variation, thus providing objective data on the “therapeutic locations” where suboptimal decisions were being made and where there may be consensus.
We designed vignettes to explore antibiotic decisions on an individual prescriber level. The results of this study are described in a separate manuscript [46]. The vignettes we developed could be labeled “QI vignettes”. As with traditional vignettes, the vignettes were primarily developed to describe decisions. As a secondary aim, they were also intended to stimulate group discussion and promote individual reflection as well as collective reflection to support more appropriate antibiotic use. This sort of “peer comparison” has been shown to improve outpatient antibiotic prescribing in a randomized control trial comparing several antibiotic stewardship strategies in outpatient pediatrics clinics [47]. Identifying unwarranted variation and intervening with feedback can be effective in reducing variation [48]. Thus, we wanted the vignette results to describe the collective thinking of the prescribers and support practice change at the participating NICUs.
We first considered decisions that had the greatest potential to impact antibiotic exposure in early-onset sepsis cases. We focused on decisions to start antibiotics and decisions to discontinue or continue antibiotics once cultures were negative at around 48 h of life. We then added single or multiple variables (factorial method, as described above) to this base narrative and tested the prescribers’ responses to these variables. We expected this to help us quantify the impact of specific variables on antibiotic decisions.

7. What Were the Vignette Results and How Did They Benefit the QIC?

Figure 4 displays the results from the vignette exploring “starting antibiotics”. The results showed actionable stewardship opportunities with the first two cases. Each of the first two cases resulted in 30% of prescribers starting antibiotics. With 70% of prescribers choosing to withhold antibiotics, both cases could be considered reasonable targets of efforts to shift that 30% of prescribers into considering withholding antibiotics. The third case showed 80% of prescribers choosing to start antibiotics in the case of infants with respiratory signs and sepsis risk factors. This relative consensus among the group suggests that this is not a priority area to focus efforts on changing practice.
The first case objectively quantified the proportion of prescribers choosing to start antibiotics based on an elevated C-reactive protein (CRP) level. From an antibiotic stewardship perspective, using CRP in this case is not useful to rule in or rule out infection in most cases of early-onset sepsis. CRP has a high negative predictive value, if it is low. However, it has a very poor positive predictive value because it is elevated in 20% of healthy uninfected newborns [49,50]. Our vignette assessments exploring how prescribers have used and responded to elevated CRP levels from 2015 through 2022 described a longitudinal description of practice variation. In 2018, national guidelines recommended that elevated inflammatory markers should not routinely be used to guide decisions on starting or stopping antibiotics in EOS [51,52]. Our preliminary data during our current antibiotic stewardship collaborative continues to show that prescribers order the CRP in hopes of using it for its negative predictive value, yet respond to elevated CRP values by starting and continuing antibiotics, even with a negative blood culture. This vignette result is relevant for QIC administrators, participating sites, non-participating sites, policymakers, and other stakeholders interested in antibiotic stewardship.
Figure 5 shows histograms displaying the results of the vignette exploring “stopping antibiotics”. The first two cases showed consensus, with 97% and 89% of the prescribers choosing to stop antibiotics at 48 h once blood cultures were noted to be negative. The third case that was written to isolate the impact of the longer duration of respiratory signs resolving at 72 h resulted in nearly a 50/50 split, with 48% of the prescribers choosing to discontinue antibiotics, and 52% choosing to continue them beyond 72 h. This case was the case with the highest variation among the six cases. This case could also be considered an actionable target of stewardship efforts. As mentioned, the initial vignette results allowed structured conversations about drivers of decisions. Discussions among the participating sites revealed that the prescriber’s perspectives regarding “culture-negative sepsis” was driving the variability in this result.

8. Discussion: How Do the Vignettes Support the Implementation of Evidence-Based Medicine in QICs?

QICs are intended to improve adherence to evidence-based practices and lead to improved patient outcomes. Practice variation can be a notable barrier. It lies between the evidence and the complex task of implementing optimal processes to improve outcomes. An understanding of practice variation, practice consensus, and drivers of individual decisions is vital to optimize quality improvement. Our experience highlighted how the vignettes can address this gap in the traditional QIC process. The Donabedian model describes QI as a product of improvement within three categories: structure, process, and outcomes [53]. Process measures, outcome measures, and balancing measures provide objective information for assessing the performance during QI efforts. Each of these measures has been described as representing the voice of portions of the improvement process. The process measures are considered the voice of the system of care. The outcome measures are the voice of the patient. The balancing measures monitor the system for unintended consequences. Staff decisions, practice variation, and consensus fall under the structure category in the Donabedian model. The individual staff members of the participating NICUs influence the outcomes through their decisions. The voice of the staff responsible for impacting these measures is not traditionally rigorously measured in QICs. Vignette methods can bring the voice of this vital portion of QI into the open to inform the QIC efforts.
We had not previously explored formal methods of assessing individual decisions and decision drivers of the participating NICU staff. We had overlooked the importance of objectively understanding the decisions of individuals and groups. The first vignette survey allowed us to reach participants that were not directly involved with the QI teams. This objective information was useful for the collaborative administrators and the participating teams. The collaborative administrators must prioritize topics and content to reach the aim of improvement. Vignette data on respondent decisions help inform this process. The participating teams noted that information on individual decisions was vital to guiding discussions and determining where they should focus efforts. After a group meeting among their prescribers, one NICU noted they reached consensus in a particular area of antibiotic management. However, the vignette responses collected after their meeting identified specific areas where the prescribers were not in agreement. This helped this NICUs QI team understand where they should focus further efforts. Informal discussions among clinicians may not accurately capture the complexities of practice variation. The vignettes can identify and describe important practice variation details that may otherwise be misinterpreted or completely overlooked. The vignettes provide unique benefits to QIC administrators and participating QI teams.
Pragmatically developed vignettes may help bridge the gap between the current state of practice and the desired evidence-based practice. Knowledge about practice variation among the group reflects the current state of practice. If consensus guidelines are not available, the vignette results can help describe this current state and support efforts to reduce variation as a first step. If evidence-based guidelines are available, the successful application of these guidelines reflects the desired practice state. Evidence-based guidelines can be used to inform the development of vignettes to specifically target areas where practice is not consistent with the guidelines. We considered the ultimate goal of implementation of evidence-based practice during the vignette development process. This is the ultimate aim of the vignettes. The objective is to accurately describe and use the details of current practice variation to support change, moving closer to the desired evidence-based practice.
Using vignettes methods to foster improvement in QICs is supported by the more recently considered behavior change theory. Traditional behavioral change theories in healthcare focused on the individual as the target of change. The assumption was that the individual’s behavior was a product of his/her attitudes and beliefs. This understandably led to the assumption that working to change the individuals’ attitudes, knowledge, and beliefs would likely lead to a behavior change.
However, in today’s complex healthcare contexts, individual actions are heavily influenced by external factors such as peer influences, social dynamics, medicolegal concerns, and pressures to provide value-based care. Acknowledgement of these external influences should lead us to look beyond the individual toward an alternative approach to behavior change. Targeting change in the attitudes of the collective rather than the individual may be a more effective approach [54,55]. Individuals do not practice in a vacuum conveniently hidden from external influence. Based on this theory, a more fitting approach is to target the understanding and modification of peer group norms and expectations. As a behavioral change tool, the vignettes can objectively identify, display, and monitor these norms. Where indicated, desired evidence-based behaviors can be reinforced, and divergent practices can be discussed from a group perspective. We found that using data visualization of vignette results displaying individual decisions among collective decisions of the group provides a concrete method of implementing this theory-based approach. An individual can view their own decisions alongside the range of their peers’ decisions. The individual decisions can be considered within the collective current state. The desired state of practice can be envisioned as a group working towards a common goal. The question “How should I change my practice and what impact could this have on patient outcomes?” changes and becomes “Imagine the impact on patient outcomes, if we collectively adopt this practice change?”. Vignettes can provide a tangible method of approaching behavior change as an effective way to begin improving outcomes through a more systems-based approach to the implementation of evidence-based guidelines.
Our experience of using vignettes to encourage change is supported by the findings of Zamboni et al. in their systematic review exploring and identifying how context and mechanisms of change benefit the QIC [56]. They listed mechanisms of how QICs influence the participants to follow evidence-based practice. Table 1 pairs vignette functions with these mechanisms of influence. We used the vignette results reported to all participants, graphically displayed the results with data visualization, and used them as an anchor for improvement discussions. This process supports these mechanisms by which the QICs empower the participants to improve.
We are further exploring strategies using vignettes in our current and future QICs. We are planning factorial vignette assessments before, during, and after QICs. Presumably, there are distinct benefits of using vignettes at different time points during QICs. Vignettes could be used well in advance of QICs to help inform the development and structure of a project, including the development of the change package, SMART aims, measurement strategy, and areas of focus. This will allow collaboratives to more effectively meet the specific needs of the participating NICUs. Vignette assessment during a QIC could quantify the effectiveness of collaboratives in a formative way. A second vignette assessment similar to the first could help determine the impact on individual and collective practice. This will allow for prioritizing areas to target. Vignette assessment after the conclusion of a collaborative could be used to engage NICUs that were unable to participate. This would provide a formal process of peer comparison among participants and non-participants. This may accelerate the dissemination of progressive practices to promote change beyond the reach of the initial collaborative.

9. Conclusions

Practice variation is a primary barrier to improving patient outcomes. QICs are implemented to improve outcomes on a wide scale. However, systematic assessments of practice variation within QICs are not commonly performed. Factorial vignette methods are an ideal strategy to describe both quality of care and practice variation, engage stakeholders, and promote individual and collective change. We used vignette methods to supplement multicenter QICs in California NICUs. The vignettes provide unique benefits during QIC that cannot be realized with traditional QI methods. Vignette methods should be considered an essential strategy to optimize QI projects and studies.

Author Contributions

Conceptualization, K.S.E.P. and J.B.G.; methodology, K.S.E.P. and J.B.G.; software, K.S.E.P. and J.B.G.; validation, K.S.E.P. and J.B.G.; formal analysis, K.S.E.P. and J.B.G.; writing—original draft preparation, K.S.E.P. and J.B.G.; writing—review and editing, K.S.E.P. and J.B.G. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not Applicable.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Pronovost, P.J. Enhancing physicians’ use of clinical guidelines. JAMA 2013, 310, 2501–2502. [Google Scholar] [CrossRef] [PubMed]
  2. Wennberg, J.E. Unwarranted variations in healthcare delivery: Implications for academic medical centres. BMJ 2002, 325, 961–964. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  3. Cabana, M.D.; Rand, C.S.; Powe, N.R.; Wu, A.W.; Wilson, M.H.; Abboud, P.A.; Rubin, H.R. Why don’t physicians follow clinical practice guidelines? A framework for improvement. JAMA 1999, 282, 1458–1465. [Google Scholar] [CrossRef] [PubMed]
  4. Committee on the Learning Health Care System in America; Institute of Medicine. Best Care at Lower Cost: The Path to Continuously Learning Health Care in America; Smith, M., Saunders, R., Stuckhardt, L., McGinnis, J.M., Eds.; National Academies Press: Washington, DC, USA, 2013. [Google Scholar]
  5. Corallo, A.N.; Croxford, R.; Goodman, D.C.; Bryan, E.L.; Srivastava, D.; Stukel, T.A. A systematic review of medical practice variation in OECD countries. Health Policy 2014, 114, 5–14. [Google Scholar] [CrossRef]
  6. Institute of Medicine Committee on Quality of Health Care in America. Crossing the Quality Chasm: A New Health System for the 21st Century; National Academies Press: Washington, DC, USA, 2001. [Google Scholar]
  7. Djulbegovic, B. A Framework to Bridge the Gaps Between Evidence-Based Medicine, Health Outcomes, and Improvement and Implementation Science. J. Oncol. Pract. 2014, 10, 200–202. [Google Scholar] [CrossRef] [PubMed]
  8. Balakrishnan, M.; Raghavan, A.; Suresh, G.K. Eliminating Undesirable Variation in Neonatal Practice. Clin. Perinatol. 2017, 44, 529–540. [Google Scholar] [CrossRef]
  9. McDonald, P. How Factorial Survey Analysis Improves Our Understanding of Employer Preferences. Swiss J. Sociol. 2019, 45, 237–260. [Google Scholar] [CrossRef] [Green Version]
  10. Converse, L.; Barrett, K.; Rich, E.; Reschovsky, J. Methods of Observing Variations in Physicians’ Decisions: The Opportunities of Clinical Vignettes. J. Gen. Intern. Med. 2015, 30, 586–594. [Google Scholar] [CrossRef] [Green Version]
  11. Veloski, J.; Tai, S.; Evans, A.S.; Nash, D.B. Clinical Vignette-Based Surveys: A Tool for Assessing Physician Practice Variation. Am. J. Med. Qual. 2005, 20, 151–157. [Google Scholar] [CrossRef]
  12. Sheringham, J.; Kuhn, I.; Burt, J. The use of experimental vignette studies to identify drivers of variations in the delivery of health care: A scoping review. BMC Med. Res. Methodol. 2021, 21, 81. [Google Scholar] [CrossRef]
  13. Renold, E. Using vignettes in qualitative research. Build. Res. Capacit. 2002, 3, 3–5. [Google Scholar]
  14. Green, P.E. On the design of choice experiments involving multifactor alternatives. J. Consum. Res. 1974, 1, 61–68. [Google Scholar] [CrossRef]
  15. Flach, S.D.; Diener, A. Eliciting patients’ preferences for cigarette and alcohol cessation: An application of conjoint analysis. Addict. Behav. 2004, 29, 791–799. [Google Scholar] [CrossRef] [PubMed]
  16. Alexander, C.S.; Becker, H.J. The Use of Vignettes in Survey Research. Public Opin. Q. 1978, 42, 93–104. [Google Scholar] [CrossRef]
  17. Evans, S.C.; Roberts, M.C.; Keeley, J.W.; Blossom, J.B.; Amaro, C.M.; Garcia, A.M.; Stough, C.O.; Canter, K.S.; Robles, R.; Reed, G.M. Vignette methodologies for studying clinicians’ decision-making: Validity, utility, and application in ICD-11 field studies. Int. J. Clin. Health Psychol. 2015, 15, 160–170. [Google Scholar] [CrossRef] [Green Version]
  18. Gidengil, C.A.; Linder, J.A.; Beach, S.; Setodji, C.M.; Hunter, G.; Mehrotra, A. Using Clinical Vignettes to Assess Quality of Care for Acute Respiratory Infections. INQUIRY J. Health Care Organ. Provis. Financ. 2016, 53, 0046958016636531. [Google Scholar] [CrossRef] [Green Version]
  19. Peabody, J.W.; Luck, J.; Glassman, P.; Dresselhaus, T.R.; Lee, M. Comparison of vignettes, standardized patients, and chart abstraction: A prospective validation study of 3 methods for measuring quality. JAMA 2000, 283, 1715–1722. [Google Scholar] [CrossRef] [Green Version]
  20. Peabody, J.W.; Luck, J.; Glassman, P.; Jain, S.; Hansen, J.; Spell, M.; Lee, M. Measuring the quality of physician practice by using clinical vignettes: A prospective validation study. Ann. Intern. Med. 2004, 141, 771–780. [Google Scholar] [CrossRef] [Green Version]
  21. Peabody, J.W.; DeMaria, L.; Smith, O.; Hoth, A.; Dragoti, E.; Luck, J. Large-Scale Evaluation of Quality of Care in 6 Countries of Eastern Europe and Central Asia Using Clinical Performance and Value Vignettes. Glob. Health Sci. Pract. 2017, 5, 412. [Google Scholar] [CrossRef] [Green Version]
  22. Weems, L.; Strong, J.; Plummer, D.; Martin, J.; Zweng, T.N.; Lindsay, J.; Paculdo, D.; Tran, M.; Peabody, J. A Quality Collaboration in Heart Failure and Pneumonia Inpatient Care at Novant Health: Standardizing Hospitalist Practices to Improve Patient Care and System Performance. Jt. Comm. J. Qual. Patient Saf. Jt. Comm. Resour. 2019, 45, 199–206. [Google Scholar] [CrossRef]
  23. Burgon, T.; Casebeer, L.; Aasen, H.; Valdenor, C.; Tamondong-Lachica, D.; de Belen, E.; Paculdo, D.; Peabody, J. Measuring and Improving Evidence-Based Patient Care Using a Web-Based Gamified Approach in Primary Care (QualityIQ): Randomized Controlled Trial. J. Med. Internet Res. 2021, 23, e31042. [Google Scholar] [CrossRef]
  24. Su, D.; Steiner, P.M. An Evaluation of Experimental Designs for Constructing Vignette Sets in Factorial Surveys. Sociol. Methods Res. 2018, 49, 455–497. [Google Scholar] [CrossRef]
  25. Atzmüller, C.; Steiner, P.M. Experimental Vignette Studies in Survey Research. Methodology 2010, 6, 128–138. [Google Scholar] [CrossRef]
  26. Pham, T.; Roy, C.; Mariette, X.; Lioté, F.; Durieux, P.; Ravaud, P. Effect of response format for clinical vignettes on reporting quality of physician practice. BMC Health Serv. Res. 2009, 9, 128. [Google Scholar] [CrossRef] [Green Version]
  27. Bachmann, L.M.; Mühleisen, A.; Bock, A.; ter Riet, G.; Held, U.; Kessels, A.G. Vignette studies of medical choice and judgement to study caregivers’ medical decision behaviour: Systematic review. BMC Med. Res. Methodol. 2008, 8, 50. [Google Scholar] [CrossRef] [Green Version]
  28. Ellsbury, D.L.; Clark, R.H. Does quality improvement work in neonatology improve clinical outcomes? Curr. Opin. Pediatr. 2017, 29, 129–134. [Google Scholar] [CrossRef]
  29. Spitzer, A.R. Has Quality Improvement Really Improved Outcomes for Babies in the Neonatal Intensive Care Unit? Clin. Perinatol. 2017, 44, 469–483. [Google Scholar] [CrossRef]
  30. De la Perrelle, L.; Radisic, G.; Cations, M.; Kaambwa, B.; Barbery, G.; Laver, K. Costs and economic evaluations of Quality Improvement Collaboratives in healthcare: A systematic review. BMC Health Serv. Res. 2020, 20, 155. [Google Scholar] [CrossRef] [Green Version]
  31. Schouten, L.M.; Hulscher, M.E.; van Everdingen, J.J.; Huijsman, R.; Grol, R.P. Evidence for the impact of quality improvement collaboratives: Systematic review. BMJ 2008, 336, 1491–1494. [Google Scholar] [CrossRef] [Green Version]
  32. Wells, S.; Tamir, O.; Gray, J.; Naidoo, D.; Bekhit, M.; Goldmann, D. Are quality improvement collaboratives effective? A systematic review. BMJ Qual. Saf. 2018, 27, 226–240. [Google Scholar] [CrossRef]
  33. Lee, H.C.; Bennett, M.V.; Crockett, M.; Crowe, R.; Gwiazdowski, S.G.; Keller, H.; Kurtin, P.; Kuzniewicz, M.; Mazzeo, A.M.; Schulman, J.; et al. Comparison of Collaborative Versus Single-Site Quality Improvement to Reduce NICU Length of Stay. Pediatrics 2018, 142, e20171395. [Google Scholar] [CrossRef] [PubMed]
  34. Lee, H.C.; Kurtin, P.S.; Wight, N.E.; Chance, K.; Cucinotta-Fobes, T.; Hanson-Timpson, T.A.; Nisbet, C.C.; Rhine, W.D.; Risingsun, K.; Wood, M.; et al. A quality improvement project to increase breast milk use in very low birth weight infants. Pediatrics 2012, 130, e1679–e1687. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  35. Lee, H.C.; Powers, R.J.; Bennett, M.V.; Finer, N.N.; Halamek, L.P.; Nisbet, C.; Crockett, M.; Chance, K.; Blackney, D.; von Köhler, C.; et al. Implementation Methods for Delivery Room Management: A Quality Improvement Comparison Study. Pediatrics 2014, 134, e1378–e1386. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  36. Wirtschafter, D.D.; Powers, R.J.; Pettit, J.S.; Lee, H.C.; Boscardin, W.J.; Ahmad Subeh, M.; Gould, J.B. Nosocomial infection reduction in VLBW infants with a statewide quality-improvement model. Pediatrics 2011, 127, 419–426. [Google Scholar] [CrossRef] [Green Version]
  37. Gould, J.B. Building the First Statewide Quality Improvement Collaborative, the CPQCC: A Historic Perspective. Children 2020, 7, 177. [Google Scholar] [CrossRef]
  38. Wirtschafter, D.D.; Danielsen, B.H.; Main, E.K.; Korst, L.M.; Gregory, K.D.; Wertz, A.; Stevenson, D.K.; Gould, J.B.; California Perinatal Quality Care Collaborative. Promoting antenatal steroid use for fetal maturation: Results from the California Perinatal Quality Care Collaborative. J. Pediatr. 2006, 148, 606–612. [Google Scholar] [CrossRef]
  39. Institute for Healthcare Improvement. The Breakthrough Series: IHI’s Collaborative Model for Achieving Breakthrough Improvement; IHI Innovation Series white paper; Institute for Healthcare Improvement: Boston, MA, USA, 2003. [Google Scholar]
  40. Shah, V.; Warre, R.; Lee, S.K. Quality Improvement Initiatives in Neonatal Intensive Care Unit Networks: Achievements and Challenges. Acad. Pediatr. 2013, 13, S75–S83. [Google Scholar] [CrossRef]
  41. Schulman, J.; Benitz, W.E.; Profit, J.; Lee, H.C.; Duenas, G.; Bennett, M.V.; Jocson, M.A.L.; Schutzengel, R.; Gould, J.B. Newborn Antibiotic Exposures and Association With Proven Bloodstream Infection. Pediatrics 2019, 144, e20191105. [Google Scholar] [CrossRef]
  42. Schulman, J.; Dimand, R.J.; Lee, H.C.; Duenas, G.V.; Bennett, M.V.; Gould, J.B. Neonatal Intensive Care Unit Antibiotic Use. Pediatrics 2015, 135, 826–833. [Google Scholar] [CrossRef] [Green Version]
  43. Schulman, J.; Profit, J.; Lee, H.C.; Dueñas, G.; Bennett, M.V.; Parucha, J.; Jocson, M.A.L.; Gould, J.B. Variations in Neonatal Antibiotic Use. Pediatrics 2018, 142, e20180115. [Google Scholar] [CrossRef] [Green Version]
  44. Joshi, N.S.; Huynh, K.; Lu, T.; Lee, H.C.; Frymoyer, A. Epidemiology and trends in neonatal early onset sepsis in California, 2010–2017. J. Perinatol. 2022, 42, 940–946. [Google Scholar] [CrossRef] [PubMed]
  45. Schrag, S.J.; Farley, M.M.; Petit, S.; Reingold, A.; Weston, E.J.; Pondo, T.; Hudson Jain, J.; Lynfield, R. Epidemiology of Invasive Early-Onset Neonatal Sepsis, 2005 to 2014. Pediatrics 2016, 138, e20162013. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  46. Payton, K.S.E.; Wirtschafter, D.; Bennett, M.V.; Benitz, W.E.; Lee, H.C.; Kristensen-Cabrera, A.; Nisbet, C.C.; Gould, J.; Parker, C.; Sharek, P.J. Vignettes Identify Variation in Antibiotic Use for Suspected Early Onset Sepsis. Hosp. Pediatr. 2021, 11, 770–774. [Google Scholar] [CrossRef] [PubMed]
  47. Meeker, D.; Linder, J.A.; Fox, C.R.; Friedberg, M.W.; Persell, S.D.; Goldstein, N.J.; Knight, T.K.; Hay, J.W.; Doctor, J.N. Effect of Behavioral Interventions on Inappropriate Antibiotic Prescribing Among Primary Care Practices. JAMA 2016, 315, 562. [Google Scholar] [CrossRef] [Green Version]
  48. Harrison, R.; Hinchcliff, R.A.; Manias, E.; Mears, S.; Heslop, D.; Walton, V.; Kwedza, R. Can feedback approaches reduce unwarranted clinical variation? A systematic rapid evidence synthesis. BMC Health Serv. Res. 2020, 20, 40. [Google Scholar] [CrossRef] [Green Version]
  49. Mjelle, A.B.; Guthe, H.J.T.; Reigstad, H.; Bjørke Monsen, A.L.; Markestad, T. Serum concentrations of C-reactive protein in healthy term-born Norwegian infants 48–72 hours after birth. Acta Paediatr. 2018, 108, 849–854. [Google Scholar] [CrossRef]
  50. Perrone, S.; Lotti, F.; Longini, M.; Rossetti, A.; Bindi, I.; Bazzini, F.; Belvisi, E.; Sarnacchiaro, P.; Scapellato, C.; Buonocore, G. Creactive protein in healthy term newborns during the first 48 hours of life. Arch. Dis. Child. Fetal Neonatal Ed. 2017, 103, F163–F166. [Google Scholar] [CrossRef]
  51. Puopolo, K.M.; Benitz, W.E.; Zaoutis, T.E. Management of Neonates Born at ≥35 0/7 Weeks’ Gestation With Suspected or Proven Early-Onset Bacterial Sepsis. Pediatrics 2018, 142, e20182894. [Google Scholar] [CrossRef] [Green Version]
  52. Puopolo, K.M.; Benitz, W.E.; Zaoutis, T.E. Management of Neonates Born at ≤34 6/7 Weeks’ Gestation With Suspected or Proven Early-Onset Bacterial Sepsis. Pediatrics 2018, 142, e20182896. [Google Scholar] [CrossRef] [Green Version]
  53. Donabedian, A. Evaluating the quality of medical care. 1966. Milbank Q. 2005, 83, 691–729. [Google Scholar] [CrossRef] [Green Version]
  54. Johnson, M.J.; May, C.R. Promoting professional behaviour change in healthcare: What interventions work, and why? A theory-led overview of systematic reviews. BMJ Open 2015, 5, e008592. [Google Scholar] [CrossRef]
  55. Rynkiewich, K. Finding “What’s Wrong with Us”: Antibiotic Prescribing Practice Among Physicians in the United States. Front. Sociol. 2020, 5, 5. [Google Scholar] [CrossRef] [Green Version]
  56. Zamboni, K.; Baker, U.; Tyagi, M.; Schellenberg, J.; Hill, Z.; Hanson, C. How and under what circumstances do quality improvement collaboratives lead to better outcomes? A systematic review. Implement Sci. 2020, 15, 27. (In English) [Google Scholar] [CrossRef]
Healthcare 11 00007 g001 550
Figure 1. Flow map showing how case vignette details begin the narrative aimed at describing variation. Three variables are each independently added to the narrative to isolate the impact of each variable alone on the decision in response to the variable.
Figure 1. Flow map showing how case vignette details begin the narrative aimed at describing variation. Three variables are each independently added to the narrative to isolate the impact of each variable alone on the decision in response to the variable.
Healthcare 11 00007 g001
Healthcare 11 00007 g002 550
Figure 2. Flow map showing the initial details of a preterm infant case vignette narrative. Prior to the addition of the variables, nearly all respondents chose to discontinue antibiotics. Three variables are each independently added to the narrative to isolate the impact of each variable alone on the decision to discontinue antibiotics.
Figure 2. Flow map showing the initial details of a preterm infant case vignette narrative. Prior to the addition of the variables, nearly all respondents chose to discontinue antibiotics. Three variables are each independently added to the narrative to isolate the impact of each variable alone on the decision to discontinue antibiotics.
Healthcare 11 00007 g002
Healthcare 11 00007 g003 550
Figure 3. The graphic displays the initial common history of a term newborn vignette. The first variable (1) is no antenatal sepsis risk factors. The second variable (2) added to the vignette is two antenatal sepsis risk factors. The blue arrow highlights the proportion of prescribers that changed their decision by choosing to start antibiotics in response to the two antenatal sepsis risk factors.
Figure 3. The graphic displays the initial common history of a term newborn vignette. The first variable (1) is no antenatal sepsis risk factors. The second variable (2) added to the vignette is two antenatal sepsis risk factors. The blue arrow highlights the proportion of prescribers that changed their decision by choosing to start antibiotics in response to the two antenatal sepsis risk factors.
Healthcare 11 00007 g003
Healthcare 11 00007 g004 550
Figure 4. The set of histograms shows the results of a single vignette with three sets of variables added to determine the proportion of prescribers choosing to start antibiotics (grey) and the proportion choosing no antibiotics (white).
Figure 4. The set of histograms shows the results of a single vignette with three sets of variables added to determine the proportion of prescribers choosing to start antibiotics (grey) and the proportion choosing no antibiotics (white).
Healthcare 11 00007 g004
Healthcare 11 00007 g005 550
Figure 5. The set of histograms shows the results of a single vignette with three sets of variables added to determine the proportion of prescribers choosing to discontinue antibiotics (white) and the proportion choosing to continue antibiotics (grey).
Figure 5. The set of histograms shows the results of a single vignette with three sets of variables added to determine the proportion of prescribers choosing to discontinue antibiotics (white) and the proportion choosing to continue antibiotics (grey).
Healthcare 11 00007 g005
Table
Table 1. Vignette results support QIC mechanisms of influencing individuals.
Table 1. Vignette results support QIC mechanisms of influencing individuals.
How QICs Influence Individuals to Follow Evidence-Based Practice GuidelinesVignette Results
Increased commitment and confidence in using data to prioritize problems that they can impactProvide objective data on variation and decisions; Data visualizations of variation can guide discussions about priority areas to target
Increased accountability by making optimal clinical approach very clearDisplay range of variation allowing constructive discourse on optimal approaches to improvement
Provide opportunity for peer reflection and
group problem solving		Graphical displays comparing proportions of individuals making each decision provide a concrete visualization of opportunities
Bottom-up, inclusive team-oriented shared responsibility; Culture of joint problem solvingVignettes help reach individuals that may feel
distant from the QI work
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Payton, K.S.E.; Gould, J.B. Vignette Research Methodology: An Essential Tool for Quality Improvement Collaboratives. Healthcare 2023, 11, 7. https://doi.org/10.3390/healthcare11010007

AMA Style

Payton KSE, Gould JB. Vignette Research Methodology: An Essential Tool for Quality Improvement Collaboratives. Healthcare. 2023; 11(1):7. https://doi.org/10.3390/healthcare11010007

Chicago/Turabian Style

Payton, Kurlen S. E., and Jeffrey B. Gould. 2023. "Vignette Research Methodology: An Essential Tool for Quality Improvement Collaboratives" Healthcare 11, no. 1: 7. https://doi.org/10.3390/healthcare11010007

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop