Next Article in Journal
Are We Facing a Tsunami of Vaccine Hesitancy or Outdated Pandemic Policy in Times of Omicron? Analyzing Changes of COVID-19 Vaccination Trends in Poland
Next Article in Special Issue
COVID-19 Vaccine Acceptance and Hesitancy among Migrants, Refugees, and Foreign Workers: A Systematic Review and Meta-Analysis
Previous Article in Journal
Intranasal Single-Replication Influenza Vector Induces Cross-Reactive Serum and Mucosal Antibodies against SARS-CoV-2 Variants
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Project Report

Strategic Combination of Theory, Plain Language, and Trusted Messengers Contribute to COVID-19 Vaccine Uptake: Lessons Learned from Development and Dissemination of a Community Toolkit

by
Alison Caballero
1,*,
Katherine J. Leath
1 and
Allie D. Staton
2
1
Center for Health Literacy, University of Arkansas for Medical Sciences, Little Rock, AR 72212, USA
2
Arkansas Immunization Action Coalition, Little Rock, AR 72203, USA
*
Author to whom correspondence should be addressed.
Vaccines 2023, 11(6), 1064; https://doi.org/10.3390/vaccines11061064
Submission received: 26 April 2023 / Revised: 24 May 2023 / Accepted: 30 May 2023 / Published: 5 June 2023

Abstract

:
Widely accepted practices for the development of health education materials include the use of theoretically driven content, the execution of plain language writing and design strategies, the solicitation of community input, and a plan for dissemination via trusted messengers. Here, we describe the development of a COVID-19 vaccine education toolkit and share preliminary outcomes from dissemination via community health workers. The toolkit was developed to equip community messengers to educate community members about the COVID-19 vaccine. It includes an easy-to-read workbook for community learners, a Leader Guide with scripting, and additional resources for community health workers and other local messengers. The Health Belief Model was used to select content for the workbook, which was refined with input from community members. A team of trained plain language writers worked with clinicians and subject matter experts to draft content that was deemed readable, understandable, and actionable by formal measures and drafts were further refined with additional community feedback. Survey results from community health workers who used the toolkit to provide local education about COVID-19 vaccines indicate that the toolkit facilitated confidence in their ability to deliver scientific content to their community members. More than two-thirds report that use of the toolkit facilitated community members’ decisions to receive COVID-19 vaccines.

1. Introduction

Personal health literacy, or “the degree to which individuals have the ability to find, understand, and use information and services to inform health-related decisions and actions for themselves and others” [1], is less than proficient for nearly nine in ten adults in the United States [2]. This results in challenges for individuals to find accurate health information, understand the content, and use it to improve or sustain personal health—or help others do so. These challenges are more profound in certain racial and ethnic minority groups and among those who are older or have limited income or education [2].
When the work described here was launched in December 2020, the impact of COVID-19 was growing exponentially. Between 19 September 2020 and 19 December 2020, new hospitalizations per week had increased nearly fivefold, from 22,314 to 102,168 [3]. The number of weekly deaths had similarly spiked over the same timeframe, from 4273 to 22,982 [4]. Like many other health conditions, COVID-19 has had a disparate impact on some populations. In 2020, reports showed increased risk for both hospitalization and death among older adults [5] and minorities, including African Americans [6]. Because these groups can both be challenged by limited health literacy [2] and are at increased risk for severe COVID-19 disease and poor outcomes, clear communication regarding strategies that individuals could employ to reduce their susceptibility to, and severity of, the disease was critical.
A key mitigation strategy for lessening the burden of the COVID-19 disease is widespread vaccination among the public. One month following the first vaccines being authorized for use in the United States, only about 1 percent of the adult population had completed an initial COVID-19 vaccine series [7]. This fell well short of the 70 to 85 percent necessary to reach protection through community immunity [8]. Further, many U.S. adults lacked the intention to get vaccinated. As intention is a consistent predictor of behavior uptake, this observation was particularly concerning. Among older adults and African Americans in Arkansas, who are more likely to suffer from both health literacy and undesirable outcomes from COVID-19, vaccination intention was low. In December 2020, many older Arkansans reported plans to never be vaccinated (13.4 percent) or responded as undecided (20.4 percent) [9]. The same survey suggested that among African Americans in the state of Arkansas, intention was far less, with 17 percent reporting they would never be vaccinated and 40.8 percent undecided [9].
Many theories attempt to predict or explain health behaviors. Theories include many components, or constructs, that together help health care and public health providers understand and shape individual health behaviors. One such theory is the Health Belief Model, which was introduced in the 1950s and has been widely used since [10]. The Health Belief Model focuses on individual health decision making and posits that intentions are influenced by beliefs about disease risk (susceptibility and severity), benefits and barriers to the recommended health behavior, prompts or cues to act, and self-efficacy that allows an individual to successfully engage in the behavior [10].
Despite the dire need to effectively educate older and minority populations about COVID-19 disease and risk mitigation strategies, many public health education efforts fell short. One study conducted early in the pandemic assessed a sample of online consumer education materials that were produced largely by public health entities. Authors found that most of the materials did not use plain language best practices to make them understandable and actionable and only 7 percent of materials in the study sample were assessed as “easy to read” [11]. Findings from this study were also observed in many similar studies of public- and patient-facing COVID-19 education [12,13,14]. These known challenges regarding vaccine uptake along with shortcomings in the health literacy responsiveness to vital pandemic communications informed and inspired the work described here.
In addition to the need for the use of plain language in the development of public health messages, the messenger who delivers the public health information is also an important consideration, especially in minority communities [15], where trust in health information may be suboptimal. As such, a growing body of literature supports the use of trusted local messengers to deliver public health education [16,17]. One example of trusted local messengers are community health workers (CHWs), defined as “trusted members of local communities who share lived experiences with their neighbors and peers, and they are experts in navigating complex systems of care, serving as a link between clinical and community-based services and the people who need them most” [18]. The literature suggests that health information delivered by those considered “outsiders” to a community, including public health officials, may not be as successful in sharing health information compared to instances when the information is delivered by those “inside” the community. As “insiders”, CHWs may be more optimally equipped to serve as messengers for delivering public health information. CHWs were engaged during the pandemic to assist with public health education by helping create and disseminate health education materials that are culturally relevant [19].
Given the critical nature of pandemic-related public health communications for older and minority groups at elevated risk for severe COVID-19 illness and limited health literacy, a collaborative team formed to develop and disseminate COVID-19 vaccine information in communities across Arkansas. Here, we summarize outcomes from this effort and provide an in-depth description of the evidence-based processes used, including the use of survey data and health behavior theory to select content; partnerships with subject matter experts and community members to shape content; the contributions of a plain language writing team to optimize the degree to which educational content was readable, understandable, and actionable; and community dissemination, engaging community health workers as local trusted messengers. This approach can be replicated and applied by public health and health care institutions as they develop and deliver health education content on a variety of health-related topics to improve health and advance health equity.

2. Materials and Methods

2.1. About the Toolkit

The purpose of this project was to equip community leaders with a toolkit [20] to educate community members on the safety and effectiveness of COVID-19 vaccines and help inform individual decision making around vaccine uptake. The toolkit was designed between December 2020 and February 2021 for trusted messengers to deliver information about COVID-19 vaccines in their own communities. Staff at a health literacy center at an academic medical center were approached and engaged by leaders of a state department of health to develop the toolkit in late 2020, before the first COVID-19 vaccines were authorized for use. The early timing of this outreach by the state health department was due to the anticipation of the widespread outreach needed to promote vaccine uptake.
The original toolkit included a participant workbook (COVID-19 Vaccine: Learn the Facts to Stay Safe and Protect Others) along with companion materials, which included a leader guide and leader script. The participant workbook is the foundation of the toolkit and was designed to be interactive. It includes checkboxes and fillable blanks to provide learners with an opportunity to internalize the content and to promote individual decision making. As an example, in the section describing factors that increase risk for severe COVID-19 (e.g., smoking or being African American), the list includes checkboxes, and learners are encouraged to check the risks that apply to them. The leader guide and script are additional resources for trusted messengers to use when presenting the workbook content in a community group learning session.

2.2. Workbook Content Selection Using Health Behavior Theory

To develop the workbook, plain language experts at the health literacy center drafted key messages and created a content outline which addressed relevant constructs of the Health Belief Model [10]. Key messages for each construct were aligned with vaccine intent surveillance data [9]. See Table 1 for a list of constructs and associated key messages.

2.3. Workbook Content Development

After subject matter experts approved the outline, a small community panel was recruited and convened to provide formative feedback. The panel included African Americans and older adults to ensure that the team received input on the perceived severity of and susceptibility to COVID-19 and barriers and the benefits of getting a COVID-19 vaccine from these specific populations. Feedback from the input session was incorporated into the workbook outline.
Once the outline was finalized, plain language experts drafted content and planned for document design using plain language best practices [21]. Examples of plain language best practices that guided this work include the use of short sentences and words, avoiding scientific and health industry jargon, expressing numbers as whole numbers rather than percentages, and making learners’ suggested action steps clear and detailed. Drafted content was formally assessed by the writing team for readability, understandability, and actionability.

2.4. Workbook Content Plain Language Assessment

Beginning with the initial draft, iterations of the workbook were formally assessed by the writing team for readability to ensure the workbook was considered easy to read. Using standardized processes, staff selected and cleaned samples of text. For materials over eighteen hundred words, staff selected at least three samples of six hundred words each from the beginning, middle, and end of each material, ensuring that the text they selected was representative of the material and did not include information that was overly simple or difficult. For shorter materials, the entire body of text was used in the assessment. To clean the samples, staff removed bullets and numbers at the start of lists, removed punctuation that did not denote the end of a sentence, and added punctuation at the end of titles, headers, and bullets. This cleaning process ensured that the readability software correctly identified the end of each sentence. Staff inserted cleaned content samples into Seven Formulas software (Micro Power & Light Co., Dallas, TX, USA) to generate results from three validated formulas, namely Flesch-Kincaid [22], SMOG [23], and Fry Graph [24], then averaged the three scores to arrive at a mean readability score. The mean was then used to categorize the material into levels of difficulty: easy (under seventh grade), average difficulty (seventh through ninth grade), or difficult (tenth grade and above).
Because readability alone does not ensure comprehension, the health literacy team further assessed the iterations of the workbook content for understandability and actionability. Trained staff used the Patient Education Materials Assessment Tool for Print Materials for Print (PEMAT-P) [25] to assess these domains of plain language attributes. The PEMAT-P includes seventeen items to assess understandability and seven items to assess actionability and produces separate percentage scores for each area. Health literacy staff used a standardized procedure, including independent reviews by two reviewers who rated materials in accordance with the instrument’s user guide. Reviewers then met to discuss their rating differences and to come to a consensus. Findings from the PEMAT-P assessment were used to make improvements to the workbook before field testing with community members.

2.5. Workbook Field Testing

After workbook content was approved by subject matter experts, as a part of the health literacy center’s iterative design process (See Figure 1), the development team conducted a virtual field testing session to capture potential users’ perspectives about the workbook’s organization, design, formatting, tone, and ease of comprehension. Staff recruited participants from an existing database of community members who previously agreed to take part in field testing sessions. To ensure the inclusion of individuals who are at risk for limited health literacy, we screened participants using the validated Newest Vital Sign tool [26]. Before the session, participants were sent the workbook material to review and given instructions to identify items and concepts that authors made hard to understand or that could otherwise be improved.
The session was conducted via Zoom and included a total of seven participants. All were female, four were African American, one was Native American, and one was multi-racial. A trained facilitator conducted the session using a prepared facilitator’s guide. The discussion focused on the items participants identified during their independent review of the material, with a focus on how each item should be improved. As participants voiced that a certain word, phrase, or concept needed improvement, discussion focused on their preferences for those improvements.
A summary report and recommendations for updates to the workbook were presented to the subject matter experts whose approved edits were incorporated into the final workbook. One example of the recommendations from field testing was to include a recommendation for readers to ask their doctors about ingredients in the vaccine. Ingredients were not listed in the original draft, and one participant suggested we add it. Realizing that the list would vary by manufacturer, that the list would be long and thus make the overall piece longer, and that participants would not recognize many ingredient names, the session leader encouraged participants to discuss other ways to address the perceived barrier that the vaccine contained unknown and potentially unsafe ingredients.

2.6. Toolkit Supplementary Materials Development

After the community workbook was finalized, a complementary leader guide, script, and slide deck were developed using plain language best practices. The original goal of these companion materials was to help trusted leaders facilitate group learning sessions with community members. The original toolkit (which included the workbook leader guide with script and slide deck) was published in February 2021. In addition to paper copies of the workbook being printed and distributed throughout the state by the state’s department of health, the toolkit was posted on a publicly accessible state immunization coalition website.
As information about the COVID-19 vaccine changed, the workbook was updated, and a data sheet was added to the toolkit to eliminate the need for updates to the workbook based on changes to current data. The data sheet accompanies the workbook and is updated regularly with the most current COVID-19 statistics, including hospitalization and death rates by race and age. The updated workbook currently available contains “blanks” for participants to add current statistics from the updated data sheet as these are shared by the community messenger. Because the users of the toolkit were not using the slide deck, it was not updated or included in future editions of the toolkit.

2.7. Toolkit Dissemination

In spring of 2022, the state immunization action coalition collaborated with a group of approximately 150 community health workers (CHWs) who were tasked with improving COVID-19 vaccine education efforts throughout the state. This collaboration was initiated by a pharmacist serving as the Vaccine Confidence Strategist for the state immunization action coalition, who had had many unsuccessful attempts at hosting community workshops with the COVID-19 toolkit. It became apparent that a local trusted messenger was needed to deliver these messages. Additionally, while the original intent was for the toolkit’s content to be delivered in group community learning sessions, the pandemic posed challenges to group gatherings.
The pharmacist hosted training sessions with these 150 CHWs on toolkit utilization to increase their confidence in discussing COVID-19 vaccines with their community members. Training sessions included the in-depth discussion of each part of the toolkit (workbook, data sheet, leader guide, and script), with emphasis on the workbook content and addressing frequently asked questions. CHWs were encouraged to utilize the workbook to facilitate either group or individual discussions with community members. They were discouraged from using the workbook simply as a “handout” without discussion.
CHWs were encouraged during the initial training sessions and follow-up emails to submit an online survey each time they used the workbook. Completing the survey was not a requirement for using the workbook; thus, data collection was limited to those who opted in. The survey was designed to assess workbook utilization and determine perceptions regarding its value and outcomes among CHWs. See Table 2 for a complete list of questions and answer choices from the survey.

3. Results

3.1. Toolkit Materials Assessment

The results of the readability and PEMAT-P plain language assessments of toolkit materials are provided in Table 3. The grade level range across all materials was fourth to ninth grade, with mean readability calculated as grade seven. The community-facing workbook registered as “easy to read” and leader materials registered at an average level of difficulty. High percentage scores from the PEMAT-P in both the understandability and actionability domains indicate that the materials were written in a way that is easy for a lay audience to understand and act on. Toolkit documents earned a score of 100 percent for understandability and actionability, with one exception: reviewers noted in the PEMAT-P assessment that using icon arrays in the data sheet “Learn the Risk of Getting Sick with COVID-19” would make the numbers more understandable. Because icon arrays would need to be updated along with data each time new data was available, and the expectation was that this would occur frequently, the team opted against implementing this suggestion so the data sheet could be updated easily.

3.2. Community Health Worker Survey Results

A total of thirty-two individual uses of the workbook were reported via twenty-four responses. While the survey was available online and could be completed by any user of the toolkit, CHWs who attended training were encouraged to respond and thus nineteen of the twenty-four responses were from CHWs. Most CHWs used the workbook to discuss COVID-19 vaccines with individuals, rather than in a group presentation. Three respondents reported in their comments that they did not physically use the workbooks (by using them as handouts), but the information they learned from the workbook helped them to better understand COVID-19 vaccines in a way that helped them explain it to others (see Figure 2). All but one respondent reported feeling more confident in discussing COVID-19 vaccines when using the workbook.
Though it cannot be determined that the information from the workbook is solely responsible for individuals choosing to receive a COVID-19 vaccine, the majority of respondents reported that community members chose to receive a COVID-19 vaccine after speaking with someone who utilized the workbook. Only five respondents felt that their discussion did not lead to learners opting for vaccination (See Figure 3).
Results for each survey item are included in Table 4 and Table 5 below.

4. Discussion

Controlling the spread of communicable disease within a community requires individual contributions to preventive measures. As COVID-19 numbers grew, several mitigation strategies were suggested to individuals (e.g., social distancing, mask wearing, and proper hand hygiene), but vaccination remained a critical element for containment. While individual choice surrounding health behaviors is complex, as indicated by a host of health behavior theories with multiple constructs that interact with one another, information sharing is at the core of many of those constructs. As an example, theoretical constructs often point us toward shifting attitudes (i.e., about the severity of disease or the value of promoted health behaviors), but attitudes are influenced heavily by the clarity of information received about those topics and trust in those delivering the message. As vaccines were being developed, tested, and authorized for use, information on these topics was plentiful if not overwhelming. Despite mixed evidence as to the power of individual constructs within the Health Belief Model to impact intention and behavior [27], we centered our messaging on the constructs’ additive benefits. A systematic review of over one hundred studies focused on the use of the Health Belief Model in COVID-19 vaccine uptake programs found that constructs’ impacts varied by specific population and differed when examining intention to attain the primary vaccine series as opposed to a booster dose [28]. Still, the review concluded that the Health Belief Model remains a useful tool for predicting vaccine intention.
The information was new to community members and often (as previously described) not written in ways that promote understanding or action. In addition, consumers were burdened with conflicting information and the task of discerning which information was credible. Our approach of combining plain language content vetted by community members with the delivery of messaging led by CHWs, as trusted messengers can help overcome those challenges.
Our work is not without limitations. Data presented were collected from a small number of respondents, primarily CHWs, who were extended a personal invitation by the immunization action coalition team to complete the optional online survey and thus may not be generalizable to other populations of messengers who downloaded or used toolkit materials. Additionally, the survey did not ask respondents to report demographic information about themselves or the demographics of the people they spoke with, so we cannot make conclusions about how the toolkit’s material supported decision making among specific groups with known health literacy challenges or evidence of enhanced vaccine hesitancy or about how representative respondents are of the populations they serve.
Another limitation is that responses are self-reported and actual vaccinations cannot be verified at the individual level. However, CHWs were required to report the precise number of immunizations given at each community event in which they participated, which leads us to believe that the data reported in the survey are accurate.
A final limitation is that we cannot attribute all the instances of the successful vaccine uptake reported to the teaching conducted by CHWs in general or to CHWs’ use of toolkit materials specifically. As behavior change is complex, theoretical constructs not addressed by the toolkit could have been more impactful drivers of vaccine uptake.
In consideration of future impact, the results observed in this project may not be replicable in other public health efforts. The project described here was focused tightly on community education on the singular public health topic of COVID-19. When considering the opportunity to apply the approach described here to other public health education needs that are not as far-reaching or novel as COVID-19, it is possible that motivation to plan for, develop, and disseminate educational programming with such robust methods may be lacking. Because public health education is an ongoing necessity to support the prevention and management of a wide range of both chronic and infectious diseases, the exploration of the elements of the comprehensive approach described here could be a worthwhile investment.

5. Conclusions

Public health education is an essential component of managing the spread of both infectious and chronic diseases and is often designed with a goal of impacting individual health behaviors. While individual decision making regarding preventive behaviors is complex, the combination of content guided by health behavior theory and tailored through community input, expert plain language writing, and delivery by a trusted messenger can be powerful. Our work demonstrated that the educational content produced and delivered in this fashion supported desirable decision making regarding vaccination against COVID-19.
The need for this will work will be ongoing, as only 69.5 percent of U.S. adults have received a primary COVID-19 vaccination series as of May 2023 [29]. As research unveils additional information about long-term impacts of COVID-19, new public health education will be required. Information development and dissemination strategies such as those described here can help all health professionals, including CHWs, keep the public updated on the most current and accurate information. Beyond COVID-19, this combination of strategies can be employed to support health education campaigns on many other topics.

Author Contributions

Conceptualization, A.C., K.J.L. and A.D.S.; methodology (materials development), A.C. and K.J.L.; methodology (data collection), A.D.S.; writing, A.C., K.J.L. and A.D.S.; review and editing, A.C., K.J.L. and A.D.S. All authors have read and agreed to the published version of the manuscript.

Funding

This work was funded by the Arkansas Department of Health (to develop the toolkit, print copies of the workbook, and to support outreach efforts) and the Arkansas Community Foundation (to update toolkit with new information, including the data sheet). These represent contractual agreements and philanthropic funding, respectively, and thus do not have award numbers.

Institutional Review Board Statement

Review by an institutional review board was not exclusively sought for this work. The health literacy center who convened community members to provide feedback on the workbook conducts similar work frequently and its IRB has deemed its practices and purposes do not constitute human subjects research. CHWs who completed the voluntary survey about the toolkit did so as a quality improvement activity in alignment with their typical professional activities.

Informed Consent Statement

Written informed consent was waived for community members who participated in field testing sessions as the Institutional Review Board at the University of Arkansas for Medical Sciences determined this does not constitute human subjects research.

Data Availability Statement

Raw data from the readability and PEMAT-P assessments of toolkit materials are available by email request ([email protected]). Raw survey data are also available by email request ([email protected]).

Acknowledgments

The authors acknowledge the Arkansas Department of Health, whose immunization team provided subject matter expertise; Arkansas Immunization Action Coalition; the CHW supervisors and CHWs who participated in training and provided feedback; staff from the University of Arkansas for Medical Sciences, Center for Health Literacy, who formally assessed toolkit materials; staff from the University of Arkansas for Medical Sciences, Office of Community Health and Research, who supported the convening of CHWs.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Office of Disease Prevention and Health Promotion. Health Literacy—Healthy People 2020. Available online: https://www.healthypeople.gov/2020/topics-objectives/topic/social-determinants-health/interventions-resources/health-literacy (accessed on 10 March 2023).
  2. Kutner, M.; Greenburg, E.; Jin, Y.; Paulsen, C.; White, S. The Health Literacy of America’s Adults: Results from the 2003 National Assessment of Adult Literacy. Available online: https://nces.ed.gov/pubs2006/2006483.pdf (accessed on 28 March 2023).
  3. Centers for Disease Control and Prevention. COVID Data Tracker, Trends in United States COVID-19 Hospitalizations, Deaths, and Emergency Visits by Geographic Area. Available online: https://covid.cdc.gov/covid-data-tracker/#trends_weeklyhospitaladmissions_select_00 (accessed on 23 May 2023).
  4. Centers for Disease Control and Prevention. COVID Data Tracker, Trends in United States COVID-19 Hospitalizations, Deaths, and Emergency Visits by Geographic Area. Available online: https://covid.cdc.gov/covid-data-tracker/#trends_weeklydeaths_select_00 (accessed on 23 May 2023).
  5. Mueller, A.L.; McNamara, M.S.; Sinclair, D.A. Why does COVID-19 disproportionately affect the elderly? Aging 2020, 12, 9959–9981. [Google Scholar] [CrossRef] [PubMed]
  6. Abrams, E.M.; Szefler, S.J. COVID-19 and the impact of social determinants of health. Lancet Respir. Med. 2020, 8, 659–661. [Google Scholar] [CrossRef] [PubMed]
  7. COVID-19 Vaccinations in the United States. Available online: https://covid.cdc.gov/covid-data-tracker/#vaccinations (accessed on 1 February 2021).
  8. King, N. Coronavirus Updates. Available online: https://www.npr.org/sections/coronavirus-live-updates/2021/01/07/954324536/december-was-pandemics-deadliest-month-vaccine-process-has-been-slow (accessed on 28 March 2023).
  9. COVID Vaccination Insights. Available online: https://nrc.infogram.com/detailed-1220-covid-19-vaccination-insights-oct-dec-2020-1hnq410071n3p23 (accessed on 20 March 2021).
  10. Rosenstock, I.M.; Strecher, V.J.; Becker, M.H. Social Learning Theory and the Health Belief Model. Health Educ. Q. 1988, 15, 175–183. [Google Scholar] [CrossRef] [PubMed]
  11. Caballero, B.A.; Leath, K.J.; Watson, J.C. COVID-19 consumer health information needs improvement to be readable and actionable by high-risk populations. Front. Commun. 2020, 5, 56. [Google Scholar] [CrossRef]
  12. Kruse, J.; Toledo, P.; Belton, T.B.; Testani, E.J.; Evans, C.T.; Grobman, W.A.; Miller, E.S.; Lange, E.M.S. Readability, content, and quality of COVID-19 patient education materials from academic medical centers in the United States. Am. J. Infect. Control 2021, 49, 690–693. [Google Scholar] [CrossRef] [PubMed]
  13. Szmuda, T.; Özdemir, C.; Ali, S.; Singh, A.; Syed, M.T.; Słoniewski, P. Readability of online patient education material for the novel coronavirus disease (COVID-19): A cross-sectional health literacy study. Public Health 2020, 185, 21–25. [Google Scholar] [CrossRef] [PubMed]
  14. Mani, N.S.; Ottosen, T.; Fratta, M.; Yu, F. A health literacy analysis of the consumer-oriented COVID-19 information produced by ten state health departments. J. Med. Libr. Assoc. 2021, 109, 422–431. [Google Scholar] [CrossRef] [PubMed]
  15. Shen, A.K.; Browne, S.; Srivastava, T.; Kornides, M.L.; Tan, A.S.L. Trusted messengers and trusted messages: The role for community-based organizations in promoting COVID-19 and routine immunizations. Vaccine 2023, 41, 1994–2002. [Google Scholar] [CrossRef] [PubMed]
  16. Reichelt, M.; Cullen, J.P.; Mayer-Fried, S.; Russell, H.A.; Bennett, N.M.; Yousefi-Nooraie, R. Addressing COVID-19 vaccine hesitancy in rural communities: A case study in engaging trusted messengers to pivot and plan. Front. Public Health 2023, 11, 1059067. [Google Scholar] [CrossRef] [PubMed]
  17. Ibe, C.A.; Hickman, D.; Cooper, L.A. To advance health equity during COVID-19 and beyond, elevate and support community health workers. JAMA Health Forum. 2021, 2, e212724. [Google Scholar] [CrossRef] [PubMed]
  18. Peretz, P.J.; Islam, N.; Matiz, L.A. Community health workers and COVID-19—Addressing social determinants of health in times of crisis and beyond. N. Engl. J. Med. 2020, 383, e108. [Google Scholar] [CrossRef] [PubMed]
  19. Jimenez, D.J.; Gomez, O.; Meraz, R.; Pollitt, A.M.; Evans, L.; Lee, N.; Ignacio, M.; Garcia, K.; Redondo, R.; Redondo, F.; et al. Community Engagement Alliance (CEAL) against COVID-19 disparities: Academic-community partnership to support workforce capacity building among Arizona community health workers. Front. Public Health 2023, 11, 1072808. [Google Scholar] [CrossRef] [PubMed]
  20. Immunize Arkansas. Vaccine Workshop Toolkits. Available online: https://www.immunizear.org/vaccine-workshop-toolkits (accessed on 30 May 2023).
  21. Tip 6. Use Caution with Readability Formulas for Quality Reports. Available online: https://www.ahrq.gov/talkingquality/resources/writing/tip6.html (accessed on 28 March 2023).
  22. Kincaid, J.; Fishburne, R.; Rogers, R.; Chissom, B. Derivation of new readability formulas (automated readability index, fog count and flesch reading ease formula) for navy enlisted personnel. Inst. Simul. Train. 1975, 1–39. [Google Scholar] [CrossRef] [Green Version]
  23. McLaughlin, G.H. SMOG grading: A new readability formula. J. Read. 1969, 12, 639–646. [Google Scholar]
  24. Fry, E. A readability formula that saves time. J. Read. 1968, 11, 513–516. [Google Scholar]
  25. Shoemaker, S.J.; Wolf, M.S.; Brach, C. The Patient Education Materials Assessment Tool (PEMAT) and User’s Guide. Available online: https://www.ahrq.gov/sites/default/files/publications2/files/pemat_guide_0.pdf (accessed on 28 March 2023).
  26. Weiss, B.D.; Mays, M.Z.; Martz, W.; Castro, K.M.; DeWalt, D.A.; Pignone, M.P.; Mockbee, J.; Hale, F.A. Quick assessment of literacy in primary care: The newest vital sign. Ann. Fam. Med. 2005, 3, 514–522. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  27. Jones, C.L.; Jensen, J.D.; Scherr, C.L.; Brown, N.R.; Christy, K.; Weaver, J. The health belief model as an explanatory framework for communication research: Exploring parallel, serial, and moderated mediation. Health Commun. 2015, 30, 566–576. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  28. Limbu, Y.B.; Gautam, R.K. How well the constructs of the health belief model predict vaccination intention: A systematic review on COVID-19 primary series and booster vaccines. Vaccines 2023, 11, 816. [Google Scholar] [CrossRef] [PubMed]
  29. Centers for Disease Control and Prevention. COVID Data Tracker, COVID-19 Vaccinations in the United States. Available online: https://covid.cdc.gov/covid-data-tracker/#vaccinations_vacc-people-booster-percent-total (accessed on 23 May 2023).
Figure 1. Iterative Materials Design Process.
Figure 1. Iterative Materials Design Process.
Vaccines 11 01064 g001
Figure 2. Workbook Use.
Figure 2. Workbook Use.
Vaccines 11 01064 g002
Figure 3. Perceptions of Workbook Outcome.
Figure 3. Perceptions of Workbook Outcome.
Vaccines 11 01064 g003
Table 1. Workbook Key Messages and Alignment with Health Belief Model Constructs.
Table 1. Workbook Key Messages and Alignment with Health Belief Model Constructs.
Constructs of the Health Belief ModelKey Messages
Perceived susceptibility
(Perception of disease risk)
Older and minority groups are at an increased at risk of getting a COVID-19 infection.
Perceived severity
(Perception of the severity of the disease)
COVID-19 infections can cause a range of symptoms, sometimes leading to hospitalization, death, or permanent health problems.
Older adults and minorities are at an increased risk of these severe outcomes.
No individual can predict how severely a COVID-19 infection will affect them.
Perceived barriers
(Perception of barriers to getting the vaccine)
The methods for developing and authorizing COVID-19 vaccines were expedited but no steps were skipped in the process.
Testing for safety was carried out and included older adults and minorities.
COVID-19 vaccines are free for everyone in the United States, regardless of insurance status or U.S. citizenship
Perceived benefits
(Perception of benefits of getting a vaccine)
Getting a COVID-19 vaccine can protect not only the individual receiving the vaccine but also family and friends. This includes protecting those who are unable to get a COVID-19 vaccine.
Getting a COVID-19 vaccine can limit the need to miss work or school, thus supporting financial and academic goals.
Table 2. COVID-19 Vaccine Workbook Utilization Survey.
Table 2. COVID-19 Vaccine Workbook Utilization Survey.
Survey QuestionSurvey Response Options
What is your practice location type?
Clinic
Pharmacy
AFMC
Other (please specify)
Your Organization/Clinic/Pharmacy Information
Your Name (First and Last)
Name of Organization/Clinic/Pharmacy
Address of Organization/Clinic/Pharmacy
City/Town
State/Province
ZIP/Postal Code
Your Email Address
Name of person who used the Workbook (First and Last Name)(Free Text)
Date Workbook was used(Free Text)
Title of person who used the Workbook
Community Health Worker
Nurse
Physician
Pharmacist
Other (please specify)
Which language version of the Workbook was used?
English
Spanish
Marshallese
How was the Workbook used? (please complete a separate survey for different uses of the Workbook)
Individual discussion with someone in the clinic/pharmacy setting
Individual discussion with someone outside of the clinic/pharmacy setting
Discussions with people at a community event
Hosted a COVID-19 presentation using the Workbook
Other (please specify)
How many people did you speak with during this event? (an event can be an individual conversation with someone)
1
2–4
5–9
10–19
20 or more
Did using the Workbook help you feel more confident in talking to people about COVID-19 and COVID-19 vaccines?
Yes
No
If no, please explain.
Did your discussion using the Workbook lead to someone getting a COVID-19 vaccine?
Yes
No
Unsure
How many people received a dose of a COVID-19 vaccine after you spoke with them? (please type “0” if no people received a dose after you spoke with them)(Free text)
Were you asked any questions during the event that you felt unprepared to answer? If yes, please specify.
No
Yes (please specify)
Do you have any suggestions for improving the COVID-19 Workbook/Toolkit? Please answer honestly if you have a suggestion. We welcome constructive criticism.(Free text)
Success Stories: If you have a specific success story you would like us to know about, please feel free to share it with us.(Free text)
Learning Experience: If you have a specific experience that you learned from and would like us to know about, please feel free to share it with us.(Free text)
Other Comments: If you have other comments for our team, please feel free to share them below.(Free text)
Table 3. Results from Formal Plain Language Assessments of Toolkit Content.
Table 3. Results from Formal Plain Language Assessments of Toolkit Content.
Document TitleGrade Level Readability RangeLevel of DifficultyPemat-P Understandability ScorePemat-P Actionability Score
COVID-19 Vaccine: Learn the Facts to Stay Safe and Protect Others—Workbook4th to 7th Easy to read100%100%
COVID-19 Vaccine: Learn the Facts to Stay Safe and Protect Others—Leader Guide 6th to 8th Average difficulty100%100%
COVID-19 Vaccine: Learn the Facts to Stay Safe and Protect Others—Leader Script5th to 8th Average difficulty100%100%
Learn the Risk of Getting Sick with COVID-19–Data Sheet5th to 9th Average difficulty92.3%100%
Table 4. Responses to COVID-19 Vaccine Workbook Utilization Survey.
Table 4. Responses to COVID-19 Vaccine Workbook Utilization Survey.
Survey QuestionResponses (n = 24)
Response OptionsFrequencyPercentage
What is your practice location type?Clinic00%
Pharmacy1250.0%
AFMC1250.0%
Other00%
Title of person who used the Workbook CHW1979.2%
Nurse00%
Physician00%
Pharmacist312.5%
Other28.3%
Which language version of the Workbook was used? English2395.8%
Spanish00%
Marshallese14.2%
How was the Workbook used? (please complete a separate survey for different uses of the Workbook) *Individual discussion with someone in the clinic/pharmacy setting1250%
Individual discussion with someone outside of the clinic/pharmacy setting312.5%
Discussions with people at a community event1041.7%
Hosted a COVID-19 Presentation using the Workbook312.5%
Other312.5%
How many people did you speak with during this event (an event can be an individual conversation with someone)?1625%
2 to 5729.2%
6 to 914.2%
10 to 19520.8%
20 or more520.8%
Did using the Workbook help you feel more confident in talking to people about COVID-19 and COVID-19 vaccines?Yes2291.7%
No (please explain) **28.3%
Did your discussion using the Workbook lead to someone getting a COVID-19 vaccine?Yes1250.0%
No520.8%
Unsure729.2%
Were you asked any questions during the event that you felt unprepared to answer? If yes, please specify.Yes (please explain)1 (Not always able to answer extreme conspiracy theory driven questions)4.2%
No2395.8%
* Some respondents selected more than one response. ** No respondents entered explanatory text.
Table 5. Free Text Responses to COVID-19 Vaccine Workbook Utilization Survey.
Table 5. Free Text Responses to COVID-19 Vaccine Workbook Utilization Survey.
Survey QuestionResponses (Frequency)Representative Quote(s)
How many people received a dose of a COVID-19 vaccine after you spoke with them? (please type “0” if no people received a dose after you spoke with them)0 (10)
1 (3)
3 (1)
4 (1)
5 (1)
15 (1)
16 (1)
18 (1)
25 (1)
Unsure/unknown (4)
N/A
Do you have any suggestions for improving the COVID-19 Workbook/Toolkit? Please answer honestly if you have a suggestion. We welcome constructive criticism.N/A“I like the workbook. I never had the opportunity to present a class but practiced a lot with it. I think people would be able to easily follow the presentation”.
Success Stories: If you have a specific success story you would like us to know about, please feel free to share it with us.N/A“Once I have had the discussion of the importance of the vaccine, I have never had anyone turn me down”.
Learning Experience: If you have a specific experience that you learned from and would like us to know about, please feel free to share it with us.N/A“I learned a lot about the workbook and the motivational aspect on how to be comfortable talking to the community”.
Other Comments: If you have other comments for our team, please feel free to share them below.N/A“I think the workbook is a great tool. Very easy to use, read and understand. Great information in it”.
“Valuable tool and very useful for seminars and educational gatherings”.
“I learned a lot with the vaccine webinars. It made me comfortable and happy with talking to the community”.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Caballero, A.; Leath, K.J.; Staton, A.D. Strategic Combination of Theory, Plain Language, and Trusted Messengers Contribute to COVID-19 Vaccine Uptake: Lessons Learned from Development and Dissemination of a Community Toolkit. Vaccines 2023, 11, 1064. https://doi.org/10.3390/vaccines11061064

AMA Style

Caballero A, Leath KJ, Staton AD. Strategic Combination of Theory, Plain Language, and Trusted Messengers Contribute to COVID-19 Vaccine Uptake: Lessons Learned from Development and Dissemination of a Community Toolkit. Vaccines. 2023; 11(6):1064. https://doi.org/10.3390/vaccines11061064

Chicago/Turabian Style

Caballero, Alison, Katherine J. Leath, and Allie D. Staton. 2023. "Strategic Combination of Theory, Plain Language, and Trusted Messengers Contribute to COVID-19 Vaccine Uptake: Lessons Learned from Development and Dissemination of a Community Toolkit" Vaccines 11, no. 6: 1064. https://doi.org/10.3390/vaccines11061064

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop