Next Article in Journal / Special Issue
Parents’ Knowledge, Attitude and Perceptions on Childhood Vaccination in Saudi Arabia: A Systematic Literature Review
Previous Article in Journal
The Roles of Osteopontin in the Pathogenesis of West Nile Encephalitis
Previous Article in Special Issue
Attitudes and Beliefs on Influenza Vaccination during the COVID-19 Pandemic: Results from a Representative Italian Survey
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Evaluating a Technology-Mediated HPV Vaccination Awareness Intervention: A Controlled, Quasi-Experimental, Mixed Methods Study

1
Department of Health Promotion, Education, and Behavior, Arnold School of Public Health, University of South Carolina, Columbia, SC 29208, USA
2
Department of Communication, College of Charleston, Charleston, SC 29424, USA
3
Department of Exercise Science, Arnold School of Public Health, University of South Carolina, Columbia, SC 29208, USA
4
South Carolina Office of Rural Health, Lexington, SC 29072, USA
*
Author to whom correspondence should be addressed.
Vaccines 2020, 8(4), 749; https://doi.org/10.3390/vaccines8040749
Submission received: 14 November 2020 / Revised: 1 December 2020 / Accepted: 8 December 2020 / Published: 10 December 2020

Abstract

:
College-aged women and men are an important catch-up population for human papillomavirus (HPV) vaccination interventions. Limited research has explored technology-mediated HPV vaccination awareness interventions aimed at college students. The purpose was to evaluate a novel, technology-mediated, social media-based intervention to promote HPV vaccination among college students. A controlled, quasi-experimental, mixed methods study examined the feasibility of a technology-based intervention among two undergraduate classes (n = 58) at a public university in the southeastern United States of America. Classes were randomized to receive one of two cancer prevention programs (i.e., HPV vaccination (intervention) or healthy weight (control)). Both programs contained eight technology-mediated sessions, including weekly emails and private Facebook group posts. Participants completed pre-/post-test surveys and submitted weekly qualitative reflections. Data were analyzed using descriptive statistics and thematic review for qualitative data. Knowledge improved among participants in the HPV vaccination intervention relative to those in the control condition. Participants (97%) interacted on Facebook by “liking” a post or comment or posting a comment. Participants demonstrated robust engagement and high treatment satisfaction. Results suggests that social media is an effective platform to reach college students with health promotion interventions and increase HPV vaccination awareness in this important catch-up population.

1. Introduction

Human papillomavirus (HPV) is pervasive and almost all men and women will contract it at some point in their lifetime [1]. HPV is the most common sexually transmitted infection (STI) in the United States (U.S.), and college students are at high risk for STIs. Young people, aged 15–24, account for almost half (49%) of new HPV infections in the U.S. [2]. In most cases, HPV infections clear without treatment; however, the virus can lead to genital warts and cancers in men and women, including cancers of the cervix, vulva, vagina, penis, and anus, and oropharyngeal cancer [3]. In 2015, there were 43,371 new cases of HPV-associated cancer in the U.S., with oropharyngeal cancer, which is more than twice as common in men, surpassing cervical cancer as the most common HPV-related cancer [4].
HPV vaccinations have been licensed by the Food and Drug Administration (FDA) since 2006 in the U.S. The currently available vaccine in the U.S., Gardasil®9, prevents up to 90% of HPV-related cancers [5]. HPV vaccines are effective and have significantly reduced rates of genital warts, HPV infections, and cervical lesions among women [6]. A recent surveillance study found evidence of HPV vaccine effectiveness and herd immunity among young women [7]. The Advisory Committee on Immunization Practices (ACIP) recommends a two-dose HPV vaccination series for both boys and girls between 11 and 12 years of age; however, in the U.S., uptake of the HPV vaccine is slow. From 2016 to 2017, HPV vaccination initiation among adolescents aged 13–17 increased from 60.4% to 65.5%, yet only 48.6% of adolescents were up-to-date with the HPV vaccine series in 2018 [8]. For adolescents and young adults who missed HPV vaccination at earlier ages (ages 15–26), the ACIP recommends a three-dose HPV vaccination series for women aged 15 to 26, men up to 21 years, and men ages 22–26 who have sex with men. HPV vaccination rates are particularly low in South Carolina, with only 42.7% of adolescents receiving the complete vaccine series (compared with 48.6% in the U.S.) [8]. Furthermore, South Carolina lags behind the U.S. among those receiving at least one dose of the HPV vaccine series (59.6% vs. 65.5%, respectively) [8]. Slow uptake of HPV vaccination results in a large population of college-aged men and women who are partially vaccinated or unvaccinated; this population constitutes what is commonly referred to as a “catch up” population (also called late vaccination). College-age women and men are at increased risk of acquiring HPV and are considered an important catch-up population for HPV vaccination [9,10].
Transitions from high school to young adulthood have been identified as a critical period to institute health-promoting behaviors [11]. College students are a critical population to target for HPV vaccine awareness and completion as these young adults begin to assume increased responsibility and independence surrounding healthcare decisions. Initial efforts to increase knowledge and awareness about HPV and appropriate vaccination practices have indicated promise, but point to the need for additional efforts to promote broader intentions to change and greater vaccine uptake [12,13,14,15,16,17].
Technology provides an attractive platform for HPV vaccination awareness interventions targeting college students because it is ubiquitous in their everyday lives. Nearly 100% of undergraduate college students use the internet [18]. Since 2013, internet use among young adults ages 18–29 years has remained at 97% or higher [19]. Furthermore, approximately 94% own a smartphone, 88% use social media [20], and 80% report using Facebook [21]. Behavioral interventions increasingly employ social media to improve awareness and vaccination coverage [22]. Recent studies have investigated HPV vaccination communication online through social platforms, including YouTube [23], Twitter [24], and Facebook [25]. Online messages about HPV vaccination include sensational and negative posts that may perpetuate misleading coverage and lead to insufficient information [23,25]. Therefore, effective online communication about HPV vaccination is needed to counter the dissemination of manipulative and false messages.
Interventions to increase HPV vaccination have historically targeted parents and health care providers [26,27,28]. However, a growing number of HPV vaccination interventions target young women and college students to increase vaccination rates among vaccine-eligible and catch-up populations [29,30,31,32,33,34,35]. Only a small number of HPV vaccination awareness interventions have employed social media, despite the dynamic potential and widespread reach of this platform [36,37,38,39,40]. A recent study found that a Facebook-assisted cervical cancer prevention discussion was more effective than an in-person discussion [41]. There is a clear opportunity for the development and evaluation of technology-based HPV vaccination interventions targeting college students.
The purpose of this study was to examine a novel, technology-mediated HPV vaccination awareness intervention for college students. Targeting college students during this transitional and formative period offers an opportunity to increase awareness and promote cancer prevention. The goal of this intervention was to encourage those in the catch-up population who had not yet completed vaccination to obtain the full HPV vaccine series.

2. Materials and Methods

2.1. Study Design and Procedures

A controlled, quasi-experimental, mixed methods study examined the feasibility of a technology-based intervention among two undergraduate classes at a public university in the southeast region of the U.S. This study included the implementation and evaluation of two health promotion and cancer prevention programs relevant to college students. Classes were randomized by coin flip to receive either an HPV vaccination awareness intervention or a behavioral weight gain prevention intervention (Healthy Weight; control). Each group served as the control for the other group, allowing for simultaneous intervention comparisons. The intervention was adapted from a successful theory-based health communication campaign, It’s My Time [42]. The study design and results from the Healthy Weight intervention arm have been published previously [43].
Both interventions included 8 technology-mediated health promotion sessions over 9 weeks, which included weekly emails, as well as private Facebook group posts. Study personnel posted to the private Facebook message board at least 5 times each week. Posts were scheduled to allow automated distribution through a social media management tool (Hootsuite Media Inc., Vancouver, BC, Canada) to ensure consistent delivery of content. Each group received the same schedule of email newsletters disseminated automatically using MailChimp (Rocket Science Group, LLC, Atlanta, GA, USA), which offered prescheduled distribution and tracked newsletter open rate. Participants completed a weekly open-ended reflection about the intervention emails and Facebook posts, which encouraged students to describe what factors influenced their personal engagement with the intervention. Posttreatment questionnaire data were obtained 9 weeks after intervention initiation. Detailed descriptions of the HPV and control groups are provided below.
This study was approved by the Institutional Review Boards at the College of Charleston and the University of South Carolina.

2.2. Participant Recruitment and Eligibility

Undergraduate students enrolled in an advanced health communication course were recruited via email. Students elected to voluntarily participate in the study or complete an alternative assignment for course credit. Students were not graded on engagement with the intervention. A description of the study and criteria for eligibility were provided. To be eligible, students were required to be registered in the course, have internet access on a computer or mobile device, and be willing to establish or use their current email address and Facebook account. Informed consent was provided via secure website by all students prior to participation.
All otherwise eligible college students enrolled in the two designated classes included in the study were eligible to participate, including those below the age of 18 years, so as to minimize any stigma or segregation of those college students who might not yet have turned 18 years of age. A waiver of parental permission for participation was requested and approved for those individuals below the age of 18 based on the minimal risk to the subjects associated with participation in the study and the lack of any adverse effects on the rights or welfare of the subjects, coupled with the difficulty in practically carrying out the project without this waiver. Because the selected courses were upperclass courses designed for college students further along in their studies, it was anticipated that there would not be any children below the age of 18. All participants were asked to sign (electronically) an informed consent form that was approved by the College of Charleston Institutional Review Board.

2.3. HPV Vaccination Awareness Intervention

The HPV vaccination intervention was based on the health belief model [44,45], the transtheoretical model [46], and adapted content from the successful theory-based health communication campaign, It’s My Time [42]. It’s My Time was developed through formative audience research and an academic-community collaboration to implement and evaluate a peer-based intervention. Formative research showed that preventing cancer is not a motivating message (or perceived benefit) to young adults. Therefore, the main message of It’s My Time encouraged participants to consider receiving the HPV vaccination so that they have time to pursue their dreams. Messages reinforced the concept of college students embracing their own time and time for themselves rather than emphasizing cancer risk-based messaging. A secondary message reminded students that it is not too late to receive the HPV vaccination or take control of their health [42].
These messages and health education information were incorporated into content that was delivered by weekly electronic newsletters and interactive Facebook posts in the current study. College students report limited knowledge of HPV and HPV-related cancer, coupled with high levels of concern about STIs [30,33,42,47]. Therefore, a key strategy of the intervention was to increase the perceived severity (i.e., the negative consequences) of contracting HPV and increase perceived susceptibility of acquiring HPV [26,31,34,42]. Campaign messages emphasized college students pursuing their interests and dreams with the time they gained from preventing HPV and HPV-related cancer. This approach aligns with current research that successful narrative messages depend on the perceived relevance of the storyline to the audience [31,48].
Lesson content focused on themes relevant to college students contemplating HPV vaccination (see Table 1). Messages were tailored to the campus and local area, with resources for how and where to obtain the HPV vaccination at low or no cost, including the university’s Student Health Services. Study investigators moderated the Facebook page, which included providing answers to questions and stimulating interaction, as well as sharing timely news articles and studies related to HPV vaccination. The posts were designed to cultivate a social climate of support for considering initiation of the vaccination series (or completion of the series for those who had partial completion of the sequence). Polls provided social norming to stimulate vaccination-related discussion. Intervention activities (e.g., reading the newsletters and interacting on Facebook) were intended to require approximately 30 total minutes per week. The program spanned a 9-week period during the spring semester, with a hiatus during spring break week.

2.4. Control Intervention

Similar intervention components were used for the control intervention. The control group received a Healthy Weight intervention, which included 8 electronic newsletters and a minimum of 5 postings per week to a separate, private Facebook group moderated by study staff to increase knowledge and support of healthy eating, activity, and weight control practices. Intervention content was derived from social cognitive theory [49] and established behavioral weight control programs, such as the Diabetes Prevention Program Lifestyle Intervention [50]. Emphasis was placed on key behavioral strategies for healthy eating and physical activity (e.g., goal-setting, self-monitoring, overcoming barriers, relapse prevention, and social support). Students were encouraged to take 10,000 steps/day and were provided with electronic tools for objectively monitoring their physical activity and weight (Fitbit Zip and Aria Wi-Fi-enabled scale, respectively; Fitbit Inc., Boston, MA, USA). The HPV and control interventions were matched on duration, structure, and number of planned contacts. However, the control intervention did not receive content related to HPV awareness or vaccination.

2.5. Measures

Participants completed assessments at baseline and post-intervention at 9 weeks. Questionnaires were administered online for all self-reported measures (Qualtrics, Provo, UT, USA).

2.5.1. Sociodemographic Characteristics

Demographic characteristics were self-reported at baseline and included age, sex, race, and academic year. Sociodemographic characteristics included health insurance status, age at sexual debut, number of sex partners, gender of sex partners, relationship status, and contraceptive method.

2.5.2. HPV Vaccination Knowledge, Attitudes, Beliefs and Behaviors

The current study adapted validated measures from Patel and colleagues [51], the 2013 Health Information National Trends Survey (HINTS), and the 2013 Behavioral Risk Factor Surveillance System (BRFSS). Survey items measured knowledge and awareness of HPV and HPV vaccination (e.g., “Some types of HPV can cause genital warts”; “People who have been infected with HPV might not have symptoms”), factors influencing the decision to be vaccinated (e.g., concerns about getting HPV, recommendation by a healthcare provider), and intention to seek HPV vaccination. In addition, students were queried about how many vaccines in the series they had received to date and, for those who had not yet completed the series, their intentions to seek out the vaccine.

2.5.3. Intervention Engagement

HPV intervention engagement was assessed objectively on web-based platforms (MailChimp and Facebook). Engagement with HPV newsletters was defined as the number of students who opened the newsletter each week, which was obtained from MailChimp metrics. Engagement with the HPV Facebook group was measured by the number of interactions, including post likes, comments, and original posts. Interactions were aggregated for the week and tallied by research staff; the average number of interactions per week was determined.

2.5.4. Treatment Satisfaction

At the conclusion of the intervention, students reported how satisfied they were with the HPV program by rating its overall usefulness and likelihood of recommending it to a friend or relative. Level of satisfaction with specific intervention components (e.g., number of sessions, frequency, and length of Facebook group postings) was also reported. All items were rated on an agreement-oriented 7-point Likert scale anchored with strongly agree and strongly disagree.

2.5.5. Weekly Reflections

Participants received a weekly web-based survey inviting them to write a reflection critiquing the efficacy of each week’s materials and messages, including recommendations for improvement. Students were prompted to reflect on “how interesting”, “how useful”, and “how engaging” the newsletter and Facebook posts were each week. They were asked to provide specific examples and to incorporate concepts and theories from assigned readings and course discussions into their reflections.

2.6. Statistical Analysis

Descriptive statistics were used for all baseline and engagement measures. Analysis of treatment satisfaction aggregated agree, somewhat agree, and strongly agree responses, and combined disagree, somewhat disagree, and strongly disagree, and descriptive data are provided. One sample test of proportions was used to determine whether there was a significant difference in knowledge from pre-post for each group, and two sample test of proportions was used to determine whether there was a significant difference between the change in knowledge from pre-post between the two groups. A p value of less than 0.05 was used to determine statistical significance. All statistical analyses were performed using SPSS version 22.0 (IBM Corp., Armonk, NY, USA).
Qualitative data analysis of the open-ended weekly reflections was conducted using a constant comparative method [52], employing analytic tools recommended by Corbin and Strauss (2008) to ensure quality data coding. First, one author coded reflections line-by-line, which allowed new concepts to emerge. Then, a codebook was developed based on emergent concepts. Next, axial coding identified patterns and cross-cutting concepts. Researchers met frequently throughout the implementation and evaluation of the intervention and reached unanimous consensus on patterns, themes, and conclusions emerging from the data.

3. Results

3.1. Sample Characteristics

We utilized a convenience sampling approach. All registered students in two sections of the advanced health communication class were eligible for and chose to enroll in the study. Each section had 29 students. Participants (n = 58) included juniors and seniors with a median age of 21.6 ± 2.2 years. Most participants self-identified as female (81%) and white (90%). Most participants reported health insurance coverage on their parent’s plan (93%). Almost all participants reported initiating sexual activity (97%) and half of participants (50%) reported currently being in a monogamous partnership. Most participants reported using the oral contraceptive pill (62%) and/or the condom (52%) as a method of contraception. There were no significant differences in the baseline characteristics of students in the two groups. Retention rates at posttest were high, with no significant differences between conditions (see Table 2).

3.2. HPV Vaccination Knowledge, Attitudes, and Behaviors

At baseline, the majority of participants in the HPV vaccination intervention group (n = 19; 66%) reported having received all three shots of the HPV vaccine series (see Table 3). At posttest, two additional participants reported receiving all three shots of the HPV vaccine and an additional three participants indicated that they planned to get vaccinated in the next 6 months. As a result, at the conclusion of the intervention, 24 participants (83%) indicated that they had completed the HPV vaccination or had a plan to receive the HPV vaccination in the next six months. Differences in HPV vaccination between the intervention and control group were not significant, nor were there significant differences in the change between groups at posttest (see Table 3).
At baseline, almost all participants knew that HPV can be spread through sexual intercourse (97%). Fewer participants were aware that HPV can be spread through contact other than sexual intercourse (64%). Approximately 91% of participants correctly responded that people who have been infected with HPV might not have symptoms. While participants understood the link between HPV and cervical cancer (93%), fewer participants knew that some types of HPV can cause anal cancer (53%), oral cancer (57%), or genital warts (72%).
At posttest, knowledge of HPV and HPV vaccination improved among participants in the HPV vaccination awareness intervention relative to those in the control condition (Table 4). Almost all intervention participants reported that HPV can be spread through contact other than sexual intercourse (93%). Participants understood that some types of HPV can cause oral cancer (100%), anal cancer (93%), and genital warts (100%). These improvements in knowledge were statistically significant compared to the control group (see Table 4).
At pretest, approximately 35% of all participants reported that not being currently sexually active influenced their intention to receive the HPV vaccination. At posttest, only 7% of the HPV vaccination intervention group felt the same way, and the difference between the intervention and control group from pretest to posttest was significant (p = 0.03).

3.3. Participants Reported Improvements in HPV Vaccination Knowledge, Attitudes, and Behaviors

Participants’ open-ended weekly reflections demonstrated an increase in knowledge about HPV vaccination (see Table 5). According to one participant, “since the beginning of this intervention, my knowledge of HPV drastically increased.” Participants believed that the intervention addressed gaps in knowledge related to men’s susceptibility to HPV and the link between HPV and oral cancer. Many students also reported that prior to the intervention they did not realize Student Health Services provided the vaccine on campus. In addition to increasing knowledge, students reported that the intervention changed their beliefs about HPV vaccination. Participants also described how increased knowledge led to improved confidence and inspired conversations about HPV vaccination with family and friends. According to one participant, “I now feel extremely confident in discussing all aspects of HPV, including the importance of vaccination.”

3.4. Intervention Engagement and Treatment Satisfaction

Electronic newsletters were opened each week by the majority of participants in the HPV condition, with most participants (≥90%) opening each weekly newsletter (see Figure 1). The proportion of participants who had at least one interaction on the private Facebook page (i.e., liked or commented on a post shared by study investigators) was lowest in week 1 (55%), but increased in subsequent weeks, with the highest rates (97%) during weeks 2 and 5 (see Figure 2). Participants made a total of 906 comments and likes over the intervention period, resulting in an average of 4.5 ± 1.5 per person per week (not including spring break), with slight variation from week to week (see Figure 2). In total, participants averaged 31.2 ± 45.1 comments and likes over the course of the intervention. Providing health education through articles was well received by participants based on their feedback. In addition, posts seeking to normalize discussions of HPV prevention among peers by including polls, YouTube clips, and integrated other social media platforms received greater interaction than health education posts.
Participants in the HPV vaccine awareness condition rated the intervention positively. Overall, 89% of participants indicated the intervention was helpful, 78% said that they would recommend it to a friend, and 70% reported they enjoyed the program. Most participants (67%) reported that they were satisfied with the number of lessons, the number of Facebook postings (56%), the length of the Facebook postings (63%), and the extent of interaction with health educators on Facebook (81%).

3.5. Participants Reported High Intervention Engagement and Treatment Satisfaction

Based on the open-ended weekly reflections, Facebook emerged as an acceptable platform for the intervention (see Table 5). Students appreciated the accessibility, flexibility, and ease of use of the social media platform. According to one participant, “I liked the flexibility of using Facebook. It was easy to log in and jump into the discussion whenever my schedule allowed it.” Even students who did not previously have a Facebook account found the platform simple to use. Some participants believed that the intervention became repetitive over time, especially on Facebook. Other participants described “missing” posts on Facebook. According to one participant, “the Facebook content can be a little heavy; since Facebook does not have the most simple [sic] layout, it is hard to make a post really stand out.” Participants appreciated fun and interactive elements of the intervention, such as polls. Participants also commented that Facebook encouraged interaction. According to one participant, “I really enjoyed conversing and sharing opinions with peers via a social network. I felt more inclined to share more personal information because it was online versus a face-to-face conversation.”
We also monitored level of engagement with the Facebook HPV intervention content (see Table 6). These data were deidentified but monitored throughout the intervention period. One hundred percent of HPV intervention participants participated in the private HPV Facebook group as part of the HPV condition. Each week, an average of 23.9 participants (ranging from 16–28) actively engaged in the group. Active engagement was measured by how many individuals commented, liked any post, and liked comments that were made on posts. Level of engagement was stratified as the following: high (21 or more) medium (11–20), and low (0–10). On average, there were 8.2 comments (ranging from 0–24), 4.5 comment likes (ranging from 0–29), and 3.7 post likes (ranging from 0–13) per week. On the individual level, each participant commented in the group an average of 2.4 times per week (ranging from 0–9).

4. Discussion

To our knowledge, this is the first study to deliver an HPV vaccination awareness intervention through Facebook and electronic newsletters to college students. This technology-mediated, theory-based HPV vaccination awareness intervention was well accepted by participants. Findings suggest the approach was effective in improving knowledge about HPV and HPV vaccination. Participants reported that the intervention addressed gaps in knowledge related to men’s susceptibility to HPV and the link between HPV and oral cancer, which have been shown in previous studies to be important gaps limiting HPV vaccination [53]. Participants also described how increased knowledge led to improved confidence and inspired conversations about HPV vaccination with family and friends. At the start of the 9-week study, approximately half of participants had previously received the full HPV vaccination series (57%), which represents a considerably higher rate for full vaccination coverage than might have been expected based on estimates for adolescents in South Carolina (43%) [8]. This may reflect the high rates of insurance coverage in this sample since previous studies have indicated greater likelihood of completing the full HPV vaccination series among individuals with health insurance [54]. Despite the high levels of vaccination at baseline, almost half of the participants were inadequately protected from HPV and associated HPV-cancers, with rates far below the Healthy People 2020 goal of increasing HPV vaccination coverage to 80%.
At the conclusion of the HPV intervention, the number of participants initiating/completing the vaccination series increased, as did the number of young adults indicating they intended to get vaccinated in the next 6 months. The change in vaccination coverage was modest over this 9-week period and did not differ statistically from the control group in this small feasibility study. However, the demonstrated increase in knowledge about HPV and vaccination coupled with the suggestion of a shift in intensions to get vaccinated suggests that the intervention offers a promising approach to target the catch-up population and achieve the ambitious Healthy People 2020 80% full coverage goal.
Participants’ intervention engagement and treatment satisfaction suggest that this approach offers utility and scalability among college students. The electronic newsletter showed high penetration with approximately 90% of participants opening the email each week. Almost all participants interacted on Facebook by “liking” a post or comment or posting a comment. An average of more than four comments or likes were made per person each week throughout the intervention [55]. Participants reported that Facebook was easy to use, accessible, and flexible, which likely contributed to the success of the intervention. Students also described how an online social media platform encouraged interaction and personal sharing, which reinforced intervention messages.

4.1. Implications for Practice

Although HPV vaccination is effective, low coverage remains a challenge for public health professionals. College students represent an important catch-up population who are at increased risk of acquiring HPV. This technology-mediated intervention supports research that social media serves as an appropriate platform to improve awareness and vaccination coverage [22]. Intervention messages and materials countered the dissemination of manipulative and false messages online in real-time, providing a platform to address sensational and misleading coverage [23]. Findings support the promise of eHealth by filling a gap in the evaluation of theory-based programs measuring engagement in order to improve health behaviors and outcomes [56]. Specifically, this study evaluated objective measures and qualitative in-depth reflections of engagement to show that the intervention resonated with participants.

4.2. Study Limitations and Strengths

Findings should be understood within the context of the study’s limitations. The small sample size and homogeneity of the population limit the generalizability of the results. The relatively short intervention and lack of extended follow-up to confirm receipt of HPV vaccinations are limitations of this study. Additionally, the relatively high baseline rate of full vaccination left modest room to increase vaccination rates. Finally, there is a risk of contamination between conditions because students were registered in two sections of the same course, with the same professor, at the same university and both interventions were described in the informed consent document.
This pilot study offers a number of strengths that demonstrate potential for scalability and dissemination of this intervention approach. In one of the first HPV vaccination awareness interventions for college students delivered through electronic newsletters and Facebook, participants demonstrated robust engagement and high treatment satisfaction. Moreover, the study was randomized and controlled, with a high retention rate. Additional study strengths include the use of objective measures and qualitative open-ended assessment of intervention engagement. Finally, the automated delivery of electronic newsletters and Facebook posts represents an opportunity to replicate the intervention with limited time and opportunity cost.

5. Conclusions

A technology-mediated HPV vaccination awareness intervention increased college students’ knowledge of HPV and HPV vaccination. Objective measures and qualitative open-ended assessment of the intervention showed high levels of engagement with the electronic newsletters and Facebook group. The effectiveness of this pilot study suggests that social media is an appropriate platform to reach college students with health promotion interventions and increase HPV vaccination awareness in this important catch-up population.

Author Contributions

Conceptualization, H.M.B., B.S., G.T.-M., D.S.W.; Formal analysis, H.M.B. and B.S.; Methodology, H.M.B., B.S., C.M.M., G.T.-M., D.S.W.; Writing—original draft, H.M.B. and B.S.; Writing—review and editing, H.M.B., B.S., G.T.-M., C.L., M.S., K.M., A.G., D.S.W. All authors have read and agreed to the published version of the manuscript.

Funding

This study was funded by internal funding from the University of South Carolina awarded to the Technology Center to Promote Healthy Lifestyles within the Arnold School of Public Health at the University of South Carolina. The funder had no role in the study conceptualization, design, execution, analysis, or interpretation of results.

Acknowledgments

We thank the students who willingly participated in the pilot study.

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

References

  1. Markowitz, L.E.; Dunne, E.F.; Saraiya, M.; Chesson, H.W.; Curtis, C.; Robinette, C.; Gee, J.; Bocchini, J.A., Jr.; Unger, E.R.; Centers for Disease Control and Prevention (CDC). Human Papillomavirus Vaccination: Recommendations of the Advisory Committee on Immunization Practices (ACIP). Morb. Mortal. Wkly. Rep. 2014, 63, 1–30. [Google Scholar]
  2. Centers for Disease Control and Prevention. Young People Account for a Substantial Proportion of New STIs. Division of STD Prevention: Atlanta, GA, USA, National Center for HIV/AIDS, Viral Hepatitis, STD, and TB Prevention; 2015. Available online: https://www.cdc.gov/std/products/infographics.htm (accessed on 14 November 2020).
  3. Centers for Disease Control and Prevention. The Link between HPV and Cancer; National Center for Chronic Disease Prevention and Health Promotion: Atlanta, GA, USA, 2015. Available online: https://www.cdc.gov/hpv/parents/cancer.html (accessed on 14 November 2020).
  4. Van Dyne, E.A.; Henley, S.J.; Saraiya, M.; Thomas, C.C.; Markowitz, L.E.; Benard, V.B. Trends in Human Papillomavirus–Associated Cancers—United States, 1999–2015. Morb. Mortal. Wkly. Rep. 2018, 67, 918–924. [Google Scholar] [CrossRef] [Green Version]
  5. Food and Drug Administration (FDA). FDA Approves Gardasil 9 for Prevention of Certain Cancers Caused by Five Additional Types of HPV; U.S. Department of Health and Human Services: Silver Spring, MD, USA, 2014. Available online: http://www.fda.gov (accessed on 14 November 2020).
  6. Markowitz, L.E.; Drolet, M.; Perez, N.; Jit, M.; Brisson, M. Human papillomavirus vaccine effectiveness by number of doses: Systematic review of data from national immunization programs. Vaccine 2018, 36, 4806–4815. [Google Scholar] [CrossRef]
  7. Spinner, C.; Ding, L.; Bernstein, D.I.; Brown, D.R.; Franco, E.L.; Covert, C.; Kahn, J. Human Papillomavirus Vaccine Effectiveness and Herd Protection in Young Women. Pediatrics 2019, 143. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  8. Walker, T.Y. National, Regional, State, and Selected Local Area Vaccination Coverage Among Adolescents Aged 13–17 Years—United States, 2017. Morb. Mortal. Wkly. Rep. 2018, 67. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  9. Barnard, M.; George, P.; Perryman, M.L.; Wolff, L.A. Human papillomavirus (HPV) vaccine knowledge, attitudes, and uptake in college students: Implications from the Precaution Adoption Process Model. PLoS ONE 2017, 12, e0182266. [Google Scholar] [CrossRef] [PubMed]
  10. Brown, D.R.; Shew, M.L.; Qadadri, B.; Neptune, N.; Vargas, M.; Tu, W.; Juliar, B.E.; Breen, T.E.; Fortenberry, J.D. A longitudinal study of genital human papillomavirus infection in a cohort of closely followed adolescent women. J. Infect. Dis. 2005, 191, 182–192. [Google Scholar] [CrossRef] [Green Version]
  11. Baranowski, T.; Cullen, K.W.; Basen-Engquist, K.; Wetter, D.W.; Cummings, S.; Martineau, D.S.; Prokhorov, A.; Chorley, J.; Beech, B.; Hergenroeder, A. Transitions out of High School: Time of Increased Cancer Risk? Prev. Med. 1997, 26, 694–703. [Google Scholar] [CrossRef]
  12. Gerend, M.A.; Shepherd, J.E. Predicting human papillomavirus vaccine uptake in young adult women: Comparing the health belief model and theory of planned behavior. Ann. Behav. Med. Publ. Soc. Behav. Med. 2012, 44, 171–180. [Google Scholar] [CrossRef] [Green Version]
  13. Gerend, M.A.; Shepherd, M.A.; Lustria, M.L.A.; Shepherd, J.E. Predictors of provider recommendation for HPV vaccine among young adult men and women: Findings from a cross-sectional survey. Sex. Transm. Infect. 2016, 92, 104–107. [Google Scholar] [CrossRef]
  14. Herrera-Ortiz, A.; Conde-Glez, C.J.; Olamendi-Portugal, M.L.; García-Cisneros, S.; Plett-Torres, T.; Sánchez-Alemán, M.A. College women, HPV genotyping and sexual behavior before HPV vaccination: Results from samples stored for a long time. J. Infect. Public Health 2018, 11, 286–289. [Google Scholar] [CrossRef] [PubMed]
  15. Jones, S.G.; Mathis-Gamble, K.; Fenkl, E.A. Minority College Students’ HPV Knowledge, Awareness, and Vaccination History. J. Assoc. Nurses. AIDS Care 2017, 28, 675–679. [Google Scholar] [CrossRef] [PubMed]
  16. Kasymova, S.; Harrison, S.E.; Pascal, C. Knowledge and Awareness of Human Papillomavirus among College Students in South Carolina. Infect Dis. 2019, 12, 1178633718825077. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  17. McCutcheon, T.; Schaar, G. HPV knowledge and vaccination rates in college-aged males: Implications for practice. Nurse Pract. 2017, 42, 49–53. [Google Scholar] [CrossRef] [PubMed]
  18. Smith, A.; Rainie, L.; Zickuhr, K. College Students and Technology. Pew Research Center. 2018. Available online: http://www.pewinternet.org/2011/07/19/college-students-and-technology/ (accessed on 14 November 2020).
  19. Pew Research Center. Internet/Broadband Fact Sheet. Washington, DC, USA, February 2018. Available online: https://www.pewinternet.org/fact-sheet/internet-broadband/ (accessed on 14 November 2020).
  20. Pew Research Center. Mobile Fact Sheet. Washington, DC, USA, February 2018. Available online: https://www.pewinternet.org/fact-sheet/mobile/ (accessed on 14 November 2020).
  21. Gramlich, J. 10 facts about Americans and Facebook. Pew Research Center. 2019. Available online: https://www.pewresearch.org/fact-tank/2019/02/01/facts-about-americans-and-facebook/ (accessed on 14 November 2020).
  22. Odone, A.; Ferrari, A.; Spagnoli, F.; Visciarelli, S.; Shefer, A.; Pasquarella, C.; Signorelli, C. Effectiveness of interventions that apply new media to improve vaccine uptake and vaccine coverage A systematic review. Hum. Vaccines Immunother. 2015, 11, 72–82. [Google Scholar] [CrossRef] [Green Version]
  23. Briones, R.; Nan, X.; Madden, K.; Waks, L. When vaccines go viral: An analysis of HPV vaccine coverage on YouTube. Health Commun. 2012, 27, 478–485. [Google Scholar] [CrossRef]
  24. Zhou, X.; Coiera, E.; Tsafnat, G.; Arachi, D.; Ong, M.-S.; Dunn, A.G. Using social connection information to improve opinion mining: Identifying negative sentiment about HPV vaccines on Twitter. Stud. Health Technol. Inform. 2015, 216, 761–765. [Google Scholar]
  25. Sundstrom, B.; Aylor, E.; Cartmell, K.B.; Brandt, H.M.; Bryant, D.C.; Hughes Halbert, C.; Young Pierce, J. Beyond the birds and the bees: A qualitative content analysis of online HPV vaccination communication. J. Commun. Healthc. 2018, 11, 205–214. [Google Scholar] [CrossRef]
  26. Cates, J.R.; Shafer, A.; Diehl, S.J.; Deal, A.M. Evaluating a County-Sponsored Social Marketing Campaign to Increase Mothers’ Initiation of HPV Vaccine for Their Preteen Daughters in a Primarily Rural Area. Soc. Mark. Q. 2011, 17, 4–26. [Google Scholar] [CrossRef] [Green Version]
  27. Cates, J.R.; Diehl, S.J.; Crandell, J.L.; Coyne-Beasley, T. Intervention effects from a social marketing campaign to promote HPV vaccination in preteen boys. Vaccine 2014, 32, 4171–4178. [Google Scholar] [CrossRef] [Green Version]
  28. Kennedy, A.; Sapsis, K.F.; Stokley, S.; Curtis, C.R.; Gust, D. Parental attitudes toward human papillomavirus vaccination: Evaluation of an educational intervention, 2008. J. Health Commun. 2011, 16, 300–313. [Google Scholar] [CrossRef] [PubMed]
  29. Cohen, E.L.; Head, K.J.; McGladrey, M.J.; Hoover, A.G.; Vanderpool, R.C.; Bridger, C.; Carman, A.; Crosby, R.A.; Darling, E.; Tucker-McLaughlin, M.; et al. Designing for Dissemination: Lessons in Message Design from “1-2-3 Pap”. Health Commun. 2015, 30, 196–207. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  30. Cohen, E.L.; Head, K.J. Identifying knowledge-attitude-practice gaps to enhance HPV vaccine diffusion. J. Health Commun. 2013, 18, 1221–1234. [Google Scholar] [CrossRef] [PubMed]
  31. Frank, L.B.; Murphy, S.T.; Chatterjee, J.S.; Moran, M.B.; Baezconde-Garbanati, L. Telling Stories, Saving Lives: Creating Narrative Health Messages. Health Commun. 2015, 30, 154–163. [Google Scholar] [CrossRef] [Green Version]
  32. Hopfer, S. Effects of a Narrative HPV Vaccination Intervention Aimed at Reaching College Women: A Randomized Controlled Trial. Prev Sci. 2012, 13, 173–182. [Google Scholar] [CrossRef]
  33. Krawczyk, A.; Lau, E.; Perez, S.; Delisle, V.; Amsel, R.; Rosberger, Z. How to inform: Comparing written and video education interventions to increase human papillomavirus knowledge and vaccination intentions in young adults. J. Am. Coll. Health 2012, 60, 316–322. [Google Scholar] [CrossRef]
  34. Macario, E.; Matiella, A.C. A bilingual webnovela on the human papillomavirus: Will Latinas and health professionals use it? J. Commun. Healthc. 2015, 8, 55–66. [Google Scholar] [CrossRef]
  35. Sundstrom, B.; Carr, L.A.; DeMaria, A.L.; Korte, J.E.; Modesitt, S.C.; Pierce, J.Y. Protecting the Next Generation: Elaborating the Health Belief Model to Increase HPV Vaccination among College-Age Women. Soc. Mark. Q 2015, 21, 173–188. [Google Scholar] [CrossRef]
  36. Balatsoukas, P.; Kennedy, C.M.; Buchan, I.; Powell, J.; Ainsworth, J. The Role of Social Network Technologies in Online Health Promotion: A Narrative Review of Theoretical and Empirical Factors Influencing Intervention Effectiveness. J. Med. Internet Res. 2015, 17, e141. [Google Scholar] [CrossRef]
  37. Betsch, C. Social media targeting of health messages. Hum. Vaccines Immunother. 2014, 10, 2636–2637. [Google Scholar] [CrossRef] [Green Version]
  38. Maher, C.A.; Lewis, L.K.; Ferrar, K.; Marshall, S.; De Bourdeaudhuij, I.; Vandelanotte, C. Are health behavior change interventions that use online social networks effective? A systematic review. J. Med. Internet Res. 2014, 16, e40. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  39. Raviotta, J.M.; Nowalk, M.P.; Lin, C.J.; Huang, H.-H.; Zimmerman, R.K. Using FacebookTM to Recruit College-Age Men for a Human Papillomavirus Vaccine Trial. Am. J. Mens Health 2016, 10, 110–119. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  40. Zhang, N.; Tsark, J.; Campo, S.; Teti, M. Facebook for Health Promotion: Female College Students’ Perspectives on Sharing HPV Vaccine Information through Facebook. Hawaii J. Med. Public Health 2015, 74, 136–140. [Google Scholar] [PubMed]
  41. Lai, C.-Y.; Wu, W.-W.; Tsai, S.-Y.; Cheng, S.-F.; Lin, K.-C.; Liang, S.-Y. The Effectiveness of a Facebook-Assisted Teaching Method on Knowledge and Attitudes about Cervical Cancer Prevention and HPV Vaccination Intention among Female Adolescent Students in Taiwan. Health Educ. Behav. 2015, 42, 352–360. [Google Scholar] [CrossRef] [PubMed]
  42. Sundstrom, B.; Brandt, H.M.; Gray, L.; Young Pierce, J. It’s My Time: Applying the health belief model to prevent cervical cancer among college-age women. J. Commun. Manag. 2018, 22, 161–178. [Google Scholar] [CrossRef]
  43. West, D.S.; Monroe, C.M.; Turner-McGrievy, G.; Sundstrom, B.; Larsen, C.; Magradey, K.; Wilcox, S.; Brandt, H.M. A Technology-Mediated Behavioral Weight Gain Prevention Intervention for College Students: Controlled, Quasi-Experimental Study. J. Med. Internet Res. 2016, 18. [Google Scholar] [CrossRef] [PubMed]
  44. Rosenstock, I.M. The health belief model and preventive health behavior. Health Educ. Behav. 1974, 2, 354–386. [Google Scholar] [CrossRef]
  45. Rosenstock, I.M.; Strecher, V.J.; Becker, M.H. Social learning theory and the health belief model. Health Educ. Behav. 1988, 15, 175–183. [Google Scholar] [CrossRef]
  46. Prochaska, J.O.; Velicer, W.F. The Transtheoretical Model of Health Behavior Change. Am. J. Health Promot. 1997, 12, 38–48. [Google Scholar] [CrossRef]
  47. Mills, L.A.; Head, K.J.; Vanderpool, R.C. HPV vaccination among young adult women: A perspective from Appalachian Kentucky. Prev. Chronic Dis. 2013, 10. [Google Scholar] [CrossRef] [Green Version]
  48. Hopfer, S.; Garcia, S.; Duong, H.T.; Russo, J.A.; Tanjasiri, S.P. A Narrative Engagement Framework to Understand HPV Vaccination among Latina and Vietnamese Women in a Planned Parenthood Setting. Health Educ. Behav. 2017, 44, 738–747. [Google Scholar] [CrossRef] [PubMed]
  49. Bandura, A. Health promotion by social cognitive means. Health Educ. Behav. Off Publ. Soc. Public Health Educ. 2004, 31, 143–164. [Google Scholar] [CrossRef] [PubMed]
  50. Group TDPP (DPP) R. The Diabetes Prevention Program (DPP): Description of lifestyle intervention. Diabetes Care 2002, 25, 2165–2171. [Google Scholar]
  51. Patel Divya, A.; Grunzweig, K.A.; Zochowski, M.K.; Dempsey, A.F.; Carlos, R.C.; Dalton, V.K. Human Papillomavirus Vaccine Stages of Change among Male and Female University Students: Ready or Not? J. Am. Coll. Health 2013, 61, 336–346. [Google Scholar] [CrossRef] [PubMed]
  52. Corbin, J.; Strauss, A. Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory, 3rd ed.; Sage Publications, Inc.: Thousand Oaks, CA, USA, 2008; 401p. [Google Scholar]
  53. McBride, K.R.; Singh, S. Predictors of Adults’ Knowledge and Awareness of HPV, HPV-Associated Cancers, and the HPV Vaccine: Implications for Health Education. Health Educ. Behav. 2018, 45, 68–76. [Google Scholar] [CrossRef] [PubMed]
  54. Kessels, S.J.M.; Marshall, H.S.; Watson, M.; Braunack-Mayer, A.J.; Reuzel, R.; Tooher, R.L. Factors associated with HPV vaccine uptake in teenage girls: A systematic review. Vaccine 2012, 30, 3546–3556. [Google Scholar] [CrossRef]
  55. Merchant, G.; Weibel, N.; Patrick, K.; Fowler, J.H.; Norman, G.J.; Gupta, A.; Servetas, C.; Calfas, K.; Raste, K.; Pina, L.; et al. Click “Like” to Change Your Behavior: A Mixed Methods Study of College Students’ Exposure to and Engagement With Facebook Content Designed for Weight Loss. J. Med. Internet Res. 2014, 16, e158. [Google Scholar] [CrossRef]
  56. Bull, S.; Ezeanochie, N. From Foucault to Freire through Facebook: Toward an Integrated Theory of mHealth. Health Educ. Behav. 2015. [Google Scholar] [CrossRef]
Figure 1. Participant engagement with HPV vaccination awareness newsletters (n = 29). Note: No electronic newsletters were sent by the study investigators during spring break (Week 5).
Figure 1. Participant engagement with HPV vaccination awareness newsletters (n = 29). Note: No electronic newsletters were sent by the study investigators during spring break (Week 5).
Vaccines 08 00749 g001
Figure 2. Participant engagement with HPV vaccination awareness intervention Facebook group. Note: No Facebook posts were made by the study investigators during spring break (Between Weeks 4 and 5).
Figure 2. Participant engagement with HPV vaccination awareness intervention Facebook group. Note: No Facebook posts were made by the study investigators during spring break (Between Weeks 4 and 5).
Vaccines 08 00749 g002
Table 1. Weekly lesson topics delivered through electronic newsletters and Facebook posts.
Table 1. Weekly lesson topics delivered through electronic newsletters and Facebook posts.
SessionTopics
1What is HPV?
2Am I at risk for HPV?
3How do I prevent HPV?
4How do I know if I have HPV?
5What are the risks and benefits of HPV vaccination?
6Should I get the HPV vaccine?
7Where do I go to get the HPV vaccine?
8Why do I need 3 doses?
Table 2. Baseline characteristics and retention rates.
Table 2. Baseline characteristics and retention rates.
MeasureTotal (n = 58)HPV (n = 29)Control (n = 29)
Age (SD), years21.6 (2.2)21.1 (0.8)22.1 (2.9)
Sex, n (%)
Female47 (81)24 (85)23 (79)
Male11 (19)5 (17)6 (21)
Hispanic, n (%)
Yes5 (9)3 (10)2 (7)
No53 (91)26 (90)27 (93)
Race, n (%)
Caucasian52 (90)28 (97)24 (83) *
African-American3 (5)1 (3)2 (7)
Asian2 (3)0 (0)2 (7)
American Indian/Native American1 (2)0 (0)1 (3)
Academic year, n (%)
2nd year 1 (2)0 (0)1 (3)
3rd year17 (29)13 (45)4 (14)
4th year33 (57)14 (48)19 (66)
5th year7 (12)2 (7)5 (17)
Health Insurance Status, n (%)
Parent’s Plan54 (93)28 (97)26 (90)
None3 (5)1 (3)2 (7)
Another plan 1 (2)0 (0.0)1 (3)
Age at first sexual intercourse (SD), years16.9 (1.6)17.1 (1.8)16.7 (1.3)
Number of sex partners (SD)
Past month 0.9 (0.6)10 (0.6)0.9 (0.6)
Past 6 months 1.9 (2.1)1.7 (1.2)2.1 (2.7)
Lifetime 7.0 (5.7)6.6 (5.0)7.5 (6.4)
Current monogamous partnership
Yes28 (50)14 (48)14 (52)
No26 (46)14 (48)12 (44)
Don’t Know2 (4)1 (3)1 (4)
Retention rates 56 (97%)29 (100%)27 (93%)
* p value < 0.05.
Table 3. HPV vaccination status and intentions.
Table 3. HPV vaccination status and intentions.
HPV (n = 29)Control (n = 29)
PrePostPrePost
Received full vaccination series (%)19 (66)21 (78)14 (48)15 (52)
Initiated partial vaccination series (%)2 (7)0 (0)3 (10)4 (14)
Never vaccinated (%)7 (24)3 (11)9 (31)8 (28)
Intend to get vaccinated within 6 months (%)1 (3)3 (11)3 (10)2 (7)
Fisher’s exact test was used to assess whether there were significant differences between and within groups at pre and post. No significant differences were found.
Table 4. HPV and HPV vaccination knowledge (%).
Table 4. HPV and HPV vaccination knowledge (%).
HPVControl
Pre
(n = 28)
Post
(n = 27)
Pre
(n = 29)
Post
(n = 29)
Difference in Change between Groups at Posttest (p-Value)
Some types of HPV can cause oral cancer.
Correct answer: True
65.5100 *48.369.00.007
Some types of HPV can cause anal cancer.
Correct answer: True
58.692.6 *48.365.50.04
Some types of HPV can cause genital warts.
Correct answer: True
78.6100 *69.082.80.024
HPV can be spread through contact other than sexual intercourse.
Correct answer: True
69.092.6 *58.669.00.072
Condom use fully protects against the spread of HPV.
Correct answer: False
69.092.675.982.80.42
HPV can cause an abnormal Pap (cervical cancer screening) test.
Correct answer: True
82.896.382.886.20.186
People who have been infected with HPV might not have symptoms.
Correct answer: True
86.210096.696.60.33
HPV can be spread through sexual intercourse.
Correct answer: True
10010093.11001.00
Some types of HPV can cause cervical cancer.
Correct answer: True
96.610089.793.10.165
An HPV infection can be cured.
Correct answer: False
27.63765.541.40.049
Women who get the vaccine still need regular Pap (cervical cancer screening) tests.
Correct answer: True
93.185.293.186.20.76
* p value < 0.05.
Table 5. Themes with illustrative quotes.
Table 5. Themes with illustrative quotes.
ThemesIllustrative Quotes
HPV vaccination knowledge, attitudes, and behaviors“After reading all of the posts and doing some other research I now believe that the vaccine is something everyone should have. I mean there is really no reason not to get vaccinated.”
“My viewpoint has definitely changed for the better. At the beginning of this study, I did not know too terribly much about HPV. At this point I know quite a bit and I know the facts concerning the matter. I’ve been able to relay this information to my friends and family and help them even take steps in the right direction to protect themselves.”
Intervention engagement and treatment satisfaction“I believe the material presented in this week’s Facebook post were user friendly and easy to understand and read. I am not familiar with the use of Facebook but I found it pretty easy to use and follow along with.”
“Many students even got to share personal stories of people who have had issues with HPV in the past. I believe this is important to share because if you personally know someone effected by HPV or even if it is just a friend of a friend, the situation or topic of HPV automatically becomes more personal and ‘hits home’ for a lot more people. I also enjoyed reading other people’s articles that they have found from other sources. The more information or reading on this topic helps for me to get a better idea of truly how serious HPV is.”
Table 6. Level of engagement with Facebook HPV intervention content.
Table 6. Level of engagement with Facebook HPV intervention content.
Level of EngagementComments
Number of Students
Post Likes
Number of Students
Comment Likes
Number of Students
High (21+)1114
Medium (11–20)1265
Low (0–10)62220
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Brandt, H.M.; Sundstrom, B.; Monroe, C.M.; Turner-McGrievy, G.; Larsen, C.; Stansbury, M.; Magradey, K.; Gibson, A.; West, D.S. Evaluating a Technology-Mediated HPV Vaccination Awareness Intervention: A Controlled, Quasi-Experimental, Mixed Methods Study. Vaccines 2020, 8, 749. https://doi.org/10.3390/vaccines8040749

AMA Style

Brandt HM, Sundstrom B, Monroe CM, Turner-McGrievy G, Larsen C, Stansbury M, Magradey K, Gibson A, West DS. Evaluating a Technology-Mediated HPV Vaccination Awareness Intervention: A Controlled, Quasi-Experimental, Mixed Methods Study. Vaccines. 2020; 8(4):749. https://doi.org/10.3390/vaccines8040749

Chicago/Turabian Style

Brandt, Heather M., Beth Sundstrom, Courtney M. Monroe, Gabrielle Turner-McGrievy, Chelsea Larsen, Melissa Stansbury, Karen Magradey, Andrea Gibson, and Delia Smith West. 2020. "Evaluating a Technology-Mediated HPV Vaccination Awareness Intervention: A Controlled, Quasi-Experimental, Mixed Methods Study" Vaccines 8, no. 4: 749. https://doi.org/10.3390/vaccines8040749

APA Style

Brandt, H. M., Sundstrom, B., Monroe, C. M., Turner-McGrievy, G., Larsen, C., Stansbury, M., Magradey, K., Gibson, A., & West, D. S. (2020). Evaluating a Technology-Mediated HPV Vaccination Awareness Intervention: A Controlled, Quasi-Experimental, Mixed Methods Study. Vaccines, 8(4), 749. https://doi.org/10.3390/vaccines8040749

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop