Health Information on COVID-19 Vaccination: Readability of Online Sources and Newspapers in Singapore, Hong Kong, and the Philippines
1
The Laboratory for Interdisciplinary Evaluation of Public Policies (LIEPP), Sciences Po, 75006 Paris, France
2
Oxford Institute of Population Ageing, University of Oxford, Oxford OX1 4BH, UK
3
Department of Languages, Faculty of Arts, University of Helsinki, Unioninkatu 38, P.O. Box 59, 00014 Helsinki, Finland
*
Author to whom correspondence should be addressed.
Journal. Media 2022, 3(1), 228-237; https://doi.org/10.3390/journalmedia3010017
Submission received: 8 December 2021
/
Revised: 12 March 2022
/
Accepted: 14 March 2022
/
Published: 18 March 2022
(This article belongs to the Collection Role of Media and Journalism during COVID-19 Pandemic: Lessons Learned and Future Challenges)
Abstract
To address the COVID-19 pandemic, as with other infectious diseases, a key intervention is vaccination. Health communications are thus of vital importance for informing the public on the benefits and risks of vaccines. This in turn makes the readability of media content fundamental. Previous studies of COVID-19-related information have found the readability of online information considerably more difficult than recommended. However, studies on the readability of information related to COVID-19 vaccination in Asian contexts have yet to be carried out. Furthermore, especially in the case of the current pandemic, health information is communicated by a variety of information channels, including the internet and mass media. This paper investigates the readability of textual information on COVID-19 vaccination found online and in newspaper articles in parts of Asia where English is one of the main languages, namely Singapore, Hong Kong, and the Philippines. Readability was assessed using a set of readability tests (Flesch–Kincaid Reading Ease, Flesch–Kincaid Grade Level, Gunning Fog Index, Coleman–Liau Index, and Simple Measure of Gobbledygook Grade level). It was found that a low proportion of URLs scored within recommended readability thresholds, and did so consistently across locations and types of sources. Furthermore, a relatively low proportion of web searches returned information from local sources; most URLs linked to sources outside of Singapore, Hong Kong, or the Philippines. Further, local online and newspaper sources scored similarly poorly on readability on average compared to non-local sources. Understanding of fast-evolving health communications concerning COVID-19 vaccination encompasses information about vaccine development and deployment from other locations, as well as locally. Nevertheless, these findings indicated a fairly low proportion of local sources among the top search results, and relatively low (i.e., difficult-to-read) readability scores for top search results and for local newspapers. An important issue for health communications strategies addressing COVID-19 vaccination will therefore be to consider different types of media sources in order to achieve the right mix of local and non-local sources while also ensuring appropriate readability.
Keywords:
readability; health communications; COVID-19; vaccine; mass media; newspapers; internet; Asia1. Introduction
One of the most important public health interventions in the eradication of infectious diseases is vaccination, and in particular achieving high coverage (Dubé et al. 2013; Gehrau et al. 2021). Accordingly, the impact of health communication activities related to the benefits and risks of vaccinations is of great importance. Various factors influence people’s views on vaccination, including their socio-cultural background, health professional recommendations, and vaccine policy, as well as information received from the media (Dubé et al. 2013; Lazarus et al. 2020; Costantini 2021; Michel and Goldberg 2021).
The increasing mediatization of medical information has led to a debate on the impact of such information on social behaviors, particularly with regard to vaccination programs (Breban 2011). The literature on this topic highlights the important role of the media in shaping public response to vaccination campaigns (Betsch et al. 2010; Liu et al. 2020). Strategic health communication channels and access to appropriate vaccination information are essential to address uncertain and complex situations such as the COVID-19 pandemic and to enable citizens to make informed decisions about accessing vaccination (Ferreira and Borges 2020; Gehrau et al. 2021). Sources of health information may influence individuals’ attitudes regarding vaccination and impact their intentions to seek vaccination (Gehrau et al. 2021).
Recent research indicates that, with today’s hybrid media system (Chadwick 2017), people use different platforms to seek information, including information regarding the COVID-19 pandemic (Casero-Ripollés 2020, 2021; Ferreira and Borges 2020). Evidence from the United States indicates that traditional media and digital media have varying effects on people’s receptivity to COVID-19 information (Casero-Ripollés 2020, 2021; Krause et al. 2020). However, there are persistent concerns as to whether COVID-19 information exacerbates misunderstanding as to the risks associated with the vaccine (Liu et al. 2020; Piltch-Loeb et al. 2021). Notably, a reliance on inappropriate information when COVID-19 vaccines were first introduced (Puri et al. 2020) has been a major factor in the lack of understanding of the risks, and the reluctance to get vaccinated (Cuan-Baltazar et al. 2020). Considering the impact of information disseminated in the media and of citizens’ levels of educational attainment on their views of the vaccines, the readability of media content is fundamentally important to enable informed decisions regarding COVID-19 vaccination.
Previous research has found that the readability of online information concerning COVID-19 is considerably more difficult than the reading levels recommended for patient educational information (Szmuda et al. 2020). In fact, information on COVID-19 in the U.S., the United Kingdom, Canada, and Ireland (Basch et al. 2020; Worrall et al. 2020; Kruse et al. 2021) and on COVID-19 testing (Garcia et al. 2021) is presented at a level that is difficult for the average person to understand. Consequently, scholars have called for more studies of local variation in information sources (Lim et al. 2021).
This study focuses on areas of South-East Asia where English is one of the main languages, namely Singapore, Hong Kong, and the Philippines. Comparative studies on the readability of information related to COVID-19 vaccination in South-East Asia have not yet been conducted. The study examines specific platforms used to seek information, namely online searches and newspapers. Appropriate readability in the context of vaccination uptake indicates that citizens will be able to access local information regarding vaccination programs. In the areas selected for the study, online sources available in English may contain non-local sources, including from the U.S. Even so, it is crucial to ensure that web searches enable appropriate access to locally relevant information.
In addition, this study examines local newspapers, which traditionally provide information from authorized sources and which are used to disseminate official information (Ferreira and Borges 2020). Newspapers must be considered because some people prefer to obtain information from sources other than the internet (Cuan-Baltazar et al. 2020; Casero-Ripollés 2020, 2021). Indeed, newspaper consumption is considered an indicator of broader media use and impact (Meyer et al. 2016). Specifically, the aim of this paper is to understand the ease of reading for information returned by online searches and newspaper articles about COVID-19 vaccination in South-East Asia, specifically Singapore, Hong Kong, and the Philippines. However, while this study evaluates the readability of English-language information sources related to COVID-19 vaccination, it does not address the entire media landscape, which would include other information sources such as television and social media as well as information disseminated in these locations in other languages such as Chinese and Tagalog.
2. Materials and Methods
2.1. Data
Data collection comprised a cross-sectional web search and newspaper article retrieval. The web search involved searching the term “COVID-19 vaccine” using the Google Chrome browser for Singapore, Hong Kong, and the Philippines. The browser history, cookies, and cache were first cleared, then Google location choice set to the relevant location. On the same day, 17 June 2021, the first fifty URL links returned by the search were recorded for each location, excluding any advertising or sponsored links. These URLs were classified based on the type of website, noting also whether the links were to websites belonging to government or multilateral organizations, mass media (e.g., newspaper) websites, medical information providers or medical service providers such as hospital websites, and whether such a source was local to Singapore, Hong Kong, or the Philippines, or was from elsewhere (e.g., a URL from the Singapore government versus a website from the U.S. Centers for Disease Control and Prevention [CDC]). URLs for each location were then cross-referenced to identify any URLs found in the top fifty for more than one location.
Newspaper article retrieval made use of the Factiva news database to access articles on COVID-19 vaccination during the two-week period (June 4 to 17, 2021) leading up to the date of the web search. To include articles in English with similar terms, the full-text search query used was: (COVID or “COVID-19” or coronavirus) and (vaccine or vaccination). For each location (i.e., Singapore, Hong Kong, and the Philippines), three major newspapers were included. In Singapore, these were The Straits Times, The New Paper and Today; in Hong Kong, The South China Morning Post, The Standard, and The Harbour Times; and in the Philippines, The Manila Times, The Philippine Star and The Philippine Daily Inquirer. For each newspaper, all articles that were retrieved were used in the analysis, aside from duplicate articles as identified by Factiva. A total of 1091 articles were found, including 159 duplicates, resulting in 932 articles used in the analysis.
2.2. Measures of Readability
The readability of the URLs and the newspaper articles was assessed with five readability tests, namely the Flesch–Kincaid Reading Ease (FRE), Flesch–Kincaid Grade Level, (FKGL), Gunning Fog Index (GFI), Coleman–Liau Index (CLI), and Simple Measure of Gobbledygook (SMOG) Grade level (Added Bytes 2021). Each test places emphasis and weight on different aspects that contribute to readability: sentence length (FRE, FKGL, GFI, CLI), number of syllables per word (FRE and FKGL), proportion of difficult words (GFI), number of letters per word (CLI), and proportion of words with three or more syllables (SMOG). The FRE scores are on a 0–100 scale, with a higher score indicating increased reliability (scores above 80 indicate ‘easy’ reading). The other four provide a score based on U.S. school grades. Lower scores thus correspond to texts that are easier to read. In terms of age, grade 1 corresponds to ages 6–7 and grade 12 corresponds to ages 17–18. In the U.S., overall guidance is to achieve readability below grade 7, which corresponds to ages 12–13 (U.S. Department of Health and Human Services 2010; McKenzie et al. 2017). Accordingly, the analysis investigated the proportion of sources scoring below this threshold, and their average readability.
2.3. Statistical Analysis
A p-value of less than 0.05 was considered to indicate statistical significance for all analyses. The data were analyzed using STATA (Statistics Data Analysis, Version MP-13.1 for Windows, StataCorp LP, College Station, TX, USA).
3. Results
Among the 150 URLs collected (50 each for Singapore, Hong Kong, and the Philippines), there were 105 unique URLs, as some URLs were retrieved for two or more locations (Figure 1).
Considering the 50 URLs for each location, the mean readability of the respective 50 URLs (Table 1) was between grades 8.8 and 11.9 for the FKGL, GFI, CLI and SMOG and between 43 and 47 (‘difficult’) for the FRE, all of which were statistically different from the ease of reading threshold (p < 0.001). The proportion of URLs with easier readability (i.e., FRE above 80 and other tests below 7) was based on the FKGL, GFI or CLI (0–28% depending on location) and the FRE and SMOG (0% in all three locations). Thus, most readability tests indicated a low proportion of URLs scoring within readability thresholds. To assess similarity across locations, the mean readability of the 50 URLs for each location was compared to that of the 55 unique URLs from the other two locations (i.e., the 50 URLs of a given location were subtracted from the total 105 unique URLs, leaving 55 remaining unique URLs as comparators). For each of the readability tests none of the differences were statistically significant (p > 0.05).
To understand readability patterns, URLs were compared based on four source categories. Governments and multilateral agencies (e.g., WHO) accounted for 51 URLs, of which 18 were from the Canadian government (at both federal and provincial levels) and 14 were from the U.S. (at both federal and state levels), of which seven were from the CDC and Food and Drug Administration (FDA). Mass media (e.g., newspapers, CNN) accounted for 23 URLs, of which six were from mass media sources from Singapore, Hong Kong, or the Philippines, and 17 from international mass media sources. Medical information (e.g., WebMD) and healthcare services (e.g., hospitals) comprised 19 URLs. The remaining 12 URLs were grouped in an “other” category that included vaccine producers’ corporate websites, data sources, Wikipedia, and one non-profit organization.
Comparing the mix of categories by location (Table 2), the proportion of URLs was: 50–54% from governments and multilateral agencies, 18–24% from mass media sources, 12–20% from medical information and healthcare services, and 12–14% from other sources. The difference in readability by category (Table 3) relative to the easy readability thresholds was statistically significant (p < 0.001) for all four categories and five tests, with the single exception of the GFI test for other sources (p = 0.31). In addition, average readability was similar for each of the four categories, as the difference in means for each test was: 11.5 points for FRE; under 1.1 for FKGL, CLI and SMOG; and 2.4 for GFI. Thus, all four categories of URLs had broadly similar levels of readability by most measures.
In comparing across locations, an important aspect to consider is that some URLs appear in the top 50 URLs for multiple locations. Of the 150 URLs collected, 105 were unique, while 35 featured in two or three locations. Among sources returned for just one location and sources prevalent across more than one location, the overall pattern of readability was similar: neither set of sources had easier readability (Table 4). In addition, when the prevalence of local sources was assessed, it was found that of the 105 unique URLs, 21 were from sources specific to one of the three locations, while the other 84 were from outside sources. Hence, the vast majority of sources were not local. Considering URLs retrieved for only one location, most were not location specific. Of the 70 URLs that were unique to one location, 20 were from sources from within the location whereas 50 were from sources beyond the location. Among the three locations, Singapore had the most local sources (11), and the Philippines the least (3). Furthermore, local sources were not found to score easier on readability, on average. Thus, most URLs were not local to any of the locations studied, with local sources being particularly limited for the Philippines.
Given the limited availability of local URLs in the top search results, to assess access to information tailored to a local audience, articles from three major English-language newspapers in each location were compared for readability (Table 5). Comparing across the three locations, the newspapers in the Philippines were on average the hardest to read and had the narrowest range for ease of reading, with differences for the FKGL, GFI, CLI and SMOG tests of 0.4 to 1.0 grade points. In contrast, the newspapers chosen for Singapore and Hong Kong each included one newspaper that was easier to read than the other two for all but one measure. Here, differences were found for the FKGL, GFI, CLI and SMOG tests between the easiest and hardest to read articles of 0.5 to 1.8 grade points for Singapore and 1.2 to 2.1 grade points for Hong Kong.
4. Discussion and Conclusions
Access to information on COVID-19 vaccines that enables citizens to make informed decisions on vaccination uptake and arms them with the knowledge of how to access vaccinations is a key to addressing the pandemic. The rapidly evolving global pandemic, the fast development of vaccines, and the initial deployments of heterogeneous vaccines across countries suggest the importance of the cross-border flow of vaccine-related information. At the same time, vaccine uptake is subject to local availability, processes, and considerations, indicating the need for local information. Throughout, appropriate readability is critical to ensure widespread access to relevant information.
This study centered on South-East Asia, which is relatively understudied in terms of readability studies related to COVID-19, with a focus on Singapore, Hong Kong, and the Philippines. Across all three locations, the sources analyzed (i.e., top-ranking URLs and newspaper articles) had readability scores under thresholds for readability used in the U.S. These American benchmarks reflect local contexts, including English reading skills, which are affected by schooling and immigration (U.S. Department of Health and Human Services 2010), leading to recommendations for readability levels below grade 7 (McKenzie et al. 2017). The locations focused on, Singapore, Hong Kong, and the Philippines, have high adult literacy rates of 97.3%, 93.3% and 98.2%, respectively (UNESCO 2021; Social Indicators of Hong Kong 2021); proficiency in English is rated as very high for Singapore, high for the Philippines and moderate for Hong Kong (EF 2021). In all three of these locations, English is only one of several official languages, and other languages are also used in mass media sources. Thus, in the context of a pandemic, in countries where English is an official language, part of the English-language media would aim for widespread penetration of vaccination information. At the same time, certain subsets of the population with distinct educational levels may access (and be targeted by) English language media. Indeed, considering the importance of language to citizens’ diverse access to health information (Khan et al. 2020), the appropriate readability threshold in a given language would ideally also reflect literacy and availability of information in other languages. Such an assessment is, however, beyond the scope of this study. Thus, while the exclusive focus of the current study on English language sources constitutes a limitation, at the same time this does enable comparison of sources across the three locations, including access to information beyond local sources.
The analysis of URLs highlighted the importance of official government sources and of established mass media and information services, which accounted for the majority of the top 50 URLs for each location. Concerns around the potential lack of information or the prevalence of misinformation relating to COVID-19 on the internet (Cuan-Baltazar et al. 2020) are at least in part allayed in the case of searchable online content, since the results consist primarily of official sources and established mass media outlets. Indeed, just over half of the URLs were from government sources, including federal, local, and health agencies. The URLs from Canada and the U.S. accounted for the majority of government URLs. While there were five links to Singapore government URLs, there was only one government link each for Hong Kong and the Philippines. At the same time, citizens were able to access certain sites by means other than online searches, such by visiting bookmarked or memorized pages, or via apps on mobile devices. This also pertains to mass media, for which citizens may subscribe online. Given the predominance of non-local sources in internet search results, an interesting question is whether such alternative means of accessing online sources, as well as newspapers and other media, alters the balance of local and non-local sources accessed. This suggests the importance in considering sources such as major local newspapers as information sources. Indeed, the differences in readability indicated for newspapers in Singapore and Hong Kong suggests the potential for media sources to adopt distinct health communication strategies, including with regards to readability, to assist in reaching heterogeneous audiences. In addition, the main newspaper in Singapore, The Straits Times, published a distinctly higher volume of articles during the period analyzed; further research may be able to shed light on the extent to which such media coverage was intended to complement public health messaging (Araújo et al. 2021), as well as on the potential variation in readability across articles. In contrast, the newspapers in the Philippines tended to be harder to read and had a narrower range of readability. While this may have reflected their audience demographics, this also meant these newspapers provided less of a complement to web search results that had few local easier-to-read sources.
Thus, the URL analysis should be interpreted by bearing in mind that information accessed through such searches is complementary to other online and offline sources, which may in part explain the prevalence of non-local sources. While such non-local sources varied in readability, they typically remained above U.S.-recommended readability thresholds, including with regard to sources themselves traced to the U.S. However, in interpreting the documented level of readability, two additional considerations are required. First, individual URLs may differ in terms of their target audience, not least during a pandemic, and correspondingly be designed for a particular level of readability. Nonetheless, during a fast-evolving pandemic the need for the widespread dissemination of information suggests the need to aim for ease of readability. Second, the Google search ranking places weight on sites that serve as references to other sites rather than those most frequently accessed by users. In the context of a pandemic, such sites might be expected to convey information in a more technical, deeper, and/or more nuanced manner than other sites that link to them. This suggests a potential general limitation for using web search-based rankings of URLs to assess the most frequently used information sources. Thus, the impact of the ranking system on the readability of selected sites merits further study, in particular within the context of public health communications during crises such as the COVID-19 pandemic.
This in turn highlights the scope for mass media and other information sources to serve a complementary role in informing citizens. Newspaper coverage has been shown to impact vaccine uptake (Mason and Donnelly 2000). The search results had a mix of mass media and more specialist health information sources, which were from dedicated health information sites as well as from healthcare providers (such as hospitals). Characterizing these sources as a list invites the interpretation of mass media and specialist sources as substitutes, whereas they may be better understood as complementary. For instance, sources intended for a more general readership may cite more specialist sources. In addition, the pandemic has led specialist providers to curate COVID-19 collections and make these accessible. Thus, top search sites may share important interconnections in terms of how information flows from specialist sources to reach a mass audience. A relevant issue for further investigation is thus the extent to which scientific information and knowledge (e.g., medical researchers’ findings in medical journals) is utilized, particularly by newspapers and other mass media sources. As it is essential for the public to be able to access and understand such information in a timely manner during a health crisis such as COVID-19, the current pandemic represents an opportunity to understand the effectiveness of the flow of information from originators (e.g., researchers), through various intermediaries (e.g., including government agencies and the media), and ultimately to citizens. Within this flow of information, the ways in which communication is adapted, including in terms of readability, is an important issue at each stage of the process.
Finally, when looking at information provision and access with regard to COVID-19 vaccination and health issues in general, interplay with healthcare professionals is an important consideration. In particular, the evidence shows the complementarity of medical professional recommendations and media coverage in driving vaccine uptake (Ma et al. 2006; Gehrau et al. 2021), as well as the importance of information provision in shaping vaccine literacy (Michel and Goldberg 2021). Furthermore, interactions between patients and doctors is culturally nuanced (Puna et al. 2018), including in relation to locations such as those this paper focuses on. As evidenced by this pandemic, the concept of ‘health’ is culturally, economically, and politically constructed, and local understandings of health are therefore important. This points to the need for an awareness of the power of mass media and other information sources to perpetuate culturally constructed notions of health, and for this information to be grounded in an evolving base of medical knowledge and understanding.
Author Contributions
Conceptualization, H.C. and R.F.; methodology, H.C. and R.F.; analysis, H.C.; data curation, H.C.; writing—original draft preparation, H.C. and R.F.; writing—review and editing, H.C. and R.F. All authors have read and agreed to the published version of the manuscript.
Funding
For Hiroko Costantini this research was funded by AXA Research Fund.
Data Availability Statement
The data presented in this study are openly available in SciencesPo registry upon publication and up to then upon request.
Conflicts of Interest
The authors declare no conflict of interest.
References
- Added Bytes. 2021. Readable. How Readable is Your Writing? Brighton. Available online: https://readable.com (accessed on 5 April 2021).
- Araújo, Rita, Felisbela Lopes, Olga Magalhães, Alberto Sá, and Ana Aguiar. 2021. Behavior Guidance during the COVID-19 Pandemic: Health Literacy as a Weapon against the Virus. Health Communication, 1–9. [Google Scholar] [CrossRef]
- Basch, Corey H., Jan Mohlman, Grace Clarke Hillyer, and Philip Garcia. 2020. Public health communication in time of crisis: Readability of on-line COVID-19 information. Disaster Medicine and Public Health Preparedness 14: 635–37. [Google Scholar] [CrossRef] [PubMed]
- Betsch, Cornelia, Frank Renkewitz, Tilmann Betsch, and Corina Ulshöfer. 2010. The influence of vaccine-critical websites on perceiving vaccination risks. Journal of Health Psychology 15: 446–55. [Google Scholar] [CrossRef] [PubMed]
- Breban, Romulus. 2011. Health newscasts for increasing influenza vaccination coverage: An inductive reasoning game approach. PLoS ONE 6: e28300. [Google Scholar] [CrossRef] [PubMed]
- Casero-Ripollés, Andreu. 2020. Impact of COVID-19 on the media system. Communicative and democratic consequences of news consumption during the outbreak. El Profesional de la Información 29: e290223. [Google Scholar] [CrossRef]
- Casero-Ripollés, Andreu. 2021. The Impact of COVID-19 on Journalism: A Set of Transformations in Five Domains. Comunicação e Sociedade 40: 53–69. [Google Scholar] [CrossRef]
- Chadwick, Andrew. 2017. The Hybrid Media System: Politics and Power. Oxford: Oxford University Press. ISBN 978 0190696733. [Google Scholar]
- Costantini, Hiroko. 2021. COVID-19 Vaccine Literacy of Family Carers for Their Older Parents in Japan. Healthcare 9: 1038. [Google Scholar] [CrossRef]
- Cuan-Baltazar, Jose Yunam, Maria Jose Muñoz-Perez, Carolina Robledo-Vega, Maria Fernanda Pérez-Zepeda, and Elena Soto-Vega. 2020. Misinformation of COVID-19 on the Internet: Infodemiology Study. JMIR Public Health Surveill 6: e18444. [Google Scholar] [CrossRef]
- Dubé, Eve, Caroline Laberge, Maryse Guay, Paul Bramadat, Real Roy, and Julie Bettinger. 2013. Vaccine hesitancy: An overview. Human Vaccines & Immunotherapeutics 9: 1763–73. [Google Scholar] [CrossRef]
- EF. 2021. English Proficiency Index. Available online: https://www.ef.sg/epi/ (accessed on 14 April 2021).
- Ferreira, Gil Baptista, and Susana Borges. 2020. Media and Misinformation in Times of COVID-19: How People Informed Themselves in the Days Following the Portuguese Declaration of the State of Emergency. Journalism and Media 1: 108–21. [Google Scholar] [CrossRef]
- Garcia, Philip, Joseph Fera, Jan Mohlman, and Corey H. Basch. 2021. Assessing the readability of COVID-19 testing messages on the internet. Journal of Community Health 46: 913–17. [Google Scholar] [CrossRef]
- Gehrau, Volker, Sam Fujarski, Hannah Lorenz, Carla Schieb, and Bernd Blöbaum. 2021. The Impact of Health Information Exposure and Source Credibility on COVID-19 Vaccination Intention in Germany. International Journal of Environmental Research and Public Health 18: 4678. [Google Scholar] [CrossRef]
- Khan, Sobia, Ashar Asif, and Ali Emad Jaffery. 2020. Language in a time of COVID-19: Literacy bias ethnic minorities face during COVID-19 from online information in the UK. J. Racial and Ethnic Health Disparities 8: 1242–48. [Google Scholar] [CrossRef]
- Krause, Nicole M., Isabelle Freiling, Becca Beets, and Dominique Brossard. 2020. Fact-checking as risk communication: The multi-layered risk of misinformation in times of COVID-19. Journal of Risk Research 23: 1052–59. [Google Scholar] [CrossRef]
- Kruse, Jessica, Paloma Toledo, Tayler B Belton, Erica J Testani, Charlesnika T. Evans, William A. Grobman, Emily S. Miller, and Elizabeth M. S. Lange. 2021. Readability, content, and quality of COVID-19 patient education materials from academic medical centers in the United States. American Journal of Infection Control 49: 690–93. [Google Scholar] [CrossRef] [PubMed]
- Lazarus, Jeffrey V., Katarzyna Wyka, Lauren Rauh, Kenneth Rabin, Scott Ratzan, Lawrence O. Gostin, Heidi J. Larson, and Ayman El-Mohandes. 2020. Hesitant or Not? The Association of Age, Gender, and Education with Potential Acceptance of a COVID-19 Vaccine: A Country-level Analysis. Journal of Health Communication 25: 799–807. [Google Scholar] [CrossRef]
- Lim, Sean-Tee, Martin Kelly, and Sean Johnston. 2021. Re ‘Readability of online patient education material for the novel coronavirus disease (COVID-19): A cross-sectional health literacy study’. Public Health 190: 145–46. [Google Scholar] [CrossRef] [PubMed]
- Liu, Zhan, Jialu Shan, Matthieu Delaloye, Jean-Gabriel Piguet, and Nicole Glassey Balet. 2020. The Role of Public Trust and Media in Managing the Dissemination of COVID-19-Related News in Switzerland. Journalism and Media 1: 145–59. [Google Scholar] [CrossRef]
- Ma, Kimberly K., William Schaffner, C. Colmenares, J. Howser, J. Jones, and Katherine A. Poehling. 2006. Influenza vaccination of young children increased with media coverage in 2003. Pediatrics 117: e157–e63. [Google Scholar] [CrossRef]
- Mason, Brendan, and Peter Donnelly. 2000. Impact of a local newspaper campaign on the uptake of the measles mumps and rubella vaccine. Journal of Epidemiology and Community Health 54: 473–74. [Google Scholar] [CrossRef][Green Version]
- McKenzie, James F., Brad L. Neiger, and Rosemary Thackeray. 2017. Planning, Implementing, and Evaluating Health Promotion Programs: A Primer, 7th ed. New York: Pearson. [Google Scholar]
- Meyer, Samantha, Stephanie K. Lu, Laurie Hoffman-Goetz, Bryan Smale, Heather MacDougall, and Alex R. Pearce. 2016. A content analysis of newspaper coverage of the seasonal flu vaccine in Ontario, Canada, October 2001 to March 2011. Journal of Health Communication 21: 1088–97. [Google Scholar] [CrossRef]
- Michel, Jean-Pierre, and J. Goldberg. 2021. Education, Healthy Ageing and Vaccine Literacy. The Journal of Nutrition, Health & Aging 25: 698–701. [Google Scholar] [CrossRef]
- Piltch-Loeb, Rachael, Elena Savoia, Beth Goldberg, Brian Hughes, Tanner Verhey, Juliette Kayyem, Cynthia Miller-Idriss, and Marcia Testa. 2021. Examining the effect of information channel on COVID-19 vaccine acceptance. PLoS ONE 16: e0251095. [Google Scholar] [CrossRef]
- Pun, Jack, E. Angela Chan, Sophie Wang, and Diana Slade. 2018. Health professional-patient communication practices in East Asia: An integrative review of an emerging field of research and practice in Hong Kong, South Korea, Japan, Taiwan, and Mainland China. Patient Education and Counseling 101: 1193–206. [Google Scholar] [CrossRef] [PubMed]
- Puri, Neha, Eric A. Coomes, Hourmazd Haghbayan, and Keith Gunaratne. 2020. Social media and vaccine hesitancy: New updates for the era of COVID-19 and globalized infectious diseases. Human Vaccines & Immunotherapeutics 16: 2586–93. [Google Scholar]
- Social Indicators of Hong Kong. 2021. Available online: https://www.socialindicators.org.hk/en/indicators/education/7.7 (accessed on 15 March 2021).
- Szmuda, Tomasz, Cathrine Özdemir, Shan Ali, Akshita Singh, Mohammad Talha Syed, and Pawel Słoniewski. 2020. Readability of online patient education material for the novel coronavirus disease (COVID-19): A cross-sectional health literacy study. Public Health 185: 21–25. [Google Scholar] [CrossRef]
- U.S. Department of Health and Human Services. 2010. Toolkit for Making Wiring Material Clear and Effective; Baltimore: Centers for Medicare & Medicaid Services.
- UNESCO. 2021. Education and Literacy. Available online: http://uis.unesco.org/country/SG (accessed on 15 March 2021).
- Worrall, Amy P., Mary J. Connolly, Aine O’Neill, Murray O’Doherty, Kenneth P. Thornton, Cora McNally, Samuel J. McConkey, and Eoghan de Barra. 2020. Readability of online COVID-19 health information: A comparison between four English speaking countries. BMC Public Health 20: 1635. [Google Scholar] [CrossRef] [PubMed]
Figure 1.
Distribution of 50 URLs retrieved for each location, showing number of URLs retrieved for one, two, or all three locations, resulting in 105 unique URLs.
Figure 1.
Distribution of 50 URLs retrieved for each location, showing number of URLs retrieved for one, two, or all three locations, resulting in 105 unique URLs.
Table 1.
Readability test scores by location of search.
| Readability Test *: | FRE | FKGL | GFI | CLI | SMOG |
|---|---|---|---|---|---|
| (a) Singapore | |||||
| Mean | 43.3 | 9.7 | 8.8 | 11.6 | 11.1 |
| Standard deviation | 13.6 | 2.2 | 3.4 | 2.4 | 2.1 |
| Readability easier (% of URLs) ** | 0% | 10% | 28% | 6% | 0% |
| p-value *** | <0.001 | <0.001 | <0.001 | <0.001 | <0.001 |
| (b) Hong Kong | |||||
| Mean | 46.8 | 9.4 | 9.4 | 11.7 | 11.1 |
| Standard deviation | 11.5 | 2.1 | 3.0 | 2.0 | 1.9 |
| Readability easier (% of URLs) ** | 0% | 14% | 20% | 0% | 0% |
| p-value *** | <0.001 | <0.001 | <0.001 | <0.001 | <0.001 |
| (c) Philippines | |||||
| Mean | 45.3 | 9.9 | 9.7 | 11.9 | 11.7 |
| Standard deviation | 11.4 | 2.4 | 3.6 | 2.0 | 2.3 |
| Readability easier (% of URLs) ** | 0% | 10% | 22% | 2% | 0% |
| p-value *** | <0.001 | <0.001 | <0.001 | <0.001 | <0.001 |
* Flesch–Kincaid Reading Ease (FRE), Flesch–Kincaid Grade Level, (FKGL), Gunning Fog Index (GFI), Coleman–Liau Index (CLI), and Simple Measure of Gobbledygook (SMOG) Grade level. ** Readability rated as easier if score ≥80 for FRE, ‘easy’; or below grade 7 for other tests. *** Test of difference of mean from threshold for easier readability.
Table 2.
Proportion of URLs by source category for each location.
| Singapore | Hong Kong | Philippines | Average | |
|---|---|---|---|---|
| % | % | % | % | |
| Governments and multilateral agencies | 54 | 52 | 50 | 52.0 |
| Mass media sources | 14 | 24 | 18 | 18.7 |
| Medical information and healthcare services | 18 | 12 | 20 | 16.7 |
| Other | 14 | 12 | 12 | 12.7 |
Table 3.
Readability test scores by source category.
| Readability Test *: | FRE | FKGL | GFI | CLI | SMOG |
|---|---|---|---|---|---|
| (a) Governments and multilateral agencies (N = 51) | |||||
| Mean | 44.9 | 9.6 | 9.3 | 11.8 | 11.1 |
| Standard deviation | 12.0 | 2.1 | 3.4 | 2.3 | 2.1 |
| Readability easier (% of URLs) ** | 0% | 8% | 24% | 4% | 0% |
| p-value *** | <0.001 | <0.001 | <0.001 | <0.001 | <0.001 |
| (b) Mass media (N = 23) | |||||
| Mean | 48.5 | 9.5 | 9.7 | 11.5 | 11.8 |
| Standard deviation | 9.6 | 2.2 | 2.9 | 1.8 | 2.1 |
| Readability easier (% of URLs) ** | 0% | 13% | 13% | 0% | 0% |
| p-value *** | <0.001 | <0.001 | <0.001 | <0.001 | <0.001 |
| (c) Medical information and healthcare services (N = 19) | |||||
| Mean | 47.0 | 10.0 | 10.7 | 11.9 | 12.1 |
| Standard deviation | 10.5 | 2.5 | 3.3 | 1.7 | 2.4 |
| Readability easier (% of URLs) ** | 0% | 16% | 16% | 0% | 0% |
| p-value *** | <0.001 | <0.001 | <0.001 | <0.001 | <0.001 |
| (d) Other sources (N = 12) | |||||
| Mean | 37.0 | 10.6 | 8.3 | 11.3 | 11.2 |
| Standard deviation | 16.1 | 2.6 | 4.3 | 3.4 | 2.8 |
| Readability easier (% of URLs) ** | 0% | 8% | 50% | 0% | 0% |
| p-value *** | <0.001 | <0.001 | 0.31 | <0.001 | <0.001 |
* Flesch–Kincaid Reading Ease (FRE), Flesch–Kincaid Grade Level, (FKGL), Gunning Fog Index (GFI), Coleman–Liau Index (CLI), and Simple Measure of Gobbledygook (SMOG) Grade level. ** Readability rated as easier if score: ≥ 80 for FRE, ‘easy’; or below grade 7 for other tests. *** Test of difference of mean from threshold for easier readability.
Table 4.
Number of URLs and mean readability scores by presence of URL across locations and by origin of source.
Table 4.
Number of URLs and mean readability scores by presence of URL across locations and by origin of source.
| Number of URLs and Readability Tests *: | Number | FRE | FKGL | GFI | CLI | SMOG |
|---|---|---|---|---|---|---|
| (a) Sources from within Singapore, Hong Kong, or the Philippines | ||||||
| 21 URLs from sources within the three locations, which were available in: | ||||||
| Singapore, Hong Kong, and Philippines | 0 | n/a | n/a | n/a | n/a | n/a |
| Singapore and Hong Kong | 0 | n/a | n/a | n/a | n/a | n/a |
| Singapore and Philippines | 1 | 44.3 | 8.3 | 6.7 | 11.4 | 9.6 |
| Hong Kong and Philippines | 0 | n/a | n/a | n/a | n/a | n/a |
| Singapore only | 10 | 48.7 | 9.6 | 10.0 | 11.6 | 11.9 |
| Hong Kong only | 7 | 49.2 | 8.8 | 8.3 | 11.6 | 10.9 |
| Philippines only | 3 | 30.7 | 12.4 | 10.7 | 14.7 | 14.5 |
| (b) Source from elsewhere | ||||||
| 84 URLs from sources elsewhere than three locations, which were available in: | ||||||
| Singapore, Hong Kong, and Philippines | 10 | 43.8 | 9.4 | 8.2 | 11.8 | 10.6 |
| Singapore and Hong Kong | 6 | 39.0 | 10.5 | 10.4 | 13.0 | 11.5 |
| Singapore and Philippines | 11 | 49.1 | 9.4 | 9.9 | 11.9 | 11.5 |
| Hong Kong and Philippines | 7 | 48.2 | 8.6 | 8.1 | 11.2 | 10.3 |
| Singapore only | 12 | 35.3 | 10.2 | 6.8 | 10.6 | 10.3 |
| Hong Kong only | 20 | 49.4 | 9.5 | 10.5 | 11.5 | 11.6 |
| Philippines only | 18 | 45.1 | 10.7 | 11.1 | 11.8 | 12.6 |
* Flesch–Kincaid Reading Ease (FRE), Flesch–Kincaid Grade Level, (FKGL), Gunning Fog Index (GFI), Coleman–Liau Index (CLI), and Simple Measure of Gobbledygook (SMOG) Grade level.
Table 5.
Readability scores of major newspapers by location.
| Readability Test *: | ||||||
|---|---|---|---|---|---|---|
| Number of Articles ** | FRE | FKGL | GFI | CLI | SMOG | |
| (a) Singapore | ||||||
| Newspaper 1 | 28 | 53.0 | 9.5 | 10.5 | 11.3 | 12.3 |
| Newspaper 2 | 549 | 49.6 | 9.9 | 10.4 | 11.9 | 12.5 |
| Newspaper 3 | 41 | 48.5 | 11.2 | 12.3 | 11.8 | 13.6 |
| Mean | 50.3 | 10.2 | 11.1 | 11.6 | 12.8 | |
| Difference easiest to hardest | 4.5 | −1.8 | −1.8 | −0.5 | −1.3 | |
| (b) Hong Kong | ||||||
| Newspaper 1 | 21 | 52.2 | 9.7 | 10.7 | 11.9 | 12.4 |
| Newspaper 2 | 3 | 43.0 | 10.5 | 10.3 | 13.2 | 12.6 |
| Newspaper 3 | 66 | 46.0 | 11.4 | 12.4 | 12.5 | 13.6 |
| Mean | 47.1 | 10.5 | 11.1 | 12.5 | 12.9 | |
| Difference easiest to hardest | 9.2 | −1.7 | −2.1 | −1.3 | −1.2 | |
| (c) Philippines | ||||||
| Newspaper 1 | 26 | 42.2 | 11.2 | 10.9 | 13.0 | 13.6 |
| Newspaper 2 | 133 | 41.0 | 11.6 | 11.8 | 12.9 | 14.0 |
| Newspaper 3 | 65 | 39.6 | 11.6 | 11.6 | 13.1 | 14.1 |
| Mean | 40.9 | 11.5 | 11.4 | 13.0 | 13.9 | |
| Difference easiest to hardest | 2.6 | −0.5 | −1.0 | −0.2 | −0.4 | |
* Flesch–Kincaid Reading Ease (FRE), Flesch–Kincaid Grade Level, (FKGL), Gunning Fog Index (GFI), Coleman–Liau Index (CLI), and Simple Measure of Gobbledygook (SMOG) Grade level. ** Number of non-duplicate articles.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
MDPI and ACS Style
Costantini, H.; Fuse, R. Health Information on COVID-19 Vaccination: Readability of Online Sources and Newspapers in Singapore, Hong Kong, and the Philippines. Journal. Media 2022, 3, 228-237. https://doi.org/10.3390/journalmedia3010017
AMA Style
Costantini H, Fuse R. Health Information on COVID-19 Vaccination: Readability of Online Sources and Newspapers in Singapore, Hong Kong, and the Philippines. Journalism and Media. 2022; 3(1):228-237. https://doi.org/10.3390/journalmedia3010017
Chicago/Turabian StyleCostantini, Hiroko, and Rie Fuse. 2022. "Health Information on COVID-19 Vaccination: Readability of Online Sources and Newspapers in Singapore, Hong Kong, and the Philippines" Journalism and Media 3, no. 1: 228-237. https://doi.org/10.3390/journalmedia3010017
APA StyleCostantini, H., & Fuse, R. (2022). Health Information on COVID-19 Vaccination: Readability of Online Sources and Newspapers in Singapore, Hong Kong, and the Philippines. Journalism and Media, 3(1), 228-237. https://doi.org/10.3390/journalmedia3010017
