Next Article in Journal
Risk Prescriptions of Strong Opioids in the Treatment of Chronic Non-Cancer Pain by Primary Care Physicians in Catalonia: Opicat Padris Project
Next Article in Special Issue
A Shot in the Arm for Vaccination Intention: The Media and the Health Belief Model in Three Chinese Societies
Previous Article in Journal
The Association between Stressful Life Events and Emotional and Behavioral Problems in Children 0–7 Years Old: The CIKEO Study
Previous Article in Special Issue
Assessing Mechanisms Underlying the Sharing of Official and Unofficial Information during a Pandemic
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
IJERPH
Volume 19
Issue 3
10.3390/ijerph19031651
ijerph-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Social Bots’ Involvement in the COVID-19 Vaccine Discussions on Twitter

by
Menghan Zhang
1,2,
Xue Qi
2,
Ze Chen
2 and
Jun Liu
3,*
1
Centre for Chinese Urbanization Studies of Soochow University & Collaborative Innovation Center for New Urbanization and Social Governance of Universities, Suzhou 215006, China
2
School of Communication, Soochow University, Suzhou 215123, China
3
Department of Communication, University of Copenhagen, DK-2300 Copenhagen, Denmark
*
Author to whom correspondence should be addressed.
Int. J. Environ. Res. Public Health 2022, 19(3), 1651; https://doi.org/10.3390/ijerph19031651
Submission received: 15 December 2021 / Revised: 18 January 2022 / Accepted: 21 January 2022 / Published: 31 January 2022
(This article belongs to the Special Issue The Role of New Technology in Promoting Public Health during Crises)
Browse Figures
Review Reports Versions Notes

Abstract

:
During the COVID-19 pandemic, social media served as an important channel for the public to obtain health information and disseminate opinions when offline communication was severely hindered. Yet the emergence of social bots influencing social media conversations about public health threats will require researchers and practitioners to develop new communication strategies considering their influence. So far, little is known as to what extent social bots have been involved in COVID-19 vaccine-related discussions and debates on social media. This work selected a period of nearly 9 months after the approval of the first COVID-19 vaccines to detect social bots and performed high-frequency word analysis for both social bot-generated and human-generated tweets, thus working out the extent to which social bots participated in the discussion on the COVID-19 vaccine on Twitter and their participation features. Then, a textual analysis was performed on the content of tweets. The findings revealed that 8.87% of the users were social bots, with 11% of tweets in the corpus. Besides, social bots remained active over three periods. High-frequency words in the discussions of social bots and human users on vaccine topics were similar within the three peaks of discourse.

1. Introduction

As a platform promoting public engagement and discussion, social media is an important tool for the public to receive information, make judgments, and form cognition [1]. Intensive global efforts towards physical distancing and isolation to curb the spread of COVID-19 may have intensified the use of social media as individuals try to remain connected while in quarantine [2]. Social media has hence become an important channel for the public to understand and discuss vaccine-related information, acting as a channel tool to reflect and shape public perception [3].
Since the first COVID-19 vaccines were developed, social media has become one of the prominent spheres for the discussion and debate on vaccines, and information on social media will influence public attitudes towards vaccination [4]. Social media has played a central role in disseminating scientific information and online discussion during the current pandemic [5,6]. Nevertheless, increasing evidence has shown that the online content, discourse, and sentiment on social media may not be limited to genuine human-run accounts. In the earlier studies on vaccines, social bots—automated accounts controlled and manipulated by computer algorithms [7]—have been accused of participating in vaccine discussions on social media [8,9], including public sentiment manipulation [10,11]. “Weaponized” social bots can even pretend to be humans, spread false information on a large scale, and harm public health [8]. The normalization of social bots in social media may affect public perceptions of vaccine efficacy and shape public health knowledge [8], leading to a complicated opinion climate within social media and affecting public vaccination [12,13,14].
So far, scholars have found that social bots have become even more active in the era of COVID-19 [15,16]. To what extent social bots are participating in the discussion of COVID-19 vaccines online? Furthermore, the characteristics of their involvement have not been empirically investigated. The work detected social bots participating in the online discussion of the COVID-19 vaccine on Twitter and used textual analysis combined with a Python-based program to specifically analyze social bot-generated and human-generated tweets, thereby finding out the engagement characteristics of social bots.

2. Literature Review

2.1. Social Media and Public Health Discourse

Social media is defined as web-based and mobile-based Internet applications allowing the creation, access, and exchange of ubiquitously accessible, user-generated content [17]. Information interaction on social media has the characteristics of real-time and anonymity of information dissemination [18], as well as the potential to reach a large audience and disseminate information rapidly [19,20]. These characteristics of social media make it an essential driving force for acquiring and disseminating information in different fields, e.g., in crisis management [21], politics [22], and dissemination of health information [5]. Analyzing data from social media offers new opportunities to analyze several aspects of health communication [12]. Kata found that at least 80% of Internet users seek health information online, and 16% of these seek online information about vaccination [23]. With the difficulty in communicating offline during the pandemic, more and more people are likely to use social media for information about vaccines and to express their views on social media platforms. Against this background, social media can provide an additional informal source of data that can be used to identify health information not reported to medical officials or health departments and to reveal viewpoints on related topics [18].

2.2. Vaccine Content on Social Media

With the spread of the epidemic, people are increasingly relying on different social media platforms to receive information and express opinions [10,24]. Starting in mid-2020, when the COVID-19 vaccine had not yet been developed, false information about the vaccine has been spread on social media platforms [25]. The U.S. Food and Drug Administration (2020) issued the Pfizer-BioNTech COVID-19 vaccine emergency-use authorization on December 11, 2020, implementing the COVID-19 vaccine as an essential measure to combat the COVID-19 epidemic. Discussions on the vaccine controversy have subsequently become one of the foci of online discussion [26].
Social media emerged as one of the main spheres for the debate and discussion of information related to COVID-19 and its vaccines [2]. Hussain et al. extracted over 300,000 social media posts related to COVID-19 vaccines, including 23,571 Facebook posts from the United Kingdom and 144,864 from the United States, along with 40,268 tweets from the United Kingdom and 98,385 from the United States from March 1 to November 22, 2020. Their study found that the public is optimistic about vaccine development, effectiveness, and trials, as well as concerns about vaccine safety and economic feasibility [27]. However, some studies show that the spread of conspiracy theories, false information, and various rumors about the COVID-19 vaccine on social media has caused an “infodemic” [28].
During the COVID19 pandemic, social media has become the main channel for disseminating official, authoritative information [29], rumor-defying information, and serious news [30]. Many government officials worldwide used Twitter as their primary communication channel to regularly share COVID-19-related policy updates and news [31].

2.3. Rise of Social Bots

In recent years, social bots have become increasingly prevalent on social media platforms. Social bots, as mentioned, are a program “that automatically produces content and interacts with humans on social media” [7]. These “automated social agents” [32] can mimic human users [33]. A study of social bots on Twitter shows that, of all English-speaking active users, 9 to 15% exhibit bot-like behaviors [15]. Shao et al. (2018) noted that a single social bot account can forward hundreds or even thousands of times in a short period, which enables social bots to be “super-spreaders” of information [34].
Scholars have detailed how social bots have been involved in politics [35,36], business [37], health [38], and other topics. In health-related topics, social bots have been accused of promoting polarized opinions, misleading information, and manipulating public sentiments [10]. For example, social bots can also influence the public’s attitudes towards the efficiency of cannabis in dealing with mental and physical problems [39]. In the COVID-19 pandemic, some pilots studies found that social bots spread and amplify false medical information and conspiracy theories [15].

2.4. Social Bots and Vaccine Discourse Online

Scholars have located social bot’s involvement in online vaccine discussions. Broniatowski et al. (2018) have identified “influencing bots” in a series of vaccine-related tweets designed to spread vaccine conflicts on Twitter. More specifically, “influencing bots” entail two types of inorganic users: accounts of “bots” generating automated content and those of “trolls” misrepresenting their identities and attempts to instigate conflicts purposefully [8]. Yuan et al. analyzed a retweet network related to the rubella vaccine on Twitter after the 2015 California Disneyland measles outbreak and confirmed the intervention of social bots. Their bot analysis discovered that 1.45% of the corpus users are identified as likely social bots, producing 4.59% of all tweets within the dataset. Moreover, bots may deepen the trend towards highly clustering anti-vaccine movements [9].
Without denying the contribution of these studies, researchers have failed to address the questions of how to distinguish social bots from human users and which social bots circulate in vaccine-related discourse. Besides, it remains empirically understudied as to whether there are social bots within the online discussion of the COVID-19 vaccine, and what are the characteristics of social bots. This work detected the existence of social bots in COVID-19 vaccine-related discussion on Twitter and characterized social bots’ involvement. The research questions are as follows.
RQ1: How many social bots are participating in the COVID-19 vaccine discourse on Twitter?
RQ2: What are the characteristics of social bots’ involvement in the question of COVID-19 vaccines?
The following section will first address the methodological issue regarding social bots’ detection, text classification, and analysis.

3. Materials and Methods

3.1. Data Collection and Cleaning

The work took the discussion related to COVID-19 vaccines on Twitter as the content of the analysis. We selected one important time point as a timestamp for data collecting: the U.S. Food and Drug Administration (FDA) officially approved the COVID-19 vaccine produced by Pfizer as the beginning of data collection on 11 December 2020, which was an important beginning for COVID-19 vaccination. The end of data collection was on 31 August 2021.
When people discuss vaccines and vaccination on Twitter, they often use the names of pharmaceutical companies to represent their vaccines. From the 22 vaccines that have been approved for use worldwide, the four vaccines with the widest coverage were selected as keywords, including Pfizer, Moderna, Johnson & Johnson, and AstraZeneca [40]. We observed the hashtags of four pharmaceutical companies and their vaccines on Twitter. Therefore, a list of hashtags related to the COVID-19 vaccines was crafted, including #Pfizer vaccine, #Moderna vaccine, #Johnson & Johnson vaccine, and #AstraZeneca vaccine, which made our search results more comprehensive (# is the representation of hashtags on Twitter). For example, the relevant tweets of #Pfizer can also be retrieved when the #Pfizer vaccine is used for retrieval.
For selection of hashtags, when #vaccine was used as the retrieved hashtag, the tweets’ content included multiple vaccines. In other words, the public attaches the hashtag #vaccine to their tweets when discussing any vaccines, such as HPV vaccines or pet vaccines. The reason for not using #covid-vaccine and #covid vaccine is that the hashtag #covid-vaccine does not exist on Twitter. Searching with the hashtag #covid-vaccine shows tweets containing the keywords “covid” and “vaccine.” Using #covid vaccine also leads to search results with discussions related to #covid rather than covid-19 vaccines due to the space between “covid” and “vaccine,” deviating from our research direction. Using the specific hashtags of four pharmaceutical companies and their vaccines is thereby more focused on the research topic of this work, and we would like to see public discussions on more specific vaccines. In other words, more general hashtags are not used, which is our limitation. This issue will be studied in the follow-up research.
Then, the official API data interface service provided by Twitter was used to collect tweets, with all the tweets in these hashtags crawled. Twitter users’ privacy is respected without any personal information collected and displayed. A total of 314,342 vaccine-related tweets from 11 December 2020, to 31 August 2021, were obtained. Table 1 reports the complete list of search terms and the total number of tweets containing each hashtag. In the dataset we obtained, the number of tweets in English accounted for 96%, and that of non-English tweets accounted for 4%. Considering that English is the main language in the Twitter platform, we excluded non-English tweets from the dataset. We filtered out the repeated tweets, hyperlinks, and emoji symbols in the tweets. After data cleaning, 225,277 valid tweets were obtained.

3.2. Social Bot Detection

Social bots are identified by Botometer formerly (Indiana University, Bloomington, IN, USA), known as BotOrNot. The program was developed by the Institute of Network Science in Indiana University and uses a specific identification algorithm integrating user behaviors, relationships, and spatiotemporal data. It is currently the most authoritative software for identifying bot accounts [7,39,41]. As a machine-learning framework, Botometer extracts more than 1000 features of users’ behaviors through user profile data, language, social network structure, temporal activity, and sentiment. Then the program produces a score indicating the likelihood that a Twitter account is a bot. Scores closer to 1 represent a higher chance of being bots, while those closer to 0 are more likely to belong to humans [10]. Based on previous studies [7,39,41], the threshold was set to 0.5 to separate bots from humans in this work. A score above 0.5 was defined as a social bot.

3.3. Textual Analysis

The contents of tweets were analyzed to further explore what social bots and human users said in discussing the COVID-19 vaccines. Firstly, these 255,277 valid tweets were divided into two parts, i.e., posted by human users and by social bots, respectively. Besides, tweets posted by social bots and human users are classified by month.
Firstly, the high-frequency words used were summarized to understand the foci of the discussion between social bots and human users. Cipin, a professional word frequency analysis software based on Python, was applied, with NLTK Library used for word segmentation. NLTK is a leading platform for building Python programs to work with human language data. It provides easy-to-use interfaces to over 50 corpora and lexical resources such as WordNet, along with a suite of text processing libraries for classification, tokenization, stemming, tagging, parsing, and semantic reasoning, wrappers for industrial-strength NLP libraries, and an active discussion forum [42]. Loading American National Corpus, we used NLTK chunk nltk.tokenize for word segmentation. Firstly, nltk.sent_tokenize(text) was used to complete the segmentation between sentences, and nltk.word_tokenize(sentence) used to complete word segmentation.
The tweets of human and social bots were input to the software to discover word frequency statistics. We excluded these hashtags (#Pfizer vaccine, #Moderna vaccine, #Johnson & Johnson vaccine, and #AstraZeneca vaccine) used to retrieve tweets. After obtaining the statistical results, the top five high-frequency words were selected. Tweets containing these high-frequency words were retained and manual text analysis performed.

4. Findings and Discussion

4.1. Social Bots Participated in Online COVID-19 Vaccines Related Discourse

When detecting a large number of accounts, we used the API calling method provided by Botometer. The Python program was used to call the API, and the detection could obtain a value range of 0–1. The higher the score, the more likely the account was a social bot. Based on the judgment criteria of [7,10], the work adopted a conservative threshold to ensure the scientific nature of the research results, i.e., the threshold was set to 0.5 to distinguish between humans and social bots. If the score was higher than 0.5, the account is considered to be a bot.
We entered 255,277 valid tweets from 102,844 Twitter users into Botometer to obtain a score. Figure 1 shows the distribution of scores returned by Botometer. Botometer detects 9122 social bots and 93,722 human users among 102,844 Twitter users, accounting for 8.87% and 91.13% of total users, respectively. Social bots post 28,081 and humans post 227,196, accounting for 11% and 89% of the total tweets, respectively. This finding proves the first question that 8.87% of users who tweeted about the COVID-19 vaccine are social bots. Tweets posted by social bots and human users are analyzed to figure out what the social bots posted and how these actors influenced online COVID-19 vaccine discussions.

4.2. Three Booms in the Discourse of the COVID-19 Vaccine on Twitter

Regarding the number distribution of tweets, the change in the number of social bots’ tweets is consistent with that of humans. Meanwhile, Figure 2 shows that the number of tweets by human users increases by more than 25% in December–January 2020, March–April 2021, and June–July 2021. In other words, they have the highest discussion heat at these three stages. So what do social bots post when human users are arguing about a COVID-19 vaccine on social media? These three time periods are divided for investigation: December 2020 to January 2021 as the first phase of discourse (Stage 1), March to April 2021 as the second phase of discourse (Stage 2), and June to July 2021 as the third phase of discourse (Stage 3). Next, what humans and social bots discuss in these three periods is specifically analyzed.

4.3. Discourse Foci of Social Bots and Human Users

The tweets released by social bots and humans in these three periods were analyzed to further explore the three periods mentioned above. Table 2 presents high-frequency words of social bots and human users in the three periods. According to the statistics of high-frequency words, the foci of the three concentrated discussions are different.
Stage 1: Discourse on the Effect of Vaccines (from December 2020 to January 2021).
Word-frequency statistics were used to obtain the high-frequency words in social bots and human tweets. Four of the five most frequent words in social bots and human tweets coincided: Dose, Effective, Trial, and Against. Their discussions focused on whether the COVID-19 vaccines were effective. As shown in Table 3, tweets posted by humans during the Stage1, human users expressed more concerns about vaccine risks in their tweets, such as concerns about serious side effects of vaccine experiments (example 1), doubts about vaccine effectiveness (example 3), and concerns about the adequacy of vaccine quantity (example 5). On the contrary, social bots conveyed positive information that a variety of COVID-19 vaccines could be gradually approved, and the number of vaccines could be guaranteed (examples 2 and 4). They explained that the vaccines were highly effective after rigorous scientific trials and vaccination could resist COVID-19 (example 6). Besides, social bots could connect efficient vaccines with various positive words, such as Freedom (example 7), Protective (example 8), Hope (example 9), and others, by emphasizing the immunization rate after vaccination. While emphasizing the effectiveness of vaccines, they could describe life after vaccination very positively.
Stage 2: Disputes on “Blood Clots” (from March to April 2021).
Unlike the previous discussion, from March to April 2021, the focus of the discussion changed to “blood clots” caused by the COVID-19 vaccination. In this period, the words most used by humans are News, Blood Clots, Pause, CDC, and Health, and the words most used by social bots are Blood Clots, Pause, CDC, News, and Biden. Table 4 shows the typical tweets posted by social bots and human users at the Stage 2.
In example 10, some users believed that other vaccines might also cause “blood clots” or other side effects, but this had not yet been confirmed. At stage 2, social bots introduced the vaccine’s efficacy as in stage 1 and joined the discussion of “blood clots”. In response to the public’s concern that “infection with COVID-19 did not kill, but people died after COVID-19 vaccination (examples 12 and 14)”, social bots used the number of people who get “blood clots” after vaccination as the support for the argument, adopting the discourse “advantages outweigh disadvantages”. For example, they emphasized that “Two in one- million women who have had the COVID-19 vaccine have had blood clots (example 11)”. Besides, they believed that the consequences of vaccine side effects were not worth mentioning compared with the severe consequences caused by the failure of the immune barrier (example 13).
Although both social bots and human users had actively participated in discussing “blood clots”, there were differences in their discourse. When using #CDC to indicate the information source of tweets, human users sent more CDC notifications about the risk of “blood clots” (example 16), while the social bots skillfully avoided this topic. They repeatedly emphasized that the COVID-19 vaccines had high protection and showed that CDC had unsealed the Johnson & Johnson vaccine under investigation, indicating that it was still safe (example 15). Interestingly, the word “Biden” also appears in a large number of social bot tweets. In example 17, tweets of social bots containing “Biden” described one topic: Biden had purchased 500 million Pfizer vaccines and would make a “historic” donation. These vaccines would be donated to 92 developing countries to alleviate the vaccine gap caused by worldwide, uneven vaccine distribution.
Stage 3: Debate on Delta (from June to July 2021).
In July 2021, the discussion on COVID-19 vaccines climbed to a peak again. At this stage, the high-frequency words from social bots are Delta, Against, FDA, Warning, and Effective, while the high-frequency words from human users are Delta, Against, FDA, Health, and Warning. Although the high-frequency words of social bots are highly consistent with human users, there are differences in their description of whether COVID-19 vaccines can effectively resist the Delta variant. Table 5 shows the typical tweets posted by social bots and human users at the Stage 3.
The public questioned the effectiveness of existing vaccines in dealing with the Delta variant (examples 18 and 20). Yet, social bots advocated that COVID-19 vaccines were effective against the Delta variant (example 19). They said that the immune response could last at least eight months after vaccination (example 21). If people wanted to get better protection under the spread of the Delta variant, they could inject reinforcing needles (example 22). Besides, social bots were more inclined to convey such a message to human users without being fully vaccinated. If the existing vaccine was only vaccinated with one shot or not fully vaccinated, the protection rate and practical utility were limited (example 23).

4.4. Three COVID-19 Vaccines Discussion Peaks Linked to Real-World Events

Vaccine discussions on social media are constantly evolving, with trends linked to real-world events [43]. Our research further confirms this view. The three critical discussion periods were closely related to actual events. The three critical discussion periods were closely related to actual events. We show the relationship with Figure 3. People mainly discussed whether the vaccine was effective and whether the vaccine cost money from December 2020 to January 2021, because the COVID-19 vaccines had just come out. On December 11 and 18, 2020, the FDA issued emergency-use authorizations for Pfizer and Moderna vaccines to prevent COVID-19. COVID-19 mRNA vaccines were officially released to the public as an effective tool to combat the epidemic. However, could vaccines resist COVID-19? How could vaccines be obtained? Were the vaccines free? The public did not know much about the COVID-19 vaccines. They alleviated their anxiety in this information environment full of uncertainty and crisis by communicating and forwarding news on Twitter.
On 16 April 2021, AstraZeneca COVID-19 vaccines were reported to have produced a rare “blood clots” event. Meanwhile, the government proposed to suspend the company’s COVID-19 vaccine production because six women vaccinated with Johnson & Johnson had rare “blood clots” and one woman died. The “blood clots” event aroused people’s concerns and fears about the COVID-19 vaccines, which were discussed fiercely on social media.
In July 2021, the Delta virus was found in at least 98 countries and regions. According to the CNN June 29 report, the number of cases of COVID-19 infection in the United States was increasing. The reported cases almost covered the whole of the United States. The Delta virus has become the world’s leading strain of COVID-19, which has increased the number of new cases and deaths worldwide. At this time, the discussion on social media also turned to the discussion of the delta mutant strain.

5. Conclusions

In recent years, the rise of social bots has attracted the attention of scholars [7,41,44]. Researchers found that these automated agents have become even more active in the era of COVID-19 [10,17]. Social bots have been found in the discussion on the COVID-19 epidemic on Twitter [10,16]. The emergence of bots influencing social media conversations about public health threats will require researchers and practitioners to develop new communication strategies that take into account the influence of bots [45].
The work collected 314,342 tweets from 21 December 2020, to explore whether social bots participated in the discussion of the COVID-19 vaccines on Twitter. Our research was launched when the FDA approved the Pfizer vaccine, the first batch of COVID-19 vaccines in the world, on 30 August 2021. Nine months of observation and analysis were used to track the changes in this topic discussion. The number of tweets on vaccine topics by humans and social bots fluctuated. Data collection over a long period enables us to find these differences and changes between human users and social bots in tweet content and expression while observing discussion on COVID-19 vaccines. Then, Botometer was used to detect social bots. Our findings reveal and thereby enrich current discussions on social bots from the following three aspects.
(1) Our findings evidenced the participation of social bots in the discussion of COVID-19 vaccines on Twitter. We detected 28,081 (11%) tweets published by 9122 (8.87%) social bots in the dataset. (2) The activity level of social bots was consistent with that of human users. The number of tweets increased significantly from December 2020 to January 2021, March to April 2021, and June to July 2021. (3) Social bots and humans remaining active in these three periods were mainly relating their discussion to actual events.
From December to January 2020, human users were still skeptical about the COVID-19 vaccines because they had just come out. However, social bots began to promote the effectiveness of the vaccines actively. From March to April 2021, the “blood clots”, caused by AstraZeneca, a side effect after vaccination, led to public panic about the COVID-19 vaccines. Although social bots began to talk about “blood clots” at this stage, their discourses were opposite to those of human users. By citing positive sources from CDC, social bots emphasize that “blood clots” were not a common problem. From June to July 2021, the Delta virus ravaged the world. Human users were worried about whether the existing COVID-19 vaccines could resist the Delta virus, but social bots thought that the vaccines were effective against the Delta virus. Generally, social bots are optimistic towards vaccines, although social bots have not always discussed the effectiveness and importance of vaccines.
To conclude, the work enriches the existing research on social bots and online vaccine discussions and helps to comprehensively understand social bots’ characteristics and current dynamics. With the spread of the COVID-19 epidemic, vaccination against COVID-19 has become an important means to control the epidemic. The normalization of social bots in social media may affect public perceptions of vaccine effectiveness and shape public health knowledge, complicating the climate of opinion on social media. Researchers and practitioners need to understand the engagement of social bots in social media to develop new communication strategies. We hope that, through our study, researchers and policymakers can understand how social bots are participating in COVID-19 vaccine discussions.

6. Limitations and Future Directions

The work detected the participation of social bots in the online discussion of the COVID-19 vaccine on Twitter. Textual analysis was used to specifically analyze bot-generated and human-generated tweets, thus finding out the engagement characteristics of social bots. However, while bot detection algorithms can automatically label suspicious social bots, the Twitter platform’s current mechanism does not reveal the manipulators behind social bots. Therefore, although characterizing the content posted by social bots and their participation in discussions and proposing several possible explanations, the work cannot infer the intentions of social bots based on their expressions. Previous researchers have suggested that social bots can also influence people’s emotions about vaccines [9,10]. Therefore, our subsequent research will focus on the emotional expression of social bots in the discussion of the COVID-19 vaccine and whether they affect the public’s sentiments towards the COVID-19 vaccine.

Author Contributions

Conceptualization, M.Z. and J.L.; methodology, J.L., X.Q., and Z.C.; software, X.Q.; validation, M.Z. and J.L.; data curation, X.Q. and Z.C.; writing—original draft preparation, X.Q. and Z.C.; writing—review and editing, M.Z. and J.L.; visualization, X.Q.; supervision, J.L. and M.Z.; project administration, M.Z.; funding acquisition, M.Z. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by National Social Science Foundation Youth Project “Research on the Influence of Social bots on the Climate of False Opinions in International Communication” and Major projects of Jiangsu Provincial Department of Education “Research on the Influence of Social Bots on the Climate of False Opinions in International Communication and its Regulation”, grant number 21CXW028 and 2021SJZDA151. This research is the phased achievements of Collaborative Innovation Center for New Urbanization and Social Governance of Soochow University.

Institutional Review Board Statement

The study was conducted in accordance with the China’s Personal Information Protection Law, and approved by the National Social Science Fund of China (21CXW028). Ethical review and approval were waived for this retrospective study, while the personal data protection statement is available on the project’s website (https://rtgksm.wordpress.com/, in Chinese, accessed on 14 December 2021).

Informed Consent Statement

Informed consent was waived for this retrospective study, while the personal data protection statement is available on the project’s website (https://rtgksm.wordpress.com/, in Chinese, accessed on 14 December 2021).

Data Availability Statement

The data presented in this study are available on request from the first author. The data are not publicly available due to privacy protection.

Acknowledgments

This research was funded by National Social Science Foundation Youth Project “Research on the Influence of Social bots on the Climate of False Opinions in International Communication” and Major projects of Jiangsu Provincial Department of Education “Research on the Influence of Social Bots on the Climate of False Opinions in International Communication and its Regulation,” grant numbers 21CXW028 and 2021SJZDA151. This research is the phased achievements of Collaborative Innovation Center for New Urbanization and Social Governance of Soochow University.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Linvill, D.L.; Warren, P.L. Troll factories: Manufacturing specialized disinformation on Twitter. Political Commun. 2020, 37, 447–467. [Google Scholar] [CrossRef]
  2. Puri, N.; Coomes, E.A.; Haghbayan, H.; Gunaratne, K. Social media and vaccine hesitancy: New updates for the era of COVID-19 and globalized infectious diseases. Hum. Vaccines Immunother. 2020, 16, 2586–2593. [Google Scholar] [CrossRef] [PubMed]
  3. Dubé, E.; Vivion, M.; MacDonald, N.E. Vaccine hesitancy, vaccine refusal and the anti-vaccine movement: Influence, impact and implications. Expert Rev. Vaccines 2015, 14, 99–117. [Google Scholar] [CrossRef] [PubMed]
  4. Benis, A.; Sei Dm Ann, A.; Ashkenazi, S. Reasons for taking the COVID-19 vaccines by us social media users. Vaccines 2021, 9, 315. [Google Scholar] [CrossRef] [PubMed]
  5. Cuello-Garcia, C.; Pérez-Gaxiola, G.; van Amelsvoort, L. Social media can have an impact on how we manage and investigate the COVID-19 pandemic. J. Clin. Epidemiol. 2020, 127, 198–201. [Google Scholar] [CrossRef] [PubMed]
  6. Zimba, O.; Radchenko, O.; Strilchuk, L. Social media for research, education and practice in rheumatology. Rheumatol. Int. 2020, 40, 183–190. [Google Scholar] [CrossRef] [PubMed]
  7. Ferrara, E.; Varol, O.; Davis, C.; Menczer, F.; Flammini, A. The rise of social bots. Commun. ACM 2016, 59, 96–104. [Google Scholar] [CrossRef] [Green Version]
  8. Broniatowski, D.A.; Jamison, A.M.; Qi, S.H.; AlKulaib, L.; Chen, T.; Benton, A.; Quin, S.C.; Dredze, M. Weaponized health communication: Twitter bots and Russian trolls amplify the vaccine debate. Am. J. Public Health 2018, 108, 1378–1384. [Google Scholar] [CrossRef]
  9. Yuan, X.; Schuchard, R.J.; Crooks, A.T. Examining emergent communities and social bots within the polarized online vaccination debate in Twitter. Soc. Media Soc. 2019, 5, 1–12. [Google Scholar] [CrossRef] [Green Version]
  10. Shi, W.; Liu, D.; Yang, J.; Zhang, J.; Su, J. Social bots’ sentiment engagement in health emergencies: A topic-based analysis of the COVID-19 pandemic discussions on Twitter. Int. J. Environ. Res. Public Health 2020, 17, 8701. [Google Scholar] [CrossRef]
  11. Varol, O.; Ferrara, E.; Davis, C.; Menczer, F.; Flammini, A. Online human-bot interactions: Detection, estimation, and characterization. In Proceedings of the International AAAI Conference on Web and Social Media, Montreal, QC, Canada, 15–18 May 2017. [Google Scholar]
  12. Stieglitz, S.; Mirbabaie, M.; Ross, B.; Neuberger, C. Social media analytics—Challenges in topic discovery, data collection, and data preparation. Int. J. Inf. Manag. 2018, 39, 156–168. [Google Scholar] [CrossRef]
  13. Jolley, D.; Douglas, K.M. The effects of anti-vaccine conspiracy theories on vaccination intentions. PLoS ONE 2014, 9, e89177. [Google Scholar] [CrossRef] [PubMed]
  14. Dixon, G.; Clarke, C. The effect of falsely balanced reporting of the autism–vaccine controversy on vaccine safety perceptions and behavioral intentions. Health Educ. Res. 2013, 28, 352–359. [Google Scholar] [CrossRef] [Green Version]
  15. Kim, A. Nearly Half of the Twitter Accounts Discussing ‘Reopening America’ May Be Bots, Researchers Say. CNN. 22 May 2020. Available online: https://edition.cnn.com/2020/05/22/tech/twitter-bots-trnd/index.html (accessed on 10 September 2020).
  16. Gallotti, R.; Valle, F.; Castaldo, N.; Sacco, P.; De Domenico, M. Assessing the risks of ‘infodemics’ in response to COVID-19 epidemics. Nat. Hum. Behav. 2020, 4, 1285–1293. [Google Scholar] [CrossRef] [PubMed]
  17. Kaplan, A.M.; Haenlein, M. Users of the world, unite! The challenges and opportunities of social media. Bus. Horiz. 2010, 53, 59–68. [Google Scholar] [CrossRef]
  18. Charles-Smith, L.E.; Reynolds, T.L.; Cameron, M.A.; Conway, M.; Lau, E.H.Y.; Olsen, J.M.; Pavlin, J.A.; Shigematsu, M.; Streichert, L.C.; Suda, K.J.; et al. Using social media for actionable disease surveillance and outbreak management: A systematic literature review. PLoS ONE 2015, 10, e0139701. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  19. Betsch, C.; Brewer, N.T.; Brocard, P.; Davies, P.; Gaissmaier, W.; Haase, N.; Leask, J.; Renkewitz, F.; Renner, B.; Reyna, V.F. Opportunities and challenges of web 2.0 for vaccination decisions. Vaccine 2012, 30, 3727–3733. [Google Scholar] [CrossRef] [Green Version]
  20. Blankenship, E.B.; Goff, M.E.; Yin, J.; Tse, Z.T.H.; Fu, K.-W.; Liang, H.; Saroha, N.; Fung, I.C.H. Sentiment, contents, and retweets: A study of two vaccine-related Twitter datasets. Perm. J. 2018, 22, 17–138. [Google Scholar] [CrossRef]
  21. Stieglitz, S.; Bunker, D.; Mirbabaie, M.; Ehnis, C. Sense—Making in social media during extreme events. J. Contingencies Crisis Manag. 2018, 26, 4–15. [Google Scholar] [CrossRef] [Green Version]
  22. Stieglitz, S.; Dang-Xuan, L. Social media and political communication: A social media analytics framework. Soc. Netw. Anal. Min. 2013, 3, 1277–1291. [Google Scholar] [CrossRef]
  23. Kata, A. Anti-vaccine activists, Web 2.0, and the postmodern paradigm–An overview of tactics and tropes used online by the anti-vaccination movement. Vaccine 2012, 30, 3778–3789. [Google Scholar] [CrossRef] [PubMed]
  24. Xue, J.; Chen, J.; Hu, R.; Chen, C.; Zheng, C.; Su, Y.; Zhu, T. Twitter discussions and emotions about the COVID-19 pandemic: Machine learning approach. J. Med. Internet Res. 2020, 22, e20550. [Google Scholar] [CrossRef] [PubMed]
  25. The Anti-Vaxx Industry: How Big Tech Powers and Profits from Anti-Vaccine Misinformation.Centre for Countering Digital Hate. Available online: https://www.counterhate.com/anti-vaxx-industry (accessed on 26 October 2021).
  26. Luo, C.; Chen, A.; Cui, B.; Liao, W. Exploring public perceptions of the COVID-19 vaccine online from a cultural perspective: Semantic network analysis of two social media platforms in the United States and China. Telemat. Inform. 2021, 65, 101712. [Google Scholar] [CrossRef] [PubMed]
  27. Hussain, A.; Tahir, A.; Hussain, Z.; Sheikh, Z.; Gogate, M.; Dashtipour, K.; Ali, A.; Sheikh, A. Artificial intelligence-enabled analysis of public attitudes on Facebook and Twitter toward COVID-19 vaccines in the United Kingdom and the United States: Observational study. J. Med. Internet Res. 2021, 23, e26627. [Google Scholar] [CrossRef] [PubMed]
  28. Jamison, A.M.; Broniatowski, D.A.; Dredze, M.; Sangraula, A.; Smith, M.C.; Quinn, S.C. Not just conspiracy theories: Vaccine opponents and proponents add to the COVID-19; ‘Infodemic’on Twitter. Harv. Kennedy Sch. Misinformation Rev. 2020, 19, 1–19. [Google Scholar] [CrossRef] [PubMed]
  29. La, V.P.; Pham, T.H.; Ho, M.T.; Nguyen, M.H.; PNguyen, K.L.; Vuong, T.T.; Nguyen, H.-K.T.; Tran, T.; Khuc, Q.; Ho, M.-T.; et al. Policy response, social media and science journalism for the sustainability of the public health system amid the COVID-19 outbreak: The Vietnam lessons. Sustainability 2020, 12, 2931. [Google Scholar] [CrossRef] [Green Version]
  30. Bridgman, A.; Merkley, E.; Loewen, P.J.; Owen, T.; Ruths, D.; Teichmann, L.; Zhilin, O. The causes and consequences of COVID-19 misperceptions: Understanding the role of news and social media. Harv. Kennedy Sch. Misinformation Rev. 2020, 21, 1–21. [Google Scholar] [CrossRef]
  31. Rufai, S.R.; Bunce, C. World leaders usage of Twitter in response to the COVID-19 pandemic: A content analysis. J. Public Health 2020, 42, 510–516. [Google Scholar] [CrossRef]
  32. Abokhodair, N.; Yoo, D.; McDonald, D.W. Dissecting a social botnet: Growth, content, and influence in Twitter. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing, Vancouver, BC, Canada, 14–18 March 2015. [Google Scholar]
  33. Stieglitz, S.; Brachten, F.; Ross, B.; Jung, A.K. Do social bots dream of electric sheep? A categorization of social media bot accounts. In Proceedings of the Australasian Conference on Information Systems, Hobart, Australia, 4–6 December 2017. [Google Scholar]
  34. Shao, C.; Ciampaglia, G.L.; Varol, O.; Yang, K.C.; Flammini, A.; Menczer, F. The spread of low-credibility content by social bots. Nat. Commun. 2018, 9, 1–9. [Google Scholar] [CrossRef] [Green Version]
  35. Woolley, S.C.; Howard, P.N. Automation, algorithms, and politics | political communication, computational propaganda, and autonomous agents—Introduction. Int. J. Commun. 2016, 10, 4882–4890. [Google Scholar]
  36. Bastos, M.T.; Mercea, D. The Brexit botnet and user-generated hyperpartisan news. Soc. Sci. Comput. Rev. 2019, 37, 38–54. [Google Scholar] [CrossRef] [Green Version]
  37. Fan, R.; Talavera, O.; Tran, V. Social media bots and stock markets. Eur. Financ. Manag. 2020, 26, 753–777. [Google Scholar] [CrossRef]
  38. Allem, J.P.; Escobedo, P.; Dharmapuri, L. Cannabis surveillance with Twitter data: Emerging topics and social bots. Am. J. Public Health 2020, 110, 357–362. [Google Scholar] [CrossRef] [PubMed]
  39. Badawy, A.; Lerman, K.; Ferrara, E. Who falls for online political manipulation? In Proceedings of the Companion Proceedings of The 2019 World Wide Web Conference, Association for Computing Machinery, San Francisco, CA, USA, 13–17 May 2019. [Google Scholar]
  40. Statista. The First Seven COVID-19 Vaccines to Have Been Approved Internationally Received Orders for over 15 Billion Doses Altogether. Statista. 15 September 2021. Available online: https://www-statista-com.ezproxy.gavilan.edu/study/86257/covid19-vaccination-status-quo/ (accessed on 13 October 2021).
  41. Luceri, L.; Deb, A.; Giordano, S.; Ferrara, E. Evolution of bot and human behavior during elections. First Monday 2019, 24, 1–12. [Google Scholar] [CrossRef]
  42. Natural Language Toolkit. Team NLTK. Available online: https://www.nltk.org/ (accessed on 11 October 2021).
  43. Bessi, A.; Ferrara, E. Social bots distort the 2016 US presidential election online discussion. First Monday 2016, 21, 1–14. [Google Scholar] [CrossRef]
  44. Gunaratne, K.; Coomes, E.A.; Haghbayan, H. Temporal trends in anti-vaccine discourse on Twitter. Vaccine 2019, 37, 67–71. [Google Scholar] [CrossRef]
  45. Sutton, J. Health communication trolls and bots versus public health agencies’ trusted voices. Am. J. Public Health 2018, 108, 1281–1282. [Google Scholar] [CrossRef] [PubMed]
Ijerph 19 01651 g001 550
Figure 1. Score Distribution Returned by Botometer.
Figure 1. Score Distribution Returned by Botometer.
Ijerph 19 01651 g001
Ijerph 19 01651 g002 550
Figure 2. Number of monthly tweets of bots and Humans from December 2020 to August 2021.
Figure 2. Number of monthly tweets of bots and Humans from December 2020 to August 2021.
Ijerph 19 01651 g002
Ijerph 19 01651 g003 550
Figure 3. Trends of Tweets Published by Social Bots and Human Users Related to Real-World from December 2020 to August 2021.
Figure 3. Trends of Tweets Published by Social Bots and Human Users Related to Real-World from December 2020 to August 2021.
Ijerph 19 01651 g003
Table
Table 1. List of Hashtags Related to the COVID-19 Vaccination.
Table 1. List of Hashtags Related to the COVID-19 Vaccination.
HashtagsAll TweetsValid Tweets
#Pfizer vaccines63,64145,609
#Moderna vaccines54,10938,777
#Johnson & Johnson vaccines139,823100,205
#AstraZeneca vaccines56,76940,686
Summation314,342225,277
Table
Table 2. High-Frequency Words of Social Bots and Human Users in Three Periods.
Table 2. High-Frequency Words of Social Bots and Human Users in Three Periods.
2020.12–2021.1
DoseEffectiveAgainstTrialEUNews
Human users10,3988841598564776353-
Social bots9791282714516-821
2021.3–2021.4
NewsBlood clotPauseCDCHealthBiden
Human users82757224692238887422-
Social bots60817261535917-710
2021.6–2021.7
DeltaAgainstFDAWarningHealthEffective
Human users80217710721345316049-
Social bots1957186915901416-1018
Table
Table 3. Typical Tweets—Examples of Social Bots and Human Users at the Stage 1.
Table 3. Typical Tweets—Examples of Social Bots and Human Users at the Stage 1.
StageHumansBots
Stage 1Example 1: #COVID19 vaccine side effects: What it is like in #Pfizer, #Moderna trials The FDA says Pfizer’s COVID vaccine is safe and effective. However, trial participants warn of intense…Example 2: #AstraZeneca to produce 90 million #COVID19 vaccine shots in Japan East Asia News @Straits_Times AstraZeneca to produce 90 million COVID-19 vaccine shots in Japan, East Asia News -Announcement comes as concerns mount over whether the country will have enough to start its delayed inoculation drive. Read more at straitstimes.com
Example 3: I heard that COVID 19 $AZN #AstraZeneca/#Oxford #COVID19 AstraZeneca vaccine is reportedly only 8% effective in populations older than 65…Example 4: #Pfizer BioNTech #Astrazeneca#Pfizer”“One step closer to freedom. You can help stop the pandemic by getting a COVID-19 vaccine when it is available to you. Join the global effort! Together against the COVID-19!
Example 5: #AstraZeneca offers eight million more doses of its #coronavirus vaccine to the #EU to try to defuse a row over supplies, but the bloc says that was too far short of what was originally promised, according to an EU official. Example 6: Did you know all Phase 1 trials test several dose levels of vaccines? The dose levels carried forward to Phase 2; 3 in the #Pfizer and #Moderna trials are the best balance of safety and efficacy. Sign up for no cost COVID-19 updates: https://bit.ly/3rk9QP9 (accessed on 14 December 2021)
-Example 7: #PfizerBioNTech #astrazeneca#Pfizer”“One step closer to freedom. You can help stop the pandemic by getting a covid-19 vaccine when it is available to you.
-Example 8: #Moderna Vaccine Is Highly Protective Against COVID-19, the F.D.A. Finds #Moderna Says Vaccine Still Protects Against Virus Variants #Johnson; Johnson’s Vaccine Offers Strong Protection but Fuels Concern About Variants
-Example 9: Replying to @KulganofCrydee and @IainDale The first of many great Brexit success stories. Let’s hope the vaccines start to speed up across Europe so we can all have our freedoms returned. #Brexit #AstraZeneca #CovidVaccine
Table
Table 4. Typical Tweets—Examples of Social Bots and Human Users at Stage 2.
Table 4. Typical Tweets—Examples of Social Bots and Human Users at Stage 2.
StageHumansBots
Stage 2Example10: #Janssen vaccine also has ‘possible link’ to blood clots: https://onenewspage.com/video/20210420/13805259/ (accessed on 14 December 2021) Janssen-vaccine Has possible link to blood clots.htm (accessed on 14 December 2021).Example 11: 2 in 1 million women who have had the COVID-19 vaccine have had blood clots, 1 in 5000 women taking the pill have blood clots, 1 in 6 people with COVID gets blood clots.
Example 12: #Brazil#AstraZeneca A pregnant woman survived after contracting the virus, and died after receiving the#AstraZeneca vaccine. #bloodclotExample 13: The Johnson& Johnson shot has been given to some 7.2 million Americans, with six cases of women developing brain clots along with low blood platelet counts, including one death, within two weeks of getting the one-dose vaccine.#MNow #Vaccine #USUS call on resuming J&J vaccine expected this week#Science #science How The Johnson & Johnson Vaccine Pause Might Slow Down The Global COVID-19 Response? How The Johnson & Johnson Vaccine Pause Might Slow Down The Global COVID-19 Response?
Example14: Survived after contracting the COVID-19 virus, and died after receiving the #AstraZeneca vaccine. Example15: #Breaking: #US ends pause on #Johnson; Johnson #Covid vaccine after concern about clots#Breaking: #US ends pause on #Johnson & Johnson #Covid vaccine after concerning about clots (http://Forbes.com) (accessed on 14 December 2021):#Fauci #Denies Johnson; Johnson Pause Will Contribute To Vaccine Hesitancy: “In the long run … people will realize that we take safety very seriously,” Fauci said… #TrendsSpy http://newsoneplace.com/14648812012/fauci-denies-johnson-pause-will-contribute-vaccine (accessed on 14 December 2021)…
Example 16: CDC panel reviewing Johnson & Johnson vaccine pause Friday. The CDC is investigating 2 new reports of clotting tied to the Johnson & Johnson COVID-19 vaccine.-Example17: US President Joe Biden is expected to announce a “historic” US donation of half a billion COVID-19 vaccine doses for 92 poorer countries, the White House said #News #Pfizer #Vaccine #Coronavirus #Worldh24.news
Table
Table 5. Typical Tweets—Examples of Social Bots and Human Users at Stage 3.
Table 5. Typical Tweets—Examples of Social Bots and Human Users at Stage 3.
StageHumansBots
Stage 3Example 18: A study from a team of New York University researchers found the one-shot Johnson & Johnson vaccine is far less effective at preventing coronavirus infections from the Delta variant and other mutated…Example19:the Johnson & Johnson Coronavirus vaccine provides effective protection against the Delta variant, which is offering hope to many developing economies facing a summer surge of the highly contagious…
Example 20: Israel study suggests #Pfizer/BioNTech #vaccine is less effective against #Delta #variant. Data suggests the vaccine is 64% effective at preventing infection caused by the Delta variant after two doses. Israel study suggests Pfizer/BioNTech vaccine is less effective against Delta variant.Example 21: Johnson & Johnson asserts that its single-dose Janssen COVID-19 #vaccine is effective against the #Delta variant, which was first identified in India and is particularly contagious, with an immune response that can last at least eight months. (AFP)
-Example 22: #MOG Pfizer’s COVID-19 vaccine works very well against the Delta variant—but only after 2 doses. The study on 19,000 people in the UK found one Pfizer shot was about 36% effective against Delta, but it was highly effective after two shots. businessinsider.com
-Example 23: #Pfizer’s COVID-19 vaccine works very well against the Delta variant—but only after 2 doses. The study on 19,000 people in the UK found one Pfizer shot was about 36% effective against Delta, but it was highly effective after two shots.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Zhang, M.; Qi, X.; Chen, Z.; Liu, J. Social Bots’ Involvement in the COVID-19 Vaccine Discussions on Twitter. Int. J. Environ. Res. Public Health 2022, 19, 1651. https://doi.org/10.3390/ijerph19031651

AMA Style

Zhang M, Qi X, Chen Z, Liu J. Social Bots’ Involvement in the COVID-19 Vaccine Discussions on Twitter. International Journal of Environmental Research and Public Health. 2022; 19(3):1651. https://doi.org/10.3390/ijerph19031651

Chicago/Turabian Style

Zhang, Menghan, Xue Qi, Ze Chen, and Jun Liu. 2022. "Social Bots’ Involvement in the COVID-19 Vaccine Discussions on Twitter" International Journal of Environmental Research and Public Health 19, no. 3: 1651. https://doi.org/10.3390/ijerph19031651

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop