*4.2. Word Identification and Word Frequency*

The results confirm that negativity towards the COVID-19 vaccines is present on Twitter alongside tweets that are positive and neutral in sentiment. Similar studies corroborate these results [10,49,76], with suggestions that development speed and safety concerns are some of the reasons why hesitancy is expressed [77]. Chandrasekaran et al. examined the trends of sentiment of several topics associated with COVID-19 between January 2020 and May 2020 and found that although Twitter users expressed negativity about the spread and symptoms of COVID-19, they determined that positive feelings were expressed when sharing information on drugs and new therapies [55]. In the present study, the commonly used term 'people' suggests that concerns do not specifically relate to children, elderly or any other specific group. Although the hashtag '#covid19 was the most frequently occurring word in all three sentiment groups, analysis found that a higher number of negative tweets contained the hashtag (31,725) in comparison to positive (29,661) and neutral (14,399) tweets.

A study on the sentiment surrounding human papillomavirus vaccines found different keywords associated within their word clusters. The authors suggested that 'HPV' was associated with personal words including 'I' and 'me' and '#HPV' was associated with words such as 'learn' and 'prevent'. The authors considered these 'awareness-raising words' [78]. Our findings show similar results; 'people', 'don't', 'health', 'vaccines', and 'death' were noticeable in the negative groups. This could also be indicative of concerns about the risks of accepting the vaccine [79]. Words including 'people', 'please', 'help' 'vaccine' 'first' and 'need' were found to be frequently occurring in the positive group. These terms suggest that discourse leans towards promotion and encouragement of vaccinating, with similar key words found in previous studies [79]. The only similarities of the word frequencies performed by Sattar et al. (2021) and this study were 'death' and 'people' in the negative category, 'vaccine' in the positive category and 'help' and 'first' in both the positive and neutral categories. They also identified words that were not found in our study including 'party,' 'happy' and 'thank' [28].

Previous research suggests that social media users tend to interact with others who share common beliefs and ignore or argue with individuals who have opposite views [80,81], creating an echo chamber. Due to this, it has been suggested that public health interventions could reinforce vaccine hesitancy [81–83] and identifying keywords or hashtags that hesitant individuals commonly use would be a more effective strategy [84] to countering the problem. This study has identified several keywords and hashtags to assist in this process.

#### *4.3. Relative Frequency of Tweets*

We observed the frequency and relative frequency of tweets in each week of this study. Despite most of the tweets in the dataset being negative, positive tweets (14,305; 39.0%) were the most predominant during the first week of data collection between 1 July 2021 and 7 July 2021 whereas, in the final two weeks, between 8 July 2021 and 21 July 2021, negative tweets (19,691; 39.0% and 20,308; 40%) were most common. Neutral tweets were significantly lower than both negative and positive tweets throughout the entire time of collection (22.9%, 22.5% and 21.7%). Piedrahita-Valdes et al. (2021) performed sentiment analysis on vaccine-hesitant tweets between June 2011 and April 2019 and found neutral tweets were predominant throughout the study, in contrast to the present study. They also found that negative tweets peaked at times and noted that at least one of these peaks coincided with a documentary linking autism to vaccines. Similarly, they identified positiverelated peaks occurring in April which coincided with World Immunisation week [25]. Furthermore, a noticeable increase in anti-vaccine discourse was experienced on Twitter in 2015, coinciding with a measles outbreak (2014–2015), a newly released film "Vaxxed" and the publication of the book "Vaccine Whistleblower" [17], supporting the idea that conversations relating to vaccine hesitancy fluctuate over time.

The mean of neutral tweets displayed a negative sentiment compound (−0.00000132) during week 2 of the investigation, whereas, in weeks 1 and 3, neutral tweets were positive

(0.000199 and 0.000177, respectively). This is suggestive of concurrent events that the general public are exposed to [17] such as case numbers, the reporting of daily hospitalisation and death figures, the pace of the UK vaccination programme and the expansion of testing capability in addition to wider political factors including legislated social distancing, lockdowns, working from home mandates and face mask wearing. For example, on 5 July 2021 plans to remove the mandated wearing of facemasks from 19 July 2021 were announced in England. This announcement could have been a key factor in the high positive sentiment we detected in this study in week 1. By 7 July 2021, however, the UK's weekly COVID-19 cases had doubled in comparison to the week prior; and between 8 and 14 July (corresponding to week 2 in this study), cases continued to rise in the UK, with over 50,000 new cases reported on 17 July 2021 [85]. As these events unfolded, 1200 scientists formally challenged the easing of lockdown restrictions in England [86], a discussion that is likely to have added to the negative sentiment at the time. Public opinion remained polarised and by week 3 of our study, we found the highest frequency of tweets which reflected negative sentiment at the same time as the number of tweets that were positive in sentiment increased from week 2 (38.4%) to week 3 (47.6%). Whilst previous research has identified vaccine hesitancy fluctuating over time [17], it would be interesting to compare the dates of specific announcements and wider discussions with daily sentiment analysis to determine whether there is a relationship between the two.
