COVID-19 Vaccine Hesitancy: A Global Public Health and Risk Modelling Framework Using an Environmental Deep Neural Network, Sentiment Classification with Text Mining and Emotional Reactions from COVID-19 Vaccination Tweets
Abstract
:1. Background
2. Literature Review
2.1. Previous Studies
2.2. Existing Gap in Literature
- I.
- Some previous research projects were limited to only sentiment analysis. We believe that extracting emotions, in addition to its sentiments, will enhance the study.
- II.
- Some previous works were based on datasets downloaded from other institutions. We believe there is a need for a shift in the dataset used to analyze the study with updated records. Consequently, addressing updated COVID-19 vaccination hesitancy tweets is another crucial matter to be considered.
- III.
- Building complex emotions could enhance the discussions on COVID-19 vaccine hesitancy. Most previous studies were limited in scope on classifications. We believe that integrating the topic with another field, such as psychology, could enhance the research study.
3. Methodology
3.1. Overview
3.2. Data Collection
3.3. Text Preprocessing
3.4. NRCLexicon
3.4.1. Daily Emotions and Sentiments
3.4.2. Complex Emotional Effects
3.5. Neural Network Models
3.5.1. 1DCNN
3.5.2. LSTM (Long Short-Term Memory)
3.5.3. Multiple-Layer Perceptron
3.5.4. BERT (Bidirectional Encoder Representations from Transformers)
4. Results
5. Discussions
6. Contributions
- We designed, developed, and evaluated deep neural network models with ten classes (positive, negative, joy, sadness, trust, disgust, fear, anger, surprise, and anticipation). Most previous studies were limited to three classes of sentiment (positive, negative, and neutral). For instance, [38] analyzed a Twitter dataset based on VADER sentiment computation with three classes (positive, negative, and neutral) using LSTM and BiLSTM neural networks. Ref. [54] performed sentiment analysis using three sentiment classes (positive, negative, and neutral) by applying a deep neural network model. Ref. [52] applied ANN, LSTM, and BERT neural networks using four classes (pain, fever, fatigue, and headache). Ref. [44] experimented with BERT, LinearSVM, Logistic Regression, and LSTM using four classes (sad, joy, fear, and anger). Ref. [39] classified their research’s sentiments into five classes (positive, extremely positive, negative, extremely negative, and neutral).
- The proposed deep neural network models demonstrated an improvement over previous research studies. For example, [41] performed modeling, such as Random Forest, XGBoost, LinearSVC, Decision Tree, LSTM, BiLSTM, and CNN-LSTM, with three classes (positive, negative, and neutral) using TextBlob sentiment computation. The best accuracy of their work was 93%. Ref. [42] performed LinearSVC, Multinomial Naïve Bayes, BiLSTM, and BERT models with three classes of sentiments using TextBlob computation. Their best performance was the BERT neural network with an accuracy of 91.6%. Furthermore, [45] used Decision Tree, GaussianN, Random Forest, KNN, Fusion Model, Simple Neural Network, CNN, LSTM, and BERT. The best performance was achieved by the BERT neural network with an accuracy of 92.35%. Our research study performed better (accuracy of 96.71%) when using the BERT neural network model.
- Our research study identified eight basic emotions from the tweets (joy, sadness, trust, disgust, fear, anger, surprise, and anticipation) in addition to the two sentiments (positive and negative). Furthermore, we extended the basic emotions into complex ones (aggressiveness, contempt, remorse, disapproval, awe, submission, love, and optimism). Most of the previous classes were based on the sentiments only (positive, negative, and neutral) [45,51,54,55]. Another study classified their dataset into four basic emotions (sad, joy, fear, and anger) [44]. Finally, a study used a combination of primary and complex emotions (optimistic, thankful, emphatic, pessimistic, anxious, sad, annoyed, denial, official, report, surprise, and joking) [40]. Our research is based on a ten-class solution using deep learning models of LSTM, BiLSTM, and BERT to classify COVID-19 vaccine hesitancy tweets into positive, negative, joy, sadness, trust, disgust, fear, anger, surprise, and anticipation.
7. Conclusions
8. Future Work
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Kumar, A.; Singh, R.; Kaur, J.; Pandey, S.; Sharma, V.; Thakur, L.; Sati, S.; Mani, S.; Asthana, S.; Sharma, T.K.; et al. Wuhan to World: The COVID-19 Pandemic. Front. Cell. Infect. Microbiol. 2021, 11, 596201. [Google Scholar] [CrossRef] [PubMed]
- Mohan, B.S.; Nambiar, V. COVID-19: An Insight into SARS-CoV-2 Pandemic Originated at Wuhan City in Hubei Province of China. J. Infect. Dis. Epidemiol. 2020, 6, 146. [Google Scholar] [CrossRef]
- Li, H.; Liu, S.-M.; Yu, X.-H.; Tang, S.-L.; Tang, C.-K. Coronavirus disease 2019 (COVID-19): Current status and future perspectives. Int. J. Antimicrob. Agents 2020, 55, 105951. [Google Scholar] [CrossRef] [PubMed]
- The World Bank. Urgent, Effective Action Required to Quell the Impact of COVID-19 on Education Worldwide; The World Bank: Washington, DC, USA, 2021. [Google Scholar]
- Du, Z.; Wang, L.; Cauchemez, S.; Xu, X.; Wang, X.; Cowling, B.J.; Meyers, L.A. Risk for Transportation of Coronavirus Disease from Wuhan to Other Cities in China. Emerg. Infect. Dis. 2020, 26, 1049–1052. [Google Scholar] [CrossRef]
- Qorib, M.; Oladunni, T.; Denis, M.; Ososanya, E.; Cotae, P. COVID-19 vaccine hesitancy: Text mining, sentiment analysis and machine learning on COVID-19 vaccination Twitter dataset. Expert Syst. Appl. 2023, 212, 118715. [Google Scholar] [CrossRef]
- UNESCO. Education: From Disruption to Recovery; UNESCO: Paris, France, 2022. [Google Scholar]
- Hoofman, J.; Secord, E. The Effect of COVID-19 on Education. Pediatr. Clin. N. Am. 2021, 68, 1071–1079. [Google Scholar] [CrossRef]
- Liang, L.; Ren, H.; Cao, R.; Hu, Y.; Qin, Z.; Li, C.; Mei, S. The Effect of COVID-19 on Youth Mental Health. Psychiatr. Q. 2020, 91, 841–852. [Google Scholar] [CrossRef]
- Soest, T.V.; Bakken, A.; Pedersen, W.; Sletten, M.A. Life satisfaction among adolescents before and during the COVID-19 pandemic. Tidsskr. Nor. Laegeforen. 2020, 140, 10. [Google Scholar] [CrossRef]
- U.S. Department of Education. Education in a Pandemic: The Disparate Impacts of COVID-19 on America’s Students; U.S. Department of Education: Washington, DC, USA, 2021.
- Wolfson, J.A.; Leung, C.W. Food Insecurity and COVID-19: Disparities in Early Effects for US Adults. Nutrients 2020, 12, 1648. [Google Scholar] [CrossRef]
- Fegert, J.M.; Vitiello, B.; Plener, P.L.; Clemens, V. Challenges and burden of the Coronavirus 2019 (COVID-19) pandemic for child and adolescent mental health: A narrative review to highlight clinical and research needs in the acute phase and the long return to normality. Child Adolesc. Psychiatry Ment. Health 2020, 14, 20. [Google Scholar] [CrossRef]
- Pfizer, Real-World Evidence Confirms High Effectiveness of Pfizer-BioNTech COVID-19 Vaccine and Profound Public Health Impact of Vaccination One Year after Pandemic Declared. 2021. Available online: https://www.pfizer.com/news/press-release/press-release-detail/real-world-evidence-confirms-high-effectiveness-pfizer (accessed on 1 April 2023).
- Pedrosa, A.L.; Bitencourt, L.; Fróes, A.C.F.; Cazumbá, M.L.B.; Campos, R.G.B.; de Brito, S.B.C.S.; Simões, E.; Silva, A.C. Emotional, Behavioral, and Psychological Impact of the COVID-19 Pandemic. Front. Psychol. 2020, 11, 566212. [Google Scholar] [CrossRef] [PubMed]
- Montemurro, N. The emotional impact of COVID-19: From medical staff to common people. Brain Behav. Immun. 2020, 87, 23–24. [Google Scholar] [CrossRef] [PubMed]
- Benjamin, A.; Kuperman, Y.; Eren, N.; Rotkopf, R.; Amitai, M.; Rossman, H.; Shilo, S.; Meir, T.; Keshet, A.; Nuttman-Shwartz, O.; et al. Stress-related emotional and behavioural impact following the first COVID-19 outbreak peak. Mol. Psychiatry 2021, 26, 6149–6158. [Google Scholar] [CrossRef] [PubMed]
- Panchal, N.; Kamal, R.; Cox, C.; Garfield, R. The Implications of COVID-19 for Mental Health and Substance Use; KFF: Oakland, CA, USA, 2021. [Google Scholar]
- Bobo, E.; Lin, L.; Acquaviva, E.; Caci, H.; Franc, N.; Gamon, L.; Picot, M.C.; Pupier, F.; Speranza, M.; Falissard, B.; et al. How do children and adolescents with Attention Deficit Hyperactivity Disorder (ADHD) experience lockdown during the COVID-19 outbreak? Encephale 2020, 46, S85–S92. [Google Scholar] [CrossRef] [PubMed]
- Jones, L.; Palumbo, D.; Brown, D. Coronavirus: How the pandemic has changed the world economy. BBC News, 24 January 2021. [Google Scholar]
- OECD. The Territorial Impact of COVID-19: Managing the Crisis across Levels of Government; OECD: Washington, DC, USA, 2020. [Google Scholar]
- Rothe, J.; Buse, J.; Uhlmann, A.; Bluschke, A.; Roessner, V. Changes in emotions and worries during the COVID-19 pandemic: An online-survey with children and adults with and without mental health conditions. Child Adolesc. Psychiatry Ment. Health 2021, 15, 11. [Google Scholar] [CrossRef]
- Maryland Emergency Management Agency. Maryland Coronavirus Rumor Control. Maryland Government. January 2023. Available online: https://govstatus.egov.com/md-coronavirus-rumor-control/vaccine-rumors (accessed on 3 March 2022).
- Pertwee, E.; Simas, C.; Larson, H.J. An epidemic of uncertainty: Rumors, conspiracy theories and vaccine hesitancy. Nat. Med. 2022, 28, 456–459. [Google Scholar] [CrossRef]
- Douglas, K.M.; Joelley, D. The Effects of Anti-Vaccine Conspiracy Theories on Vaccination Intentions. PLoS ONE 2014, 9, e89177. [Google Scholar]
- Loomba, S.; de Figueiredo, A.; Piatek, S.J.; de Graaf, K.; Larson, H.J. Measuring the impact of COVID-19 vaccine misinformation on vaccination intent in the UK and USA. Nat. Hum. Behav. 2021, 5, 337–348. [Google Scholar] [CrossRef]
- Bujnowska-Fedak, M.M.; Waligóra, J.; Mastalerz-Migas, A. The Internet as a Source of Health Information and Services. Adv. Innov. Health Sci. 2019, 1211, 1–16. [Google Scholar] [CrossRef]
- McClain, C.; Vogels, E.A.; Perrin, A.; Sechopoulos, S.; Rainie, L. The Internet and the Pandemic; Paw Research Center: Washington, DC, USA, 2021. [Google Scholar]
- Wong, A.; Ho, S.; Olusanya, O.; Antonini, M.V.; Lyness, D. The use of social media and online communications in times of pandemic COVID-19. J. Intensiv. Care Soc. 2020, 22, 255–260. [Google Scholar] [CrossRef]
- Tao, D.; Yang, P.; Feng, H. Utilization of text mining as a big data analysis tool for food science and nutrition. Compr. Rev. Food Sci. Food Saf. 2020, 19, 875–894. [Google Scholar] [CrossRef] [PubMed]
- Oyebode, O.; Ndulue, C.; Adib, A.; Mulchandani, D.; Suruliraj, B.; Orji, F.A.; Chambers, C.T.; Meier, S.; Orji, R. Health, Psychosocial, and Social Issues Emanating From the COVID-19 Pandemic Based on Social Media Comments: Text Mining and Thematic Analysis Approach. JMIR Public Health Surveill. 2021, 9, e22734. [Google Scholar] [CrossRef] [PubMed]
- Piltch-Loeb, R.; Savoia, E.; Goldberg, B.; Hughes, B.; Verhey, T.; Kayyem, J.; Miller-Idriss, C.; Testa, M. Examining the effect of information channel on COVID-19 vaccine acceptance. PLoS ONE 2021, 16, e0251095. [Google Scholar] [CrossRef] [PubMed]
- Raza, G.M.; Butt, Z.S.; Latif, S.; Wahid, A. Sentiment Analysis on COVID Tweets: An Experimental Analysis on the Impact of Count Vectorizer and TF-IDF on Sentiment Predictions using Deep Learning Models. In Proceedings of the 2021 International Conference on Digital Futures and Transformative Technologies (ICoDT2), Islamabad, Pakistan, 20–21 May 2021. [Google Scholar] [CrossRef]
- Khan, R.; Rustam, F.; Kanwal, K.; Mehmood, A.; Choi, G.S. US Based COVID-19 Tweets Sentiment Analysis Using TextBlob and Supervised Machine Learning Algorithms. In Proceedings of the 2021 International Conference on Artificial Intelligence (ICAI), Islamabad, Pakistan, 5–7 April 2021. [Google Scholar] [CrossRef]
- Shamrat, F.M.J.M.; Chakraborty, S.; Imran, M.M.; Muna, J.N.; Billah, M.M.; Das, P.; Rahman, M.O. Sentiment analysis on twitter tweets about COVID-19 vaccines using NLP and supervised KNN classification algorithm. Indones. J. Electr. Eng. Comput. Sci. 2021, 23, 463–470. [Google Scholar]
- Wang, G.; Kwok, S.W.H. Using K-Means Clustering Method with Doc2Vec to Understand the Twitter Users’ Opinions on COVID-19 Vaccination. In Proceedings of the 2021 IEEE EMBS International Conference on Biomedical and Health Informatics (BHI), Athens, Greece, 27–30 July 2021. [Google Scholar] [CrossRef]
- Shahi, T.B.; Sitaula, C.; Paudel, N. A Hybrid Feature Extraction Method for Nepali COVID-19-Related Tweets Classification. Comput. Intell. Neurosci. 2022, 2022, 5681574. [Google Scholar] [CrossRef]
- Alam, K.N.; Khan, S.; Dhruba, A.R.; Khan, M.M.; Al-Amri, J.F.; Masud, M.; Rawashdeh, M. Deep Learning-Based Sentiment Analysis of COVID-19 Vaccination Responses from Twitter Data. Comput. Math. Methods Med. 2021, 2021, 4321131. [Google Scholar] [CrossRef]
- Devi, S.K.; Upadhyay, A.; Dimri, S. COVID-19 Tweets Classification Using RNN in Deep Learning. Ann. Rom. Soc. Cell Biol. 2021, 25, 5177–5185. [Google Scholar]
- Chandra, R.; Krishna, A. COVID-19 sentiment analysis via deep learning during the rise of novel cases. PLoS ONE 2021, 16, e0255615. [Google Scholar] [CrossRef]
- Rustam, F.; Khalid, M.; Aslam, W.; Rupapara, V.; Mehmood, A.; Choi, G.S. A performance comparison of supervised machine learning models for COVID-19 tweets sentiment analysis. PLoS ONE 2021, 16, e0245909. [Google Scholar] [CrossRef]
- To, Q.; To, K.; Huynh, V.-A.; Nguyen, N.; Ngo, D.; Alley, S.; Tran, A.; Tran, A.; Pham, N.; Bui, T.; et al. Applying Machine Learning to Identify Anti-Vaccination Tweets during the COVID-19 Pandemic. Int. J. Environ. Res. Public Health 2021, 18, 4069. [Google Scholar] [CrossRef]
- Cotfas, L.-A.; Delcea, C.; Gherai, R. COVID-19 Vaccine Hesitancy in the Month Following the Start of the Vaccination Process. Int. J. Environ. Res. Public Health 2021, 18, 10438. [Google Scholar] [CrossRef] [PubMed]
- Chintalapudi, N.; Battineni, G.; Amenta, F. Sentimental Analysis of COVID-19 Tweets Using Deep Learning Models. Infect. Dis. Rep. 2021, 13, 329–339. [Google Scholar] [CrossRef]
- Madhusudhana, K. Text Classification Models for Automatic Detection of Fake COVID Products and News on Social Media; ProQuest: Ann Arbor, MI, USA, 2021. [Google Scholar]
- Liu, Y.; Yin, Z.; Ni, C.; Yan, C.; Wan, Z.; Malin, B. Examining Rural and Urban Sentiment Difference in COVID-19–Related Topics on Twitter: Word Embedding–Based Retrospective Study. J. Med. Internet Res. 2023, 25, e42985. [Google Scholar] [CrossRef] [PubMed]
- Hernández, S.C.; Cruz, M.P.T.; Sánchez, J.M.E.; Tzili, A.P. Deep Learning Model for COVID-19 Sentiment Analysis on Twitter. New Gener. Comput. 2023, 1–24. [Google Scholar] [CrossRef]
- Mishra, A.; Wajid, M.S.; Dugal, U. A Comprehensive Analysis of Approaches for Sentiment Analysis Using Twitter Data on COVID-19 Vaccine. J. Inform. Electr. Electron. Eng. 2021, 2, 1–10. [Google Scholar] [CrossRef]
- Liu, S.; Liu, J. Public attitudes toward COVID-19 vaccines on English-language Twitter: A sentiment analysis. Vaccine 2021, 39, 5499–5505. [Google Scholar] [CrossRef]
- Luo, Y.; Kejriwal, M. Understanding COVID-19 Vaccine Reaction Through Comparative Analysis on Twitter. arXiv 2021, arXiv:2111.05823. [Google Scholar]
- Sitaula, C.; Basnet, A.; Mainali, A.; Shahi, T.B. Deep Learning-Based Methods for Sentiment Analysis on Nepali COVID-19-Related Tweets. Comput. Intell. Neurosci. 2021, 2021, 2158184. [Google Scholar] [CrossRef]
- Jarynowski, A.; Semenov, A.; Kamiński, M.; Belik, V. Mild Adverse Events of Sputnik V Vaccine in Russia: Social Media Content Analysis of Telegram via Deep Learning. J. Med. Internet Res. 2021, 23, e30529. [Google Scholar] [CrossRef]
- Chakraborty, A.K.; Das, S.; Kolya, A.K. Sentiment Analysis of COVID-19 Tweets Using Evolutionary Classification-Based LSTM Model. In Proceedings of the Research and Applications in Artificial Intelligence: RAAI, Kolkata, India, 19–20 December 2020. [Google Scholar] [CrossRef]
- Islam, M.N.; Khan, N.I.; Roy, A.; Rahman, M.; Mukta, S.H.; Islam, A.K.M.N. Sentiment Analysis of Bangladesh-specific COVID-19 Tweets using Deep Neural Network. In Proceedings of the 2021 62nd International Scientific Conference on Information Technology and Management Science of Riga Technical University (ITMS), Riga, Latvia, 14–15 October 2021. [Google Scholar] [CrossRef]
- Kumar, V. Spatiotemporal sentiment variation analysis of geotagged COVID-19 tweets from India using a hybrid deep learning model. Sci. Rep. 2022, 12, 1849. [Google Scholar] [CrossRef]
- Mohammad, S.; Turney, P.D. Crowdsourcing a Word–Emotion Association Lexicon. Comput. Intell. 2013, 29, 436–465. [Google Scholar] [CrossRef]
- Beltrán, M.; Navarrete, A.A.S.; Vidal-Castro, C.; Rubio-Manzano, C.; Martínez-Araneda, C. Improving the affective analysis in texts Automatic method to detect affective intensity in lexicons based on Plutchik’s wheel of emotions. Electron. Libr. 2019, 37, 984–1006. [Google Scholar] [CrossRef]
- Plutchik, R. The Emotions; University Press of America: Lanham, MA, USA, 1991. [Google Scholar]
- Mohammad, S.M. NRC Word-Emotion Association Lexicon (Aka EmoLex). May 2021. Available online: https://saifmohammad.com/WebPages/NRC-Emotion-Lexicon.htm (accessed on 17 May 2021).
- American Psychological Association. Dictionary APA. May 2022. Available online: https://dictionary.apa.org/complex-emotion (accessed on 21 March 2021).
No. | Title | Authors | Datasets | Classes | Method | Conclusion |
---|---|---|---|---|---|---|
1 | A performance Comparison of Supervised Machine Learning models for COVID-19 Tweets Sentiment Analysis | Furqan Rustam, Madiha Khalid, Waqar Aslam, Vaibhav Rupapara, Arif Mehmood, and Gyu Sang Choi. | IEEE data port, 7528 tweets | Positive, negative, and neutral | RF, XGBoost, LinearSVC, ETC, DT, LSTM, BiLSTM, and CNN-LSTM | ETC achieves the highest accuracy of 93%, LSTM accuracy is 57.7%, BiLSTM accuracy is 57.9%, and CNN-LSTM accuracy is 61% [41]. |
2 | Applying Machine Learning to Identify Anti-Vaccination Tweets during the COVID-19 Pandemic | Quyen G. To, Kien G. To, Van-Anh N. Huynh, Nhung T. Q. Nguyen, Diep T. N. Ngo, Stephanie J. Alley, Anh N. Q. Tran, Anh N. P. Tran, Ngan T. T. Pham, Thanh X. Bui, and Corneel Vandelanotte. | 1,651,687 English tweets | Positive, negative, and neutral | SVC, NB, BiLSTM, and BERT | BERT has the highest accuracy performance of 91.6% [42]. |
3 | A Hybrid Feature Extraction Method for Nepali COVID-19-Related Tweets Classification | Tej Bahadur Shahi, Chiranjibi Sitaula, and Nawaraj Paudel. | Kaggle, 33,458 tweets | Positive, negative, and neutral | Regression, RF, KNN, Naïve Bayes, SVM AdaBoost, and MLP-ANN | SVM + RBF model has the highest precision of 74.4%, recall of 76.9%, and F1-score of 75.6% [37]. |
4 | Examining Rural and Urban Sentiment Difference in COVID-19—Related Topics on Twitter: Word Embedding—Based Retrospective Study | Yongtai Liu, Zhijun Yin, Congning Ni, Chao Yan, Zhiyu Wan, and Bradley Malin. | Tweepy Python library, 407 million Geotagged tweets | Positive and negative | Vector Clustering and Inference Analysis | The study showed that there was a statistically significant difference in the sentiments of urban and rural Twitter users regarding a wide range of COVID-19–related topics with p-value of p < 0.001 [46]. |
5 | Deep Learning Model for COVID-19 Sentiment Analysis on Twitter | Salvador Contreras Hernández, María Patricia Tzili Cruz, José Martín Espínola Sánchez, and Angélica Pérez Tzili. | 33,776 tweets | Positive and negative | SVM + Naïve Bayes, Logistic Regression, Decision Trees, and BERT | BERT model achieves 97% accuracy in training and 81% in testing [47]. |
6 | Deep Learning-Based Sentiment Analysis of COVID-19 Vaccination Responses from Twitter Data | Kazi Nabiul Alam, Shakib Khan, Abdur Rab Dhruba, Mohammad Monirujjaman Khan, Jehad F. Al-Amri, Mehedi Masud, and Majdi Rawashdeh. | Kaggle, 125,906 tweets | Positive, negative, and neutral | LSTM and BiLSTM | The accuracy of LSTM is 90.59%, and BiLSTM has an accuracy of 90.83% [38]. |
7 | COVID 19 Tweets Classification Using RNN in Deep Learning | S. Kiruthika Devi, Aditya Upadhyay, and Saket Dimri. | 288,500 tweets | Positive, extremely positive, negative, extremely negative, and neutral | LSTM | The model has a precision of 77.52% and an F1-score of 77% [39]. |
8 | COVID-19 sentiment analysis via deep learning during the rise of novel cases | Rohitash Chandra and Aswin Krishna. | Lamsal R datasets consisting of 150,000 tweets from India, 18,000 tweets from Maharashtra (state), and 18,000 tweets from Delhi | Optimistic, thankful, emphatic, pessimistic, anxious, sad, annoyed, denial, official report, surprise, and joking | LSTM, BiLSTM, and BERT | The most tweets are associated with joking, optimistic, or annoyed, and a few are associated with thankful [40]. |
9 | COVID-19 Vaccine Hesitancy in the Month Following the Start of the Vaccination Process | Liviu-Adrian Cotfas, Camelia Delcea, and Rares Gherai. | 1,221,694 cleaned tweets | In favor, neutral, and against | MNB, RF, SVM, BERT and RoBERTa | The model that has the highest accuracy is RoBERTa (78.63%). Negative sentiments are associated with mistrust, freedom, side effects, hiding relevant information, unsafety, inefficiency, existence of alternatives, scam, and moral and religious issues [43]. |
10 | Deep Learning-Based Methods for Sentiment Analysis on Nepali COVID-19-Related Tweets | C. Sitaula, A. Basnet, A. Mainali, and T. B. Shahi. | 624,316 tweets | Positive, negative, and neutral | LinearSVM, RBF (Radial Basis Function) SVM, XGBoost, Artificial Neural Networks, RF, NB, LR, and KNN | RBF-SVM has the highest precision (70.2%), while the highest accuracy is XGBoost (66.7%). The accuracy of CNN-c is 68.7%, while CNN-ft has an accuracy of 68.1% [51]. |
11 | Mild Adverse Events of Sputnik V Vaccine in Russia: Social Media Content Analysis of Telegram via Deep Learning | Andrzej Jarynowski, Alexander Semenov, Mikołaj Kamiński, and Vitaly Belik. | 11,515 self-reported Sputnik on Telegram | Pain, fever, fatigue, and headache | ANN, LSTM, and BERT | Russian Telegram users reported mostly post-pain, fever, and fatigue after the Sputnik V vaccination. BERT has a precision of 91.5%, while LSTM has a precision of 86.6% [52]. |
12 | Sentiment Analysis of COVID-19 Tweets Using Evolutionary Classification-Based LSTM Model | Arunava Kumar Chakraborty, Sourav Das, and Anup Kumar Kolya. | 160,000 tweets | Positive and negative | LSTM | The model (LSTM) has an accuracy of 91.67% with validation accuracy of 84.46% [53]. |
13 | Sentimental Analysis of COVID-19 Tweets Using Deep Learning Models | Nalini Chintalapudi, Gopi Battineni, and Francesco Amenta. | 3090 tweets from GitHub | sad, joy, fear, and anger | BERT, SVM, LR, and LSTM | The highest model accuracy is BERT 89%. LR has an accuracy of 75%, SVM has an accuracy of 74.75%, and LSTM accuracy is 65% [44]. |
14 | Sentiment Analysis of Bangladesh- specific COVID-19 Tweets Using Deep Neural Network | Muhammad Nazrul Islam, Ayon Roy, Saddam Hossain Mukta, Nafiz Imtiaz Khan, MD. Mahbubar Rahman, and A. K. M. Najmul Islam. | 677 positive tweets, 921 negative tweets, and 256 neutral tweets | Positive, negative, and neutral | DNN | Area under the curve is 76%, with 55% people expressing negative sentiment on COVID-19, 38% expressing positive sentiment, and 7% expressing neutral sentiment [54]. |
15 | Spatiotemporal Sentiment Variation Analysis of Geotagged COVID-19 Tweets from India Using a Hybrid Deep Learning Model | Vaibhav Kumar | 128,096 tweets | Positive and negative | BiLSTM + CNN, LSTM, and CNN | The highest model accuracy is BiLSTM + CNN (89.68%) [55]. |
16 | Text Classification Models for Automatic Detection of Fake COVID Products and News on Social Media | Kruthika Madhusudhana | Twitter API | Positive, negative, and neutral | DT, GaussianN, RF, K-Nearest, Fusion Model, Simple Neural Network, CNN, LSTM, and BERT | The model with the highest accuracy is BERT (92.35%). LSTM accuracy is 75.44%, Simple Neural Network has an accuracy of 65.17%, and CNN has an accuracy of 63.75% [45]. |
Relations | Joy–Sadness | Trust–Disgust | Fear–Anger | Surprise–Anticipation | Negative–Positive |
---|---|---|---|---|---|
p-Value | 6.42904 × 10−17 | 1.03093 × 10−74 | 5.73314 × 10−89 | 6.92704 × 10−56 | 1.07326 × 10−69 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Qorib, M.; Oladunni, T.; Denis, M.; Ososanya, E.; Cotae, P. COVID-19 Vaccine Hesitancy: A Global Public Health and Risk Modelling Framework Using an Environmental Deep Neural Network, Sentiment Classification with Text Mining and Emotional Reactions from COVID-19 Vaccination Tweets. Int. J. Environ. Res. Public Health 2023, 20, 5803. https://doi.org/10.3390/ijerph20105803
Qorib M, Oladunni T, Denis M, Ososanya E, Cotae P. COVID-19 Vaccine Hesitancy: A Global Public Health and Risk Modelling Framework Using an Environmental Deep Neural Network, Sentiment Classification with Text Mining and Emotional Reactions from COVID-19 Vaccination Tweets. International Journal of Environmental Research and Public Health. 2023; 20(10):5803. https://doi.org/10.3390/ijerph20105803
Chicago/Turabian StyleQorib, Miftahul, Timothy Oladunni, Max Denis, Esther Ososanya, and Paul Cotae. 2023. "COVID-19 Vaccine Hesitancy: A Global Public Health and Risk Modelling Framework Using an Environmental Deep Neural Network, Sentiment Classification with Text Mining and Emotional Reactions from COVID-19 Vaccination Tweets" International Journal of Environmental Research and Public Health 20, no. 10: 5803. https://doi.org/10.3390/ijerph20105803