Dramatic Increases in Telehealth-Related Tweets during the Early COVID-19 Pandemic: A Sentiment Analysis
Abstract
:1. Introduction
2. Materials and Methods
2.1. Raw Data
2.2. Manual Data Annotation
2.3. Automatic NLP-Based Annotation
3. Results
4. Discussion
5. Conclusions
Supplementary Materials
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Guo, Y.R.; Cao, Q.D.; Hong, Z.S.; Tan, Y.Y.; Chen, S.D.; Jin, H.J.; Tan, K.S.; Wang, D.Y.; Yan, Y. The origin, transmission and clinical therapies on coronavirus disease 2019 (COVID-19) outbreak - an update on the status. Mil. Med. Res. 2020, 7, 11. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Ali, N.A.; Khoja, A. Telehealth: An Important Player during the COVID-19 Pandemic. Ochsner J. 2020, 20, 113–114. [Google Scholar] [CrossRef] [PubMed]
- Ohannessian, R.; Duong, T.A.; Odone, A. Global Telemedicine Implementation and Integration within Health Systems to Fight the COVID-19 Pandemic: A Call to Action. JMIR. Public Health Surveill 2020, 6, e18810. [Google Scholar] [CrossRef] [PubMed]
- Gottlieb, M.; Dyer, S. Information and Disinformation: Social Media in the COVID-19 Crisis. Acad. Emerg. Med. 2020, 27, 640–641. [Google Scholar] [CrossRef] [PubMed]
- Ahn, J.; Jeon, Y.A.; Murthy, D. Differences in Health-Related Social Media Usage by Organizations and Individuals. Telemed. J. E Health 2020, 26, 812–820. [Google Scholar] [CrossRef] [PubMed]
- Ayers, J.W.; Leas, E.C.; Allem, J.P.; Benton, A.; Dredze, M.; Althouse, B.M.; Cruz, T.B.; Unger, J.B. Why do people use electronic nicotine delivery systems (electronic cigarettes)? A content analysis of Twitter, 2012–2015. PLoS ONE 2017, 12, e0170702. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Esteva, A.; Robicquet, A.; Ramsundar, B.; Kuleshov, V.; DePristo, M.; Chou, K.; Cui, C.; Corrado, G.; Thrun, S.; Dean, J. A guide to deep learning in healthcare. Nat. Med. 2019, 25, 24–29. [Google Scholar] [CrossRef] [PubMed]
- Low, D.M.; Rumker, L.; Talkar, T.; Torous, J.; Cecchi, G.; Ghosh, S.S. Natural Language Processing Reveals Vulnerable Mental Health Support Groups and Heightened Health Anxiety on Reddit During COVID-19: Observational Study. J. Med. Internet Res. 2020, 22, e22635. [Google Scholar] [CrossRef] [PubMed]
- Alnemer, K.A.; Alhuzaim, W.M.; Alnemer, A.A.; Alharbi, B.B.; Bawazir, A.S.; Barayyan, O.R.; Balaraj, F.K. Are Health-Related Tweets Evidence Based? Review and Analysis of Health-Related Tweets on Twitter. J. Med. Internet Res. 2015, 17, e246. [Google Scholar] [CrossRef] [PubMed]
- Devlin, J.; Chang, M.-W.; Lee, K.; Toutanova, K. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv 2018, arXiv:1810.04805. [Google Scholar]
- Devlin, J.; Chang, M.-W. Open Sourcing BERT: State-of-the-Art Pre-training for Natural Language Processing. Google AI Blog. Available online: https://ai.googleblog.com/2018/11/open-sourcing-bert-state-of-art-pre.html (accessed on 18 May 2021).
- Hernandez-Suarez, A.; Sanchez-Perez, G.; Toscano-Medina, K.; Martinez-Hernandez, V.; Perez-Meana, H.; Olivares-Mercado, J.; Sanchez, V. Social Sentiment Sensor in Twitter for Predicting Cyber-Attacks Using ℓ₁ Regularization. Sensors 2018, 18, 1380. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Lanning, K.; Pauletti, R.E.; King, L.A.; McAdams, D.P. Personality development through natural language. Nat. Hum. Behav. 2018, 2, 327–334. [Google Scholar] [CrossRef] [PubMed]
- Bhat, M.; Qadri, M.; Beg, N.U.; Kundroo, M.; Ahanger, N.; Agarwal, B. Sentiment analysis of social media response on the Covid19 outbreak. Brain Behav. Immun 2020, 87, 136–137. [Google Scholar] [CrossRef] [PubMed]
- Miller, R.J. Internet marketing 101. Facial Plast. Surg. Clin. N. Am. 2010, 18, 509–516. [Google Scholar] [CrossRef] [PubMed]
- Kam, X.N.; Stoyneshka, I.; Tornyova, L.; Fodor, J.D.; Sakas, W.G. Bigrams and the richness of the stimulus. Cogn. Sci. 2008, 32, 771–787. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- World Health Organization. Rolling Updates on a Coronavirus Disease (COVID-19). Available online: https://www.who.int/emergencies/diseases/novel-coronavirus-2019/events-as-they-happen (accessed on 15 December 2020).
- Madden, N.; Emeruwa, U.N.; Friedman, A.M.; Aubey, J.J.; Aziz, A.; Baptiste, C.D.; Coletta, J.M.; D’Alton, M.E.; Fuchs, K.M.; Goffman, D.; et al. Telehealth Uptake into Prenatal Care and Provider Attitudes during the COVID-19 Pandemic in New York City: A Quantitative and Qualitative Analysis. Am. J. Perinatol. 2020, 37, 1005–1014. [Google Scholar] [CrossRef] [PubMed]
- Greaves, F.; Laverty, A.A.; Cano, D.R.; Moilanen, K.; Pulman, S.; Darzi, A.; Millett, C. Tweets about hospital quality: A mixed methods study. BMJ. Qual. Saf. 2014, 23, 838–846. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Savage, M.; Devine, F.; Cunningham, N.; Taylor, M.; Li, Y.; Hjellbrekke, J.; Le Roux, B.; Friedman, S.; Miles, A. A New Model of Social Class? Findings from the BBC’s Great British Class Survey Experiment. Sociology 2013, 47, 219–250. [Google Scholar] [CrossRef] [Green Version]
- Denecke, K.; Deng, Y. Sentiment analysis in medical settings: New opportunities and challenges. Artif. Intell. Med. 2015, 64, 17–27. [Google Scholar] [CrossRef] [PubMed]
- Shepherd, A.; Sanders, C.; Doyle, M.; Shaw, J. Using social media for support and feedback by mental health service users: Thematic analysis of a twitter conversation. BMC. Psychiatry 2015, 15, 29. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Larson, S.; Popov, V.; Ali, A.M.; Ramanathan, P.; Jung, S. Healthcare Professionals’ Perceptions of Telehealth: Analysis of Tweets from Pre- and During the COVID-19 Pandemic. In Proceedings of the International Conference on Quantitative Ethnography, Malibu, CA, USA, 1–3 February 2021; Ruis, A.R., Lee, S.B., Eds.; Springer: Berlin, Germany, 2021. [Google Scholar] [CrossRef]
BERT Model | Accuracy | Precision | Recall | F1 | AUROC * | |
---|---|---|---|---|---|---|
Telehealth | BERT-base | 98.3% | 98.5% | 99.7% | 99.1 | 0.982 |
BERT-telehealth | 98.5% | 98.8% | 99.5% | 99.2 | 0.989 | |
Sentiment | BERT-base | 67.8% | 63.6% | 56.3% | 58.8 | N/A |
BERT-telehealth | 70.4% | 70.0% | 61.7% | 64.5 | N/A | |
User | BERT-base | 67.5% | 53.8% | 53.7% | 53.7 | N/A |
BERT-telehealth | 69.0% | 57.6% | 54.7% | 56.0 | N/A | |
COVID-19 | BERT-base | 93.6% | 91.3% | 83.2% | 87.1 | 0.940 |
BERT-telehealth | 94.9% | 94.5% | 85.2% | 89.6 | 0.952 |
Category | Definition | User Count | (%) |
---|---|---|---|
Clinician | A person who treats patients | 15,136 | (7.9) |
Consumer | A patient or other user of telehealth | 6381 | (3.3) |
Policymaker | A person who makes or influences governmental policy | 1544 | (0.8) |
Vendor | Any user with an economic interest in telehealth | 24,888 | (12.9) |
Other | Any other user who cannot be classified as above | 144,481 | (75.1) |
Category | Definition | Example Tweet | n | (%) |
---|---|---|---|---|
Positive | Supports use of telehealth | Telehealth may be especially helpful as an initial option for COVID-19 | 112,721 | (58.6) |
Neutral | No overt positive or negative sentiment | Telehealth During COVID-19: New Rules and Considerations | 72,369 | (37.6) |
Negative | Dissatisfaction with telehealth | I have a telehealth appointment with my tomorrow and it’s going to be so weird | 7340 | (3.8) |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Champagne-Langabeer, T.; Swank, M.W.; Manas, S.; Si, Y.; Roberts, K. Dramatic Increases in Telehealth-Related Tweets during the Early COVID-19 Pandemic: A Sentiment Analysis. Healthcare 2021, 9, 634. https://doi.org/10.3390/healthcare9060634
Champagne-Langabeer T, Swank MW, Manas S, Si Y, Roberts K. Dramatic Increases in Telehealth-Related Tweets during the Early COVID-19 Pandemic: A Sentiment Analysis. Healthcare. 2021; 9(6):634. https://doi.org/10.3390/healthcare9060634
Chicago/Turabian StyleChampagne-Langabeer, Tiffany, Michael W. Swank, Shruthi Manas, Yuqi Si, and Kirk Roberts. 2021. "Dramatic Increases in Telehealth-Related Tweets during the Early COVID-19 Pandemic: A Sentiment Analysis" Healthcare 9, no. 6: 634. https://doi.org/10.3390/healthcare9060634