Integration and Assessment of ChatGPT in Medical Case Reporting: A Multifaceted Approach
Abstract
:1. Key Messages
- ChatGPT was mainly used for content generation and medical information retrieval in 64 case reports.
- Over 50% of ChatGPT applications began with information retrieval, later evolving for various needs.
- About 70% of authors noted inaccuracies. The second most commonly mentioned limitation was clinical context issues.
- ChatGPT was applied across various medical domains, indicating its widespread applicability.
- Effective but requires oversight; enhancements needed for accuracy and context.
2. Introduction
3. Materials and Methods
3.1. Search Strategy for Case Report Using ChatGPT
3.2. Data Extraction
3.3. Data Analysis
4. Results
4.1. Characteristics of Case Reports Using ChatGPT
4.2. Application of ChatGPT Mentioned in Case Reports
4.3. Limitation of ChatGPT Mentioned in Case Reports
5. Discussion
6. Conclusions
Supplementary Materials
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Hassani, H.; Silva, E.S. The role of ChatGPT in data science: How AI-assisted conversational interfaces are revolutionizing the field. Big Data Cogn. Comput. 2023, 7, 62. [Google Scholar] [CrossRef]
- Singh, H.; Singh, A. ChatGPT: Systematic review, applications, and agenda for multidisciplinary research. J. Chin. Econ. Bus. Stud. 2023, 21, 193–212. [Google Scholar] [CrossRef]
- Liu, H.; Azam, M.; Bin Naeem, S.; Faiola, A. An overview of the capabilities of ChatGPT for medical writing and its implications for academic integrity. Health Inf. Libr. J. 2023, 40, 440–446. [Google Scholar] [CrossRef] [PubMed]
- Haque, M.A. A brief analysis of ‘ChatGPT’—A revolutionary tool designed by OpenAI. EAI Endorsed Trans. AI Robot. 2023, 1, e15. [Google Scholar] [CrossRef]
- Zohery, M. ChatGPT in academic writing and publishing: A comprehensive guide. In Artificial Intelligence in Academia, Research and Science: ChatGPT as a Case Study; Achtago: London, UK, 2023; pp. 10–61. [Google Scholar]
- Rahman, M.; Terano, H.J.; Rahman, N.; Salamzadeh, A. ChatGPT and academic research: A review and recommendations based on practical examples. J. Educ. Manag. Dev. Stud. 2023, 3, 1–12. [Google Scholar] [CrossRef]
- Xie, Y.; Seth, I.; Hunter-Smith, D.J.; Rozen, W.M.; Ross, R.; Lee, M. Aesthetic surgery advice and counseling from artificial intelligence: A rhinoplasty consultation with ChatGPT. Aesthetic Plast. Surg. 2023, 47, 1985–1993. [Google Scholar] [CrossRef] [PubMed]
- Hopkins, A.M.; Logan, J.M.; Kichenadasse, G.; Sorich, M.J. Artificial intelligence chatbots will revolutionize how cancer patients access information: ChatGPT represents a paradigm-shift. JNCI Cancer Spectr. 2023, 7, pkad010. [Google Scholar] [CrossRef] [PubMed]
- Srivastava, M. A day in the life of ChatGPT as an academic reviewer: Investigating the potential of large language model for scientific literature review. OSF Prepr. 2023. [Google Scholar] [CrossRef]
- Ruksakulpiwat, S.; Kumar, A.; Ajibade, A. Using ChatGPT in medical research: Current status and future directions. J. Multidiscip. Healthc. 2023, 16, 1513–1520. [Google Scholar] [CrossRef]
- Liu, S.; Wright, A.P.; Patterson, B.L.; Wanderer, J.P.; Turer, R.W.; Nelson, S.D.; McCoy, A.B.; Sittig, D.F.; Wright, A. Using AI-generated suggestions from ChatGPT to optimize clinical decision support. J. Am. Med. Inform. Assoc. 2023, 30, 1237–1245. [Google Scholar] [CrossRef]
- Tenhundfeld, N.L. Two Birds with One Stone: Writing a Paper Entitled ‘ChatGPT as a Tool for Studying Human-AI Interaction in the Wild’ with ChatGPT. Available online: https://www.researchgate.net/publication/368300523_Two_Birds_With_One_Stone_Writing_a_Paper_Entitled_ChatGPT_as_a_Tool_for_Studying_Human-AI_Interaction_in_the_Wild_with_ChatGPT?channel=doi&linkId=63e118c9c97bd76a82765195&showFulltext=true. [CrossRef]
- Loh, E. ChatGPT and generative AI chatbots: Challenges and opportunities for science, medicine and medical leaders. BMJ Lead. 2024, 8, 51–54. [Google Scholar] [CrossRef] [PubMed]
- Preiksaitis, C.; Sinsky, C.A.; Rose, C. ChatGPT is not the solution to physicians’ documentation burden. Nat. Med. 2023, 29, 1296–1297. [Google Scholar] [CrossRef] [PubMed]
- Sirrianni, J.; Sezgin, E.; Claman, D.; Linwood, S.L. Medical text prediction and suggestion using generative pretrained transformer models with dental medical notes. Methods Inf. Med. 2022, 61, 195–200. [Google Scholar] [CrossRef] [PubMed]
- Waisberg, E.; Ong, J.; Masalkhi, M.; Kamran, S.A.; Zaman, N.; Sarker, P.; Lee, A.G.; Tavakkoli, A. GPT-4 and ophthalmology operative notes. Ann. Biomed. Eng. 2023, 51, 2353–2355. [Google Scholar] [CrossRef] [PubMed]
- Singh, S.; Djalilian, A.; Ali, M.J. ChatGPT and ophthalmology: Exploring its potential with discharge summaries and operative notes. Semin. Ophthalmol. 2023, 38, 503–507. [Google Scholar] [CrossRef] [PubMed]
- Bosbach, W.A.; Senge, J.F.; Nemeth, B.; Omar, S.H.; Mitrakovic, M.; Beisbart, C.; Horváth, A.; Heverhagen, J.; Daneshvar, K. Ability of ChatGPT to generate competent radiology reports for distal radius fracture using RSNA template items and integrated AO classifier. Curr. Probl. Diagn. Radiol. 2024, 53, 102–110. [Google Scholar] [CrossRef] [PubMed]
- Elkassem, A.A.; Smith, A.D. Potential use cases for ChatGPT in radiology reporting. AJR Am. J. Roentgenol. 2023, 221, 373–376. [Google Scholar] [CrossRef] [PubMed]
- Rau, A.; Rau, S.; Zoeller, D.; Fink, A.; Tran, H.; Wilpert, C.; Nattenmueller, J.; Neubauer, J.; Bamberg, F.; Reisert, M.; et al. A context-based chatbot surpasses trained radiologists and generic ChatGPT in following the ACR Appropriateness Guidelines. Radiology 2023, 308, e230970. [Google Scholar] [CrossRef]
- Patel, S.B.; Lam, K. ChatGPT: The future of discharge summaries? Lancet Digit. Health 2023, 5, e107–e108. [Google Scholar] [CrossRef]
- Ariyaratne, S.; Iyengar, K.P.; Nischal, N.; Chitti Babu, N.; Botchu, R. A comparison of ChatGPT-generated articles with human-written articles. Skelet. Radiol. 2023, 52, 1755–1758. [Google Scholar] [CrossRef]
- Au Yeung, J.; Kraljevic, Z.; Luintel, A.; Balston, A.; Idowu, E.; Dobson, R.J.; Teo, J.T. AI chatbots not yet ready for clinical use. Front. Digit. Health 2023, 5, 1161098. [Google Scholar] [CrossRef] [PubMed]
- Barrot, J.S. Using ChatGPT for second language writing: Pitfalls and potentials. Assess. Writ. 2023, 57, 100745. [Google Scholar] [CrossRef]
- Fournier-Tombs, E.; McHardy, J. A medical ethics framework for conversational artificial intelligence. J. Med. Internet Res. 2023, 25, e43068. [Google Scholar] [CrossRef] [PubMed]
- Dave, T.; Athaluri, S.A.; Singh, S. ChatGPT in medicine: An overview of its applications, advantages, limitations, future prospects, and ethical considerations. Front. Artif. Intell. 2023, 6, 1169595. [Google Scholar] [CrossRef] [PubMed]
- Eke, D.O. ChatGPT and the rise of generative AI: Threat to academic integrity? J. Responsib. Technol. 2023, 13, 100060. [Google Scholar] [CrossRef]
- Mijwil, M.M.; Hiran, K.K.; Doshi, R.; Dadhich, M.; Al-Mistarehi, A.-H.; Bala, I. ChatGPT and the future of academic integrity in the artificial intelligence era: A new frontier. Al-Salam J. Eng. Technol. 2023, 2, 116–127. [Google Scholar] [CrossRef]
- Stokel-Walker, C. ChatGPT listed as author on research papers: Many scientists disapprove. Nature 2023, 613, 620–621. [Google Scholar] [CrossRef]
- Su, Y.; Lin, Y.; Lai, C. Collaborating with ChatGPT in argumentative writing classrooms. Assess. Writ. 2023, 57, 100752. [Google Scholar] [CrossRef]
- Zheng, H.; Zhan, H. ChatGPT in scientific writing: A cautionary tale. Am. J. Med. 2023, 136, 725–726. [Google Scholar] [CrossRef]
- Yang, J.; Chen, Y.-L.; Por, L.Y.; Ku, C.S. A systematic literature review of information security in chatbots. Appl. Sci. 2023, 13, 6355. [Google Scholar] [CrossRef]
- Nakagawa, K.; Moukheiber, L.; Celi, L.A.; Patel, M.; Mahmood, F.; Gondim, D.; Hogarth, M.; Levenson, R. AI in pathology: What could possibly go wrong? Semin. Diagn. Pathol. 2023, 40, 100–108. [Google Scholar] [CrossRef] [PubMed]
- Meskó, B. Prompt Engineering as an Important Emerging Skill for Medical Professionals: Tutorial. J. Med. Internet Res. 2023, 25, e50638. [Google Scholar] [CrossRef]
- Vaishya, R.; Misra, A.; Vaish, A. ChatGPT: Is this version good for healthcare and research? Diabetes Metab. Syndr. Clin. Res. Rev. 2023, 17, 102744. [Google Scholar] [CrossRef] [PubMed]
- Hosseini, M.; Rasmussen, L.M.; Resnik, D.B. Using AI to write scholarly publications. Account. Res. 2023, 1–9. [Google Scholar] [CrossRef]
- Seth, I.; Rodwell, A.; Tso, R.; Valles, J.; Bulloch, G.; Seth, N. A conversation with an open artificial intelligence platform on osteoarthritis of the hip and treatment. J. Orthop. Sports Med. 2023, 5, 112–120. [Google Scholar] [CrossRef]
- Ricci, F.L.; Consorti, F.; Pecoraro, F.; Luzi, D.; Tamburis, O. A Petri-Net-Based Approach for Enhancing Clinical Reasoning in Medical Education. IEEE Trans. Learn. Technol. 2022, 15, 167–178. [Google Scholar] [CrossRef]
- Lingard, L. Writing with ChatGPT: An illustration of its capacity, limitations & implications for academic writers. Perspect. Med. Educ. 2023, 12, 261–270. [Google Scholar] [CrossRef]
- Sallam, M.; Salim, N.; Barakat, M.; Al-Tammemi, A. ChatGPT applications in medical, dental, pharmacy, and public health education: A descriptive study highlighting the advantages and limitations. Narra J. 2023, 3, e103. [Google Scholar] [CrossRef] [PubMed]
- Van Bulck, L.; Moons, P. What if your patient switches from Dr. Google to Dr. ChatGPT? A vignette-based survey of the trustworthiness, value and danger of ChatGPT-generated responses to health questions. Eur. J. Cardiovasc. Nurs. 2024, 23, 95–98. [Google Scholar] [CrossRef]
- Gupta, R.; Herzog, I.; Weisberger, J.; Chao, J.; Chaiyasate, K.; Lee, E.S. Utilization of ChatGPT for plastic surgery research: Friend or foe? J. Plast. Reconstr. Aesthet. Surg. 2023, 80, 145–147. [Google Scholar] [CrossRef]
- Oh, N.; Choi, G.S.; Lee, W.Y. ChatGPT goes to the operating room: Evaluating GPT-4 performance and its potential in surgical education and training in the era of large language models. Ann. Surg. Treat. Res. 2023, 104, 269–273. [Google Scholar] [CrossRef] [PubMed]
Characteristics | No. (Percentage) |
---|---|
Language employed when using ChatGPT | |
English | 65 (98.5%) |
Spanish | 1 (1.5%) |
Specified usage of ChatGPT | |
Mentioned in content | 66 (100%) |
Limitation of ChatGPT | |
Mentioned in content | 32 (48.5%) |
Not mentioned | 34 (51.5%) |
Version of ChatGPT | |
Web version only | 64 (97.0%) |
Web version with extended function | 1 (1.5%) |
Web and desktop version | 1 (1.5%) |
Specialty of case reports | |
Internal Medicine | 14 (21.2%) |
Otolaryngology | 6 (9.1%) |
Dermatology | 5 (7.6%) |
Neurosurgery | 4 (6.1%) |
Cardiology | 4 (6.1%) |
Others | 33 (50.0%) |
Published Journal | |
Cureus | 63 (95.5%) |
Arch. Soc. Esp. Oftalmol. | 1 (1.5%) |
Clin. Case Rep. | 1 (1.5%) |
eNeurologicalSci | 1 (1.5%) |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Lin, K.-C.; Chen, T.-A.; Lin, M.-H.; Chen, Y.-C.; Chen, T.-J. Integration and Assessment of ChatGPT in Medical Case Reporting: A Multifaceted Approach. Eur. J. Investig. Health Psychol. Educ. 2024, 14, 888-901. https://doi.org/10.3390/ejihpe14040057
Lin K-C, Chen T-A, Lin M-H, Chen Y-C, Chen T-J. Integration and Assessment of ChatGPT in Medical Case Reporting: A Multifaceted Approach. European Journal of Investigation in Health, Psychology and Education. 2024; 14(4):888-901. https://doi.org/10.3390/ejihpe14040057
Chicago/Turabian StyleLin, Kuan-Chen, Tsung-An Chen, Ming-Hwai Lin, Yu-Chun Chen, and Tzeng-Ji Chen. 2024. "Integration and Assessment of ChatGPT in Medical Case Reporting: A Multifaceted Approach" European Journal of Investigation in Health, Psychology and Education 14, no. 4: 888-901. https://doi.org/10.3390/ejihpe14040057
APA StyleLin, K. -C., Chen, T. -A., Lin, M. -H., Chen, Y. -C., & Chen, T. -J. (2024). Integration and Assessment of ChatGPT in Medical Case Reporting: A Multifaceted Approach. European Journal of Investigation in Health, Psychology and Education, 14(4), 888-901. https://doi.org/10.3390/ejihpe14040057