Dermato 2024, 4(4), 124-135; https://doi.org/10.3390/dermato4040013 - 30 Sep 2024
Viewed by 390
Abstract
►
Show Figures
Large language models (LLMs) are trained using large datasets and may be applied to language-based tasks. Studies have demonstrated their ability to perform and pass postgraduate medical examinations, and with the increasingly sophisticated deep learning algorithms and incorporation of image-analysis capabilities, they may
[...] Read more.
Large language models (LLMs) are trained using large datasets and may be applied to language-based tasks. Studies have demonstrated their ability to perform and pass postgraduate medical examinations, and with the increasingly sophisticated deep learning algorithms and incorporation of image-analysis capabilities, they may also be applied to the Specialty Certificate Examination (SCE) in Dermatology. The Dermatology SCE sample questions were used to assess the performance of five freely available and high-performance LLMs. The LLMs’ performances were recorded by comparing their output on multiple-choice questions against the sample answers. One hundred questions, four of which included photographs, were entered into the LLMs. The responses were recorded and analysed, with the pass mark set at 77%. The accuracies for Claude-3.5 Sonnet, Copilot, Gemini, ChatGPT-4o, and Perplexity were 87, 88, 75, 90, and 87, respectively (p = 0.023). The LLMs were generally capable of interpreting and providing reasoned responses to clinical scenarios and clinical data. This continues to demonstrate the potential of LLMs in both medical education and clinical settings.
Full article
Figure 1
