Leveraging LLMs for Efficient Topic Reviews
Abstract
:1. Introduction
- A novel framework for semi-automatic literature review processes is proposed, utilizing the synergistic potential of LLMs and BERTopic. This framework is tailored to enhance the depth and breadth of literature analyses, ensuring comprehensive coverage across scholarly databases.
- A single case study within this framework is developed and presented. This case study encompasses a specific domain demonstrating the applicability and robustness of the proposed method.
- Specific metrics to assess the quality of the identified topics have been defined, contributing to the validation and continuous improvement of the framework and literature review process.
- Through surveys and statistical tests such as Fleiss’ Kappa, the case study has been rigorously evaluated by subject-matter experts. This expert validation, along with the statistical analysis, underscores the effectiveness of the framework in extracting relevant and profound insights from extensive scientific literature.
2. Literature Review
3. Framework
3.1. Core Components
3.1.1. Embedding
3.1.2. Dimensional Reduction
3.1.3. Clustering
3.1.4. Feature Extraction
3.1.5. Topic Representation
3.2. Parameter Settings and Configurations
3.3. Metrics
- Similarity based on indexed keywords: In this study, the cosine-based similarity technique using BERT embeddings is employed to compare the relationship between Scopus indexed keywords and keywords generated by the KeyBERT tool [58]. This technique relies on the vector representation of texts, where each word is converted into a numerical vector in a high-dimensional space. The similarity between two texts is calculated by evaluating the angle between the vectors representing these texts. The closer the angle is to zero, the higher the similarity between the texts. This approach allows for a quantitative and accurate comparison of keywords, facilitating the evaluation of the quality of the clustered topic generation process. Additionally, statistical tests are conducted to determine the significance of the observed similarity results, using a p-value threshold to assess whether the results are statistically significant at a 95% confidence level.
- Thematic coherence analysis: In our study, we assess the relevance of clustered themes derived from the TR process, utilizing a survey administered to seven expert raters. The evaluation focuses on two key parameters: meaningfulness and importance; both evaluated as high, medium, and low. Meaningfulness pertains to the extent to which the AI-generated custom names for each topic cluster accurately and significantly represent concepts within the fields of SLR, ML, and NLP, reflecting the depth and relevance of these topics in the broader area of knowledge. Importance evaluates the perceived significance of each custom name, considering its impact, influence, or critical value within the area of knowledge research. To quantify the agreement among raters regarding these evaluations, we employ the Fleiss’ score. Fleiss’ Kappa () is a statistical measure for assessing the reliability of agreement between a fixed number of raters when assigning categorical ratings to a number of items. It is expressed as:
3.4. Experiment
Hardware
4. Results and Discussion
4.1. Results and Analysis
4.1.1. Visualization
4.1.2. Evaluation of Topic Generation Quality
4.1.3. Expert Analysis
4.2. Discussion
5. Conclusions and Future Work
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
Abbreviations
Symbol | Description |
Cosine Similarity | |
Standard Deviation | |
Proportion of times that raters agree | |
Values—representing proportions of ratings across High, Medium | |
and Low categories | |
Hypothetical probability of chance agreement | |
Acronym | Description |
AI | Artificial intelligence |
ATC | Automated Text Classification |
BAAI | Beijing Academy of Artificial Intelligence |
BERT | Bidirectional Encoder Representations from Transformers |
c-TF-IDF | Class-based Term Frequency-Inverse Document Frequency |
DTA | Diagnostic Test Accuracy |
GPT | Generative Pre-trained Transformer |
HDBSCAN | Hierarchical Density-Based Spatial Clustering of Applications with Noise |
HFSRM | Hybrid Feature Selection Rule Measures |
LDA | Latent Dirichlet Allocation |
LIS | Library & Information Science |
LLM | Large Language Model |
MECCIR | Methodological Expectations of Campbell Collaboration Intervention Reviews |
Mesh | Medical Subject Headings |
ML | Machine Learning |
MTEB | Massive Text Embedding Benchmark |
NLP | Natural Language Processing |
PCA | Principal Component Analysis |
PRISMA | Preferred Reporting Items for Systematic reviews and Meta-Analyses |
PLSA | Probabilistic Latent Semantic Analysis |
SLR | Systematic Literature Review |
TR | Topic Review |
SGPT | GPT Sentence Embeddings for Semantic Search |
SVM | Support Vector Machines |
t-SNE | t-distributed Stochastic Neighbor Embedding |
UMAP | Uniform Manifold Approximation and Projection |
References
- Sundaram, G.; Berleant, D. Automating systematic literature reviews with natural language processing and text mining: A systematic literature review. In Proceedings of the International Congress on Information and Communication Technology, London, UK, 20–23 February 2023; Springer: Singapore, 2023; pp. 73–92. [Google Scholar]
- De la Torre-López, J.; Ramírez, A.; Romero, J.R. Artificial intelligence to automate the systematic review of scientific literature. Computing 2023, 105, 2171–2194. [Google Scholar] [CrossRef]
- Moreno-Garcia, C.F.; Jayne, C.; Elyan, E.; Aceves-Martins, M. A novel application of machine learning and zero-shot classification methods for automated abstract screening in systematic reviews. Decis. Anal. J. 2023, 6, 100162. [Google Scholar] [CrossRef]
- Adeva, J.G.; Atxa, J.P.; Carrillo, M.U.; Zengotitabengoa, E.A. Automatic text classification to support systematic reviews in medicine. Expert Syst. Appl. 2014, 41, 1498–1508. [Google Scholar] [CrossRef]
- Fu, L.; Aliferis, C. Using content-based and bibliometric features for machine learning models to predict citation counts in the biomedical literature. Scientometrics 2010, 85, 257–270. [Google Scholar] [CrossRef]
- Ali, Z.; Kefalas, P.; Muhammad, K.; Ali, B.; Imran, M. Deep learning in citation recommendation models survey. Expert Syst. Appl. 2020, 162, 113790. [Google Scholar] [CrossRef]
- Larsen, K.R.; Hovorka, D.; Dennis, A.; West, J.D. Understanding the elephant: The discourse approach to boundary identification and corpus construction for theory review articles. J. Assoc. Inf. Syst. 2019, 20, 15. [Google Scholar] [CrossRef]
- Kunnath, S.N.; Herrmannova, D.; Pride, D.; Knoth, P. A meta-analysis of semantic classification of citations. Quant. Sci. Stud. 2021, 2, 1170–1215. [Google Scholar] [CrossRef]
- Nasar, Z.; Jaffry, S.W.; Malik, M.K. Information extraction from scientific articles: A survey. Scientometrics 2018, 117, 1931–1990. [Google Scholar] [CrossRef]
- Wagner, G.; Lukyanenko, R.; Paré, G. Artificial intelligence and the conduct of literature reviews. J. Inf. Technol. 2022, 37, 209–226. [Google Scholar] [CrossRef]
- Antons, D.; Breidbach, C.F.; Joshi, A.M.; Salge, T.O. Computational literature reviews: Method, algorithms, and roadmap. Organ. Res. Methods 2023, 26, 107–138. [Google Scholar] [CrossRef]
- Da Silva Júnior, E.M.; Dutra, M.L. A roadmap toward the automatic composition of systematic literature reviews. Iberoam. J. Sci. Meas. Commun. 2021, 1, 1–22. [Google Scholar] [CrossRef]
- Tauchert, C.; Bender, M.; Mesbah, N.; Buxmann, P. Towards an Integrative Approach for Automated Literature Reviews Using Machine Learning. In Proceedings of the 53rd Hawaii International Conference on System Sciences, HICSS 2020, Maui, HI, USA, 7–10 January 2020. [Google Scholar]
- Grootendorst, M. BERTopic: Neural topic modeling with a class-based TF-IDF procedure. arXiv 2022, arXiv:2203.05794. [Google Scholar]
- Garcia, J.; Villavicencio, G.; Altimiras, F.; Crawford, B.; Soto, R.; Minatogawa, V.; Franco, M.; Martínez-Muñoz, D.; Yepes, V. Machine learning techniques applied to construction: A hybrid bibliometric analysis of advances and future directions. Autom. Constr. 2022, 142, 104532. [Google Scholar] [CrossRef]
- Hofmann, T. Unsupervised learning by probabilistic latent semantic analysis. Mach. Learn. 2001, 42, 177–196. [Google Scholar] [CrossRef]
- Blei, D.; Ng, A.; Jordan, M. Latent dirichlet allocation. J. Mach. Learn. Res. 2003, 3, 993–1022. [Google Scholar]
- Lee, D.; Seung, H. Learning the parts of objects by non-negative matrix factorization. Nature 1999, 401, 788–791. [Google Scholar] [CrossRef] [PubMed]
- Arora, S.; Ge, R.; Moitra, A. Learning topic models–going beyond SVD. In Proceedings of the 2012 IEEE 53rd Annual Symposium on Foundations of Computer Science, New Brunswick, NJ, USA, 20–23 October 2012; pp. 1–10. [Google Scholar] [CrossRef]
- Pourreza, M.; Ensan, F. Towards semantic-driven boolean query formalization for biomedical systematic literature reviews. Int. J. Med. Inform. 2023, 170, 104928. [Google Scholar] [CrossRef]
- Scells, H.; Forbes, C.; Clark, J.; Koopman, B.; Zuccon, G. The Impact of Query Refinement on Systematic Review Literature Search: A Query Log Analysis. In Proceedings of the 2022 ACM SIGIR International Conference on Theory of Information Retrieval, Madrid, Spain, 11–12 July 2022; pp. 34–42. [Google Scholar]
- O’Keefe, H.; Rankin, J.; Wallace, S.A.; Beyer, F. Investigation of text-mining methodologies to aid the construction of search strategies in systematic reviews of diagnostic test accuracy—A case study. Res. Synth. Methods 2023, 14, 79–98. [Google Scholar] [CrossRef]
- Sutton, A.; O’Keefe, H.; Johnson, E.E.; Marshall, C. A mapping exercise using automated techniques to develop a search strategy to identify systematic review tools. Res. Synth. Methods 2023, 14, 874–881. [Google Scholar] [CrossRef]
- Young, S.; Bethel, A.; Keenan, C.; Ghezzi-Kopel, K.; Moreton, E.; Pickup, D.; Premji, Z.A.; Rogers, M.; Viinholt, B.C. PROTOCOL: Searching and reporting in Campbell Collaboration systematic reviews: An assessment of current methods. Campbell Syst. Rev. 2021, 17, e1208. [Google Scholar] [CrossRef]
- Page, M.J.; McKenzie, J.E.; Bossuyt, P.M.; Boutron, I.; Hoffmann, T.C.; Mulrow, C.D.; Shamseer, L.; Tetzlaff, J.M.; Akl, E.A.; Brennan, S.E.; et al. The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ 2021, 372, n71. [Google Scholar] [CrossRef] [PubMed]
- Almeida, H.; Meurs, M.J.; Kosseim, L.; Tsang, A. Data sampling and supervised learning for HIV literature screening. IEEE Trans. Nanobiosci. 2016, 15, 354–361. [Google Scholar] [CrossRef] [PubMed]
- Norman12, C.; Leeflang, M.; Névéol, A. LIMSI@CLEF ehealth 2017 task 2: Logistic regression for automatic article ranking. In Proceedings of the CEUR Workshop Proceedings: Working Notes of CLEF 2019: Conference and Labs of the Evaluation Forum, Lugano, Switzerland, 9–12 September 2019. [Google Scholar]
- Norman, C.R.; Leeflang, M.M.; Névéol, A. LIMSI@CLEF eHealth 2018 Task 2: Technology Assisted Reviews by Stacking Active and Static Learning. In Proceedings of the CLEF 2018—Working Notes of CLEF 2018 Conference and Labs of the Evaluation Forum, Avignon, France, 10–14 September 2018; Volume 2125, pp. 1–13. [Google Scholar]
- van den Bulk, L.M.; Bouzembrak, Y.; Gavai, A.; Liu, N.; van den Heuvel, L.J.; Marvin, H.J. Automatic classification of literature in systematic reviews on food safety using machine learning. Curr. Res. Food Sci. 2022, 5, 84–95. [Google Scholar] [CrossRef] [PubMed]
- Torii, M.; Liu, H. Classifier Ensemble for Biomedical Document Retrieval. In LBM (Short Papers); CEUR-WS: Washington, DC, USA, 2007; pp. 5.1–5.17. [Google Scholar]
- Qin, X.; Liu, J.; Wang, Y.; Liu, Y.; Deng, K.; Ma, Y.; Zou, K.; Li, L.; Sun, X. Natural language processing was effective in assisting rapid title and abstract screening when updating systematic reviews. J. Clin. Epidemiol. 2021, 133, 121–129. [Google Scholar] [CrossRef]
- Tsubota, T.; Bollegala, D.; Zhao, Y.; Jin, Y.; Kozu, T. Improvement of intervention information detection for automated clinical literature screening during systematic review. J. Biomed. Inform. 2022, 134, 104185. [Google Scholar] [CrossRef] [PubMed]
- Van De Schoot, R.; De Bruin, J.; Schram, R.; Zahedi, P.; De Boer, J.; Weijdema, F.; Kramer, B.; Huijts, M.; Hoogerwerf, M.; Ferdinands, G.; et al. An open source machine learning framework for efficient and transparent systematic reviews. Nat. Mach. Intell. 2021, 3, 125–133. [Google Scholar] [CrossRef]
- Ding, Q.; Ding, D.; Wang, Y.; Guan, C.; Ding, B. Unraveling the landscape of large language models: A systematic review and future perspectives. J. Electron. Bus. Digit. Econ. 2023, 3, 3–19. [Google Scholar] [CrossRef]
- Guizzardi, S.; Colangelo, M.T.; Mirandola, P.; Galli, C. Modeling new trends in bone regeneration, using the BERTopic approach. Regen. Med. 2023, 18, 719–734. [Google Scholar] [CrossRef]
- Chen, W.; Rabhi, F.; Liao, W.; Al-Qudah, I. Leveraging State-of-the-Art Topic Modeling for News Impact Analysis on Financial Markets: A Comparative Study. Electronics 2023, 12, 2605. [Google Scholar] [CrossRef]
- Wang, Z.; Chen, J.; Chen, J.; Chen, H. Identifying interdisciplinary topics and their evolution based on BERTopic. In Scientometrics; Springer: Berlin/Heidelberg, Germany, 2023; pp. 1–26. [Google Scholar]
- Gan, L.; Yang, T.; Huang, Y.; Yang, B.; Luo, Y.Y.; Richard, L.W.C.; Guo, D. Experimental Comparison of Three Topic Modeling Methods with LDA, Top2Vec and BERTopic. In Artificial Intelligence and Robotics, Proceedings of the 8th International Symposium, ISAIR 2023, Beijing, China, 21–23 October 2023; Springer: Singapore, 2023; pp. 376–391. [Google Scholar]
- Xiao, S.; Liu, Z.; Zhang, P.; Muennighoff, N. C-Pack: Packaged Resources to Advance General Chinese Embedding. arXiv 2023, arXiv:2309.07597v2. [Google Scholar]
- Kim, D.; Park, C.; Kim, S.; Lee, W.; Song, W.; Kim, Y.; Kim, H.; Kim, Y.; Lee, H.; Kim, J.; et al. SOLAR 10.7 B: Scaling Large Language Models with Simple yet Effective Depth Up-Scaling. arXiv 2023, arXiv:2312.15166. [Google Scholar]
- Mikolov, T.; Chen, K.; Corrado, G.; Dean, J. Efficient Estimation of Word Representations in Vector Space. In Proceedings of the Workshop at ICLR, Scottsdale, AZ, USA, 2–4 May 2013. [Google Scholar]
- Pennington, J.; Socher, R.; Manning, C.D. Glove: Global Vectors for Word Representation. In Proceedings of the EMNLP, Doha, Qatar, 25–29 October 2014; Volume 14, pp. 1532–1543. [Google Scholar]
- Muennighoff, N.; Tazi, N.; Magne, L.; Reimers, N. MTEB: Massive Text Embedding Benchmark. arXiv 2022, arXiv:2210.07316. [Google Scholar] [CrossRef]
- Muennighoff, N. SGPT: GPT Sentence Embeddings for Semantic Search. arXiv 2022, arXiv:2202.08904. [Google Scholar]
- McInnes, L.; Healy, J. UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction. J. Open Source Softw. 2018, 3, 861. [Google Scholar] [CrossRef]
- Van der Maaten, L.; Hinton, G. Visualizing data using t-SNE. J. Mach. Learn. Res. 2008, 9, 2579–2605. [Google Scholar]
- Tang, J.; Liu, J.; Zhang, M.; Mei, Q. Visualizing Large-scale and High-dimensional Data. In WWW ’16, Proceedings of the 25th International Conference on World Wide Web, Montréal, QC, Canada, 11–15 April 2016; International World Wide Web Conferences Steering Committee: Geneva, Switzerland, 2016. [Google Scholar] [CrossRef]
- Campello, R.; Moulavi, D.; Sander, J. Density-Based Clustering Based on Hierarchical Density Estimates. In Advances in Knowledge Discovery and Data Mining, Proceedings of the 17th Pacific-Asia Conference, PAKDD 2013, Gold Coast, Australia, 14–17 April 2013; Proceedings, Part II; Springer: Berlin/Heidelberg, Germany, 2013; Volume 7819, pp. 160–172. [Google Scholar] [CrossRef]
- Allaoui, M.; Kherfi, M.L.; Cheriet, A. Considerably Improving Clustering Algorithms Using UMAP Dimensionality Reduction Technique: A Comparative Study; Springer: Berlin/Heidelberg, Germany, 2020; pp. 317–325. [Google Scholar]
- García, J.; Leiva-Araos, A.; Diaz-Saavedra, E.; Moraga, P.; Pinto, H.; Yepes, V. Relevance of Machine Learning Techniques in Water Infrastructure Integrity and Quality: A Review Powered by Natural Language Processing. Appl. Sci. 2023, 13, 12497. [Google Scholar] [CrossRef]
- Asyaky, M.S.; Mandala, R. Improving the Performance of HDBSCAN on Short Text Clustering by Using Word Embedding and UMAP. In Proceedings of the 2021 8th International Conference on Advanced Informatics: Concepts, Theory and Applications (ICAICTA), Bandung, Indonesia, 29–30 September 2021; pp. 1–6. [Google Scholar] [CrossRef]
- Bojanowski, P.; Grave, E.; Joulin, A.; Mikolov, T. Enriching Word Vectors with Subword Information. Trans. Assoc. Comput. Linguist. 2016, 5, 135–146. [Google Scholar] [CrossRef]
- Färber, M.; Steyer, A. Towards Full-Fledged Argument Search: A Framework for Extracting and Clustering Arguments from Unstructured Text. arXiv 2021, arXiv:2112.00160. [Google Scholar] [CrossRef]
- David, U.; Karabatak, M. Text Clustering of COVID-19 Vaccine Tweets. In Proceedings of the 2022 10th International Symposium on Digital Forensics and Security (ISDFS), Istanbul, Turkey, 6–7 June 2022; pp. 1–6. [Google Scholar] [CrossRef]
- Gelar, T.; Sari, A.N. Bertopic and NER Stop Words for Topic Modeling on Agricultural Instructional Sentences. In Proceedings of the International Conference on Applied Science and Technology on Engineering Science 2023 (iCAST-ES 2023), Tarakan, Indonesia, 20–22 October 2024; Atlantis Press: Paris, France, 2024; pp. 129–140. [Google Scholar] [CrossRef]
- Dettmers, T.; Pagnoni, A.; Holtzman, A.; Zettlemoyer, L. QLoRA: Efficient Finetuning of Quantized LLMs. arXiv 2023, arXiv:2305.14314. [Google Scholar]
- Brown, T.B.; Mann, B.; Ryder, N.; Subbiah, M.; Kaplan, J.; Dhariwal, P.; Neelakantan, A.; Shyam, P.; Sastry, G.; Askell, A.; et al. Language Models are Few-Shot Learners. arXiv 2020, arXiv:2005.14165. [Google Scholar] [CrossRef]
- Grootendorst, M. KeyBERT: Minimal Keyword Extraction with BERT. 2020. Available online: https://zenodo.org/records/8388690 (accessed on 28 November 2023).
- Landis, J.R.; Koch, G.G. The measurement of observer agreement for categorical data. Biometrics 1977, 33, 159–174. [Google Scholar] [CrossRef] [PubMed]
- Gana Castillo, B.P. Leveraging LLMs for Efficient Topic Reviews. 2024. Available online: https://zenodo.org/records/13346401 (accessed on 19 August 2024).
- Gana Castillo, B. Topic-Modeling-BERTopic-SOLAR. 2024. Available online: https://github.com/Zickbad/Topic-modeling-BERTopic-SOLAR (accessed on 19 August 2024).
Topic | Year | Scope | SLR Focus Task | Citation |
---|---|---|---|---|
Classic machine learning algorithms in text mining for abstract screening | 2007–2022 | Explores the use of SVMs and ensemble methods in distinguishing relevant citations, with a focus on overcoming class imbalance. | Select relevant studies, Assess study quality | [29,30] |
Hybrid Feature Selection with supervised learning for automatic ranking of articles | 2016–2018 | Discusses the application of HFSRM and logistic regression classifiers in DTA reviews, significantly reducing screening time and effort. | Assess study quality, Analyze and interpret data | [26,27,28] |
Adherence of search methods to SLR frameworks | 2021 | Examines the quality and nature of search methods in Campbell systematic reviews, emphasizing adherence to MECCIR and PRISMA guidelines. | Develop review protocol, Conduct exhaustive literature search | [24] |
ML-Aided pipeline for screening in systematic reviews | 2021 | Proposes a machine learning-aided pipeline, ASReview, to significantly improve the efficiency and quality of systematic reviews and meta-analyses. | Select relevant studies, Extract data from selected studies | [33] |
Systematic reviews automation based on query log analysis | 2022 | Analyzes query logs from a specialized tool to reveal intuitive user behaviors in query formulation for medical systematic reviews. | Conduct exhaustive literature search | [21] |
Automatic query generation using pre-trained language models | 2023 | Discusses an automatic query generation approach using pre-trained language models, significantly surpassing traditional models in precision, recall, and F-measures. | Conduct exhaustive literature search | [20] |
Utility of Text-Mining Tools | 2023 | Assesses the utility of semi-automated text-mining tools in systematic reviews of diagnostic test accuracy, yielding additional relevant articles with varied precision. | Select relevant studies | [22] |
MeSH Term Identification | 2023 | Focuses on using MeSH term identification, highlighting the strategy’s efficiency. | Conduct exhaustive literature search | [23] |
BERTopic applications in various domains | 2023 | BERTopic has been used across multiple domains including LLM research, medical research, financial sector, and interdisciplinary research, showing superiority in topic clustering, analysis, and identifying key research trends across diverse datasets. | Analyze and interpret data | [34,35,36,37,38] |
Our work | 2024 | Proposes an advanced systematic review process incorporating AI techniques for improved accuracy, speed, and comprehensiveness in literature reviews. | Semi-automatic exhaustive literature search and screening framework |
Topic | Custom Name | Count |
---|---|---|
6 | Automated Methods for Literature Screening in Medical Systematic Reviews | 399 |
29 | Biomedical Text Mining for Gene and Protein Extraction | 131 |
54 | Clinical Natural Language Processing for Medical Text Analysis | 67 |
Index Keywords | Author Keywords | |||
---|---|---|---|---|
KeyBERT (6) |
Mean of Selected Topics | KeyBERT (6) |
Mean of Selected Topics | |
Min | 0.4956 | 0.5333 | 0.3577 | 0.3071 |
Max | 0.8149 | 0.6940 | 0.8600 | 0.6847 |
Mean | 0.6956 | 0.6323 | 0.6971 | 0.5958 |
Median | 0.7007 | 0.6309 | 0.7064 | 0.6019 |
Std Dev | 0.0531 | 0.0245 | 0.0717 | 0.0432 |
Shapiro-Wilk p-value | 0.1909 | 0.2110 | 0.4551 | 0.2572 |
t-Test Paired p-value | 0.0135 | 0.0224 |
Context Parameter | Average Score | Fleiss’ | |||
---|---|---|---|---|---|
Meaningfulness | 2.423 | 0.65 | 0.42 | 0.54 | 0.0402 |
Importance | 2.355 | 0.68 | 0.40 | 0.44 | 0.0650 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Gana, B.; Leiva-Araos, A.; Allende-Cid, H.; García, J. Leveraging LLMs for Efficient Topic Reviews. Appl. Sci. 2024, 14, 7675. https://doi.org/10.3390/app14177675
Gana B, Leiva-Araos A, Allende-Cid H, García J. Leveraging LLMs for Efficient Topic Reviews. Applied Sciences. 2024; 14(17):7675. https://doi.org/10.3390/app14177675
Chicago/Turabian StyleGana, Bady, Andrés Leiva-Araos, Héctor Allende-Cid, and José García. 2024. "Leveraging LLMs for Efficient Topic Reviews" Applied Sciences 14, no. 17: 7675. https://doi.org/10.3390/app14177675