Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
5.3
2.5
Journals
Applied Sciences
Special Issues
Advanced Large Language Models and Natural Language Processing Applications
applsci-logo
Submit to Special Issue Submit Abstract to Special Issue Review for Applied Sciences Propose a Special Issue

Journal Menu

► Journal Menu

Journal Browser

► Journal Browser
share announcement

Advanced Large Language Models and Natural Language Processing Applications

A special issue of Applied Sciences (ISSN 2076-3417). This special issue belongs to the section "Computing and Artificial Intelligence".

Deadline for manuscript submissions: 30 May 2025 | Viewed by 11896

Special Issue Editors

Dr. Man-Fai Leung

E-Mail Website
Guest Editor
School of Computing and Information Science, Faculty of Science and Engineering, Anglia Ruskin University, Cambridge, UK
Interests: intelligent systems; machine learning; biological system optimization
Special Issues, Collections and Topics in MDPI journals
Dr. Duo Li

E-Mail Website
Guest Editor
School of Engineering, Newcastle University, Newcastle NE1 7RU, UK
Interests: spatio-temporal computing; optimization and control; AI applications in transportation
Dr. Jin Zhang

E-Mail Website
Guest Editor
School of Computing and Information Science, Faculty of Science and Engineering, Anglia Ruskin University, Cambridge, UK
Interests: Internet of Things; electronic systems modelling; smart antennas; remote sensing; signal processing

Special Issue Information

Dear Colleagues,

The rapid growth of big data and the advancements in computational power have spurred remarkable progress in the field of natural language processing (NLP). Central to this progress are large language models (LLMs) like GPT, BERT, and T5, which have shown exceptional capabilities in understanding and generating human-like text. These models, a subset of artificial intelligence (AI), leverage deep learning techniques to build predictive models that can handle diverse NLP tasks such as language translation, summarization, and sentiment analysis. LLMs and NLP applications have revolutionized various industries, from automating customer support to enhancing content generation and improving healthcare analytics.

However, despite their impressive performance, challenges remain in the interpretability, scalability, and ethical implications of these models. This Special Issue aims to bring together researchers and practitioners to discuss and exchange the latest advancements in large language models and NLP applications. We welcome original research articles and comprehensive reviews that provide insights into this rapidly evolving field. Potential topics include, but are not limited to, the following: training and fine-tuning of large language models; efficient architectures and optimization techniques for LLMs; multilingual and cross-lingual LLMs; summarization, translation, and question-answering systems; zero-shot and few-shot learning in NLP; text generation and creative writing using LLMs; integration of LLMs with computer vision and multimodal learning; ethical, societal, and interpretability challenges of LLMs; robustness and security issues in large language models; and data privacy and bias mitigation in NLP applications.

Dr. Man-Fai Leung
Dr. Duo Li
Dr. Jin Zhang
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Applied Sciences is an international peer-reviewed open access semimonthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2400 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

  • NLP
  • large language models
  • text generation
  • sentiment analysis

Benefits of Publishing in a Special Issue

  • Ease of navigation: Grouping papers by topic helps scholars navigate broad scope journals more efficiently.
  • Greater discoverability: Special Issues support the reach and impact of scientific research. Articles in Special Issues are more discoverable and cited more frequently.
  • Expansion of research network: Special Issues facilitate connections among authors, fostering scientific collaborations.
  • External promotion: Articles in Special Issues are often promoted through the journal's social media, increasing their visibility.
  • e-Book format: Special Issues with more than 10 articles can be published as dedicated e-books, ensuring wide and rapid dissemination.

Further information on MDPI's Special Issue policies can be found here.

Published Papers (6 papers)

Download All Papers
Order results
Result details
Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:

Research

18 pages, 1183 KiB  
Article
Increasing the Accessibility of Causal Domain Knowledge via Causal Information Extraction Methods: A Case Study in the Semiconductor Manufacturing Industry
by Houssam Razouk, Leonie Benischke, Daniel Gärber and Roman Kern
Appl. Sci. 2025, 15(5), 2573; https://doi.org/10.3390/app15052573 - 27 Feb 2025
Viewed by 426
Abstract
Causal domain knowledge is commonly documented using natural language either in unstructured or semi-structured forms. This study aims to increase the usability of causal domain knowledge in industrial documents by transforming the information into a more structured format. The paper presents our work [...] Read more.
Causal domain knowledge is commonly documented using natural language either in unstructured or semi-structured forms. This study aims to increase the usability of causal domain knowledge in industrial documents by transforming the information into a more structured format. The paper presents our work on developing automated methods for causal information extraction from real-world industrial documents in the semiconductor manufacturing industry, including presentation slides and FMEA (Failure Mode and Effects Analysis) documents. Specifically, we evaluate two types of causal information extraction methods: single-stage sequence tagging (SST) and multi-stage sequence tagging (MST). The presented case study showcases that the proposed MST methods for extracting causal information from industrial documents are suitable for practical applications, especially for semi-structured documents such as FMEAs, with a 93% F1 score. Additionally, the study shows that extracting causal information from presentation slides is more challenging. The study highlights the importance of choosing a language model that is more aligned with the domain and in-domain pre-training. Full article
(This article belongs to the Special Issue Advanced Large Language Models and Natural Language Processing Applications)
►▼ Show Figures

Figure 1

34 pages, 315 KiB  
Article
Optimizing Large Language Models: A Deep Dive into Effective Prompt Engineering Techniques
by Minjun Son, Yun-Jae Won and Sungjin Lee
Appl. Sci. 2025, 15(3), 1430; https://doi.org/10.3390/app15031430 - 30 Jan 2025
Cited by 2 | Viewed by 2793
Abstract
Recent advancements in Natural Language Processing (NLP) technologies have been driven at an unprecedented pace by the development of Large Language Models (LLMs). However, challenges remain, such as generating responses that are misaligned with the intent of the question or producing incorrect answers. [...] Read more.
Recent advancements in Natural Language Processing (NLP) technologies have been driven at an unprecedented pace by the development of Large Language Models (LLMs). However, challenges remain, such as generating responses that are misaligned with the intent of the question or producing incorrect answers. This paper analyzes various Prompt Engineering techniques for large-scale language models and identifies methods that can optimize response performance across different datasets without the need for extensive retraining or fine-tuning. In particular, we examine prominent Prompt Engineering techniques including In-Context Learning (ICL), Chain of Thought (CoT), Retrieval-Augmented Generation (RAG), Step-by-Step Reasoning (SSR), and Tree of Thought (ToT), and we apply these techniques to leading LLMs such as Gemma2, LlaMA3, and Mistral. The performance of these models was evaluated using the AI2 Reasoning Challenge (ARC), HellaSwag, Massive Multitask Language Understanding (MMLU), TruthfulQA, Winogrande, and Grade School Math (GSM8k) datasets across metrics such as BLEU, ROUGE, METEOR, BLEURT, and BERTScore. The experimental results indicate that the most suitable Prompt Engineering technique can vary depending on the characteristics of each dataset. Specifically, for datasets emphasizing mathematical and logical reasoning, Prompt Engineering strategies centered around CoT, SSR, and ToT were found to be advantageous. For datasets focusing on natural language understanding, ICL-centric strategies were more effective, while RAG-based strategies were beneficial for datasets where factual accuracy is crucial. However, it was also observed that the optimal combination of Prompt Engineering techniques could differ depending on the specific LLM, indicating that fine-tuning the Prompt Engineering approach to the model and dataset is essential for achieving the best performance. The findings indicate that as LLMs become more advanced, their reliance on Prompt Engineering (PE) techniques diminishes, yet the magnitude of their performance improvement when PE strategies are applied increases. Furthermore, these advanced models tend to depend less on ICL techniques while exhibiting a greater reliance on RAG strategies. It is also evident that implementing RAG with PE-based preprocessing yields superior performance enhancements compared to the mere application of RAG on raw data. Full article
(This article belongs to the Special Issue Advanced Large Language Models and Natural Language Processing Applications)
16 pages, 397 KiB  
Article
Efficient Fine-Tuning of Large Language Models via a Low-Rank Gradient Estimator
by Luoming Zhang, Zhenyu Lou, Yangwei Ying, Cheng Yang and Hong Zhou
Appl. Sci. 2025, 15(1), 82; https://doi.org/10.3390/app15010082 - 26 Dec 2024
Viewed by 1472
Abstract
In this paper, we present a Low-Rank Gradient Estimator (LoGE) to accelerate the finetune-time computation of transformers, especially large language models (LLMs). Unlike Parameter-Efficient Fine-Tuning (PEFT) methods, which primarily aim to minimize the number of fine-tuning parameters, LoGE also significantly reduces the computational [...] Read more.
In this paper, we present a Low-Rank Gradient Estimator (LoGE) to accelerate the finetune-time computation of transformers, especially large language models (LLMs). Unlike Parameter-Efficient Fine-Tuning (PEFT) methods, which primarily aim to minimize the number of fine-tuning parameters, LoGE also significantly reduces the computational load of activation gradient calculations by decomposing pre-trained weights and utilizing low-rank matrices during the backward pass. Our approach includes an effective solution for identifying sensitive and important latent subspaces in large models before training with downstream datasets. As LoGE does not alter the network structure, it can be conveniently integrated into existing models. We validated LoGE’s efficacy through comprehensive experiments across various models on various tasks. For the widely used LLaMA model equipped with LoRA, LoGE achieves up to a 1.3× speedup while maintaining graceful accuracy. Full article
(This article belongs to the Special Issue Advanced Large Language Models and Natural Language Processing Applications)
►▼ Show Figures

Figure 1

34 pages, 1427 KiB  
Article
The Impact of Prompting Techniques on the Security of the LLMs and the Systems to Which They Belong
by Teodor Ivănușcă and Cosmin-Iulian Irimia
Appl. Sci. 2024, 14(19), 8711; https://doi.org/10.3390/app14198711 - 26 Sep 2024
Viewed by 1833
Abstract
Large language models have demonstrated impressive capabilities. The recent research conducted in the field of prompt engineering showed that their base performance is just a glimpse of their full abilities. Enhanced with auxiliary tools and provided with examples of how to solve the [...] Read more.
Large language models have demonstrated impressive capabilities. The recent research conducted in the field of prompt engineering showed that their base performance is just a glimpse of their full abilities. Enhanced with auxiliary tools and provided with examples of how to solve the tasks, their adoption into our applications seems trivial. In this context, we ask an uncomfortable question. Are the models secure enough to be adopted in our systems, or do they represent Trojan horses? The idea of prompt injection and jailbreak attacks does not seem to bother the adopters too much. Even though there are a lot of studies that look into the benefits of the prompting techniques, none address their possible downside in regard to the security. We want take a step further and investigate the impact of the most popular prompting techniques on this aspect of large language models and implicitly the systems to which they belong. Using three of the most deployed GPT models to date, we conducted a few of the most popular attacks in different setup scenarios and demonstrate that prompting techniques can have a negative impact on the security of the LLMs. More than that, they also expose other system components that otherwise would have been less exposed. In the end, we try to come up with possible solutions and present future research perspectives. Full article
(This article belongs to the Special Issue Advanced Large Language Models and Natural Language Processing Applications)
►▼ Show Figures

Figure 1

15 pages, 3035 KiB  
Article
A Framework for Agricultural Intelligent Analysis Based on a Visual Language Large Model
by Piaofang Yu and Bo Lin
Appl. Sci. 2024, 14(18), 8350; https://doi.org/10.3390/app14188350 - 17 Sep 2024
Cited by 2 | Viewed by 1430
Abstract
Smart agriculture has become an inevitable trend in the development of modern agriculture, especially promoted by the continuous progress of large language models like chat generative pre-trained transformer (ChatGPT) and general language model (ChatGLM). Although these large models perform well in general knowledge [...] Read more.
Smart agriculture has become an inevitable trend in the development of modern agriculture, especially promoted by the continuous progress of large language models like chat generative pre-trained transformer (ChatGPT) and general language model (ChatGLM). Although these large models perform well in general knowledge learning, they still have certain limitations and errors when facing agricultural professional knowledge about crop disease identification, growth stage judgment, and so on. Agricultural data involves images and texts and other modalities, which play an important role in agricultural production and management. In order to better learn the characteristics of different modal data in agriculture, realize cross-modal data fusion, and thus understand complex application scenarios, we propose a framework AgriVLM that uses a large amount of agricultural data to fine-tune the visual language model to analyze agricultural data. It can fuse multimodal data and provide more comprehensive agricultural decision support. Specifically, it utilizes Q-former as a bridge between an image encoder and a language model to achieve a cross-modal fusion of agricultural images and text data. Then, we apply a Low-Rank adaptive to fine-tune the language model to achieve an alignment between agricultural image features and a pre-trained language model. The experimental results prove that AgriVLM demonstrates great performance in crop disease recognition and growth stage recognition, with recognition accuracy exceeding 90%, demonstrating its capability to analyze different modalities of agricultural data. Full article
(This article belongs to the Special Issue Advanced Large Language Models and Natural Language Processing Applications)
►▼ Show Figures

Figure 1

32 pages, 4054 KiB  
Article
HELIOS Approach: Utilizing AI and LLM for Enhanced Homogeneity Identification in Real Estate Market Analysis
by Artur Janowski and Malgorzata Renigier-Bilozor
Appl. Sci. 2024, 14(14), 6135; https://doi.org/10.3390/app14146135 - 15 Jul 2024
Cited by 1 | Viewed by 2516
Abstract
The concept of homogeneity in the real estate market is a well-known analysis aspect, yet it remains a significant challenge in practical implementation. This study aims to fill this research gap by introducing the HELIOS concept (Homogeneity Estate Linguistic Intelligence Omniscient Support), presenting [...] Read more.
The concept of homogeneity in the real estate market is a well-known analysis aspect, yet it remains a significant challenge in practical implementation. This study aims to fill this research gap by introducing the HELIOS concept (Homogeneity Estate Linguistic Intelligence Omniscient Support), presenting a new approach to real estate market analyses. In a world increasingly mindful of environmental, social, and economic concerns, HELIOS is a novel concept grounded in linguistic intelligence and machine learning to reshape how we perceive and analyze real estate data. By exploring the synergies between human expertise and technological capabilities, HELIOS aims not only to enhance the efficiency of real estate analyses but also to contribute to the broader goal of sustainable and responsible data practices in the dynamic landscape of property markets. Additionally, the article formulates a set of assumptions and suggestions to improve the effectiveness and efficiency of homogeneity analysis in mass valuation, emphasizing the synergy between human knowledge and the potential of machine technology. Full article
(This article belongs to the Special Issue Advanced Large Language Models and Natural Language Processing Applications)
►▼ Show Figures

Figure 1

Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:
 
Displaying articles 1-6
Back to TopTop