Applied Sciences

Research

19 pages, 804 KiB

Open AccessArticle

Exploring the Feasibility of Generative AI in Persona Research: A Comparative Analysis of Large Language Model-Generated and Human-Crafted Personas in Obesity Research

by Urška Smrke, Ana Rehberger, Nejc Plohl and Izidor Mlakar

Appl. Sci. 2025, 15(4), 1937; https://doi.org/10.3390/app15041937 - 13 Feb 2025

Viewed by 522

Abstract

This study investigates the perceptions of Persona descriptions generated using three different large language models (LLMs) and qualitatively developed Personas by an expert panel involved in obesity research. Six different Personas were defined, three from the clinical domain and three from the educational [...] Read more.

This study investigates the perceptions of Persona descriptions generated using three different large language models (LLMs) and qualitatively developed Personas by an expert panel involved in obesity research. Six different Personas were defined, three from the clinical domain and three from the educational domain. The descriptions of Personas were generated using qualitative methods and the LLMs (i.e., Bard, Llama, and ChatGPT). The perception of the developed Personas was evaluated by experts in the respective fields. The results show that, in general, the perception of Personas did not significantly differ between those generated using LLMs and those qualitatively developed by human experts. This indicates that LLMs have the potential to generate a consistent and valid representation of human stakeholders. The LLM-generated Personas were perceived as believable, relatable, and informative. However, post-hoc comparisons revealed some differences, with descriptions generated using the Bard model being in several Persona descriptions that were evaluated most favorably in terms of empathy, likability, and clarity. This study contributes to the understanding of the potential and challenges of LLM-generated Personas. Although the study focuses on obesity research, it highlights the importance of considering the specific context and the potential issues that researchers should be aware of when using generative AI for generating Personas. Full article

(This article belongs to the Special Issue Advances in Large Language Models: Techniques, Applications and Challenges)

► Show Figures

Figure 1

58 pages, 1548 KiB

Open AccessArticle

Large Language Models in Computer Science Classrooms: Ethical Challenges and Strategic Solutions

by Rina Azoulay, Tirza Hirst and Shulamit Reches

Appl. Sci. 2025, 15(4), 1793; https://doi.org/10.3390/app15041793 - 10 Feb 2025

Viewed by 841

Abstract

The integration of large language models (LLMs) into educational settings represents a significant technological breakthrough, offering substantial opportunities alongside profound ethical challenges. Higher education institutions face the widespread use of these tools by students, requiring them to navigate complex decisions regarding their adoption. [...] Read more.

The integration of large language models (LLMs) into educational settings represents a significant technological breakthrough, offering substantial opportunities alongside profound ethical challenges. Higher education institutions face the widespread use of these tools by students, requiring them to navigate complex decisions regarding their adoption. This includes determining whether to allow the use of LLMs, defining their appropriate scope, and establishing guidelines for their responsible and ethical application. In the context of computer science education, these challenges are particularly acute. On the one hand, the capabilities of LLMs significantly enhance the tools available to developers and software engineers. On the other hand, students’ over-reliance on LLMs risks hindering their development of foundational skills. This study examines these challenges and proposes strategies to regulate the use of LLMs while upholding academic integrity. It focuses on the specific impact of LLMs in programming education, where dependence on AI-generated solutions may erode active learning and essential skill acquisition. Through a comprehensive literature review and drawing on teaching experience and guidelines from global institutions, this study contributes to the broader discourse on the integration of these advanced technologies into educational environments. The goal is to enhance learning outcomes while ensuring the development of competent, ethical software professionals. Full article

(This article belongs to the Special Issue Advances in Large Language Models: Techniques, Applications and Challenges)

► Show Figures

Figure 1

31 pages, 1741 KiB

Open AccessArticle

Context Is King: Large Language Models’ Interpretability in Divergent Knowledge Scenarios

by Andrés Piñeiro-Martín, Francisco-Javier Santos-Criado, Carmen García-Mateo, Laura Docío-Fernández and María del Carmen López-Pérez

Appl. Sci. 2025, 15(3), 1192; https://doi.org/10.3390/app15031192 - 24 Jan 2025

Viewed by 948

Abstract

Large language models (LLMs) have revolutionized the field of artificial intelligence in both academia and industry, transforming how we communicate, search for information, and create content. However, these models face knowledge cutoffs and costly updates, driving a new ecosystem for LLM-based applications that [...] Read more.

Large language models (LLMs) have revolutionized the field of artificial intelligence in both academia and industry, transforming how we communicate, search for information, and create content. However, these models face knowledge cutoffs and costly updates, driving a new ecosystem for LLM-based applications that leverage interaction techniques to extend capabilities and facilitate knowledge updates. As these models grow more complex, understanding their internal workings becomes increasingly challenging, posing significant issues for transparency, interpretability, and explainability. This paper proposes a novel approach to interpretability by shifting the focus to understanding the model’s functionality within specific contexts through interaction techniques. Rather than dissecting the LLM itself, we explore how contextual information and interaction techniques can elucidate the model’s thought processes. To this end, we introduce the Context-Driven Divergent Knowledge Evaluation (CDK-E) methodology, along with the Divergent Knowledge Dataset (DKD), for evaluating the interpretability of LLMs in context-specific scenarios that diverge from the model’s inherent knowledge. The empirical results demonstrate that advanced LLMs achieve high alignment with divergent contexts, validating our hypothesis that contextual information significantly enhances interpretability. Moreover, the strong correlation between LLM-based metrics and semantic metrics confirms the reliability of our evaluation framework. Full article

(This article belongs to the Special Issue Advances in Large Language Models: Techniques, Applications and Challenges)

► Show Figures

Figure 1

22 pages, 2909 KiB

Open AccessArticle

Research and Application of a Multi-Agent-Based Intelligent Mine Gas State Decision-Making System

by Yi Sun and Xinke Liu

Appl. Sci. 2025, 15(2), 968; https://doi.org/10.3390/app15020968 - 20 Jan 2025

Viewed by 1102

Abstract

To address the issues of low efficiency in manual processing and lack of accuracy in judgment within traditional mine gas safety inspections, this paper designs and implements the Intelligent Mine Gas State Decision-Making System based on large language models (LLMs) and a multi-agent [...] Read more.

To address the issues of low efficiency in manual processing and lack of accuracy in judgment within traditional mine gas safety inspections, this paper designs and implements the Intelligent Mine Gas State Decision-Making System based on large language models (LLMs) and a multi-agent system. The system aims to enhance the accuracy of gas over-limit alarms and improve the efficiency of generating judgment reports. The system integrates the reasoning capabilities of LLMs and optimizes task allocation and execution efficiency of agents through the study of the hybrid multi-agent orchestration algorithm. Furthermore, the system establishes a comprehensive gas risk assessment knowledge base, encompassing historical alarm data, real-time monitoring data, alarm judgment criteria, treatment methods, and relevant policies and regulations. Additionally, the system incorporates several technologies, including retrieval-augmented generation based on human feedback mechanisms, tool management, prompt engineering, and asynchronous processing, which further enhance the application performance of the LLM in the gas status judgment system. Experimental results indicate that the system effectively improves the efficiency of gas alarm processing and the quality of judgment reports in coal mines, providing solid technical support for accident prevention and management in mining operations. Full article

(This article belongs to the Special Issue Advances in Large Language Models: Techniques, Applications and Challenges)

► Show Figures

Figure 1

13 pages, 1171 KiB

Open AccessArticle

Bayesian Optimization for Instruction Generation

by Antonio Sabbatella, Francesco Archetti, Andrea Ponti, Ilaria Giordani and Antonio Candelieri

Appl. Sci. 2024, 14(24), 11865; https://doi.org/10.3390/app142411865 - 19 Dec 2024

Viewed by 735

Abstract

The performance of Large Language Models (LLMs) strongly depends on the selection of the best instructions for different downstream tasks, especially in the case of black-box LLMs. This study introduces BOInG (Bayesian Optimization for Instruction Generation), a method leveraging Bayesian Optimization (BO) to [...] Read more.

The performance of Large Language Models (LLMs) strongly depends on the selection of the best instructions for different downstream tasks, especially in the case of black-box LLMs. This study introduces BOInG (Bayesian Optimization for Instruction Generation), a method leveraging Bayesian Optimization (BO) to efficiently generate instructions while addressing the combinatorial nature of instruction search. Over the last decade, BO has emerged as a highly effective optimization method in various domains due to its flexibility and sample efficiency. At its core, BOInG employs Bayesian search in a low-dimensional continuous space, projecting solutions into a high-dimensional token embedding space to retrieve discrete tokens. These tokens act as seeds for the generation of human-readable, task-relevant instructions. Experimental results demonstrate that BOInG achieves comparable or superior performance to state-of-the-art methods, such as InstructZero and Instinct, with substantially lower resource requirements while also enabling the use of both white-box and black-box models. This approach offers both theoretical and practical benefits without requiring specialized hardware. Full article

(This article belongs to the Special Issue Advances in Large Language Models: Techniques, Applications and Challenges)

► Show Figures

Figure 1

66 pages, 1555 KiB

Open AccessArticle

Extracting Sentence Embeddings from Pretrained Transformer Models

by Lukas Stankevičius and Mantas Lukoševičius

Appl. Sci. 2024, 14(19), 8887; https://doi.org/10.3390/app14198887 - 2 Oct 2024

Cited by 2 | Viewed by 2118

Abstract

Pre-trained transformer models shine in many natural language processing tasks and therefore are expected to bear the representation of the input sentence or text meaning. These sentence-level embeddings are also important in retrieval-augmented generation. But do commonly used plain averaging or prompt templates [...] Read more.

Pre-trained transformer models shine in many natural language processing tasks and therefore are expected to bear the representation of the input sentence or text meaning. These sentence-level embeddings are also important in retrieval-augmented generation. But do commonly used plain averaging or prompt templates sufficiently capture and represent the underlying meaning? After providing a comprehensive review of existing sentence embedding extraction and refinement methods, we thoroughly test different combinations and our original extensions of the most promising ones on pretrained models. Namely, given 110 M parameters, BERT’s hidden representations from multiple layers, and many tokens, we try diverse ways to extract optimal sentence embeddings. We test various token aggregation and representation post-processing techniques. We also test multiple ways of using a general Wikitext dataset to complement BERT’s sentence embeddings. All methods are tested on eight Semantic Textual Similarity (STS), six short text clustering, and twelve classification tasks. We also evaluate our representation-shaping techniques on other static models, including random token representations. Proposed representation extraction methods improve the performance on STS and clustering tasks for all models considered. Very high improvements for static token-based models, especially random embeddings for STS tasks, almost reach the performance of BERT-derived representations. Our work shows that the representation-shaping techniques significantly improve sentence embeddings extracted from BERT-based and simple baseline models. Full article

(This article belongs to the Special Issue Advances in Large Language Models: Techniques, Applications and Challenges)

► Show Figures

Figure 1

28 pages, 6741 KiB

Open AccessArticle

WorkloadGPT: A Large Language Model Approach to Real-Time Detection of Pilot Workload

by Yijing Gao, Lishengsa Yue, Jiahang Sun, Xiaonian Shan, Yihan Liu and Xuerui Wu

Appl. Sci. 2024, 14(18), 8274; https://doi.org/10.3390/app14188274 - 13 Sep 2024

Viewed by 1398

Abstract

The occurrence of flight risks and accidents is closely related to pilot workload. Effective detection of pilot workload has been a key research area in the aviation industry. However, traditional methods for detecting pilot workload have several shortcomings: firstly, the collection of metrics [...] Read more.

The occurrence of flight risks and accidents is closely related to pilot workload. Effective detection of pilot workload has been a key research area in the aviation industry. However, traditional methods for detecting pilot workload have several shortcomings: firstly, the collection of metrics via contact-based devices can interfere with pilots; secondly, real-time detection of pilot workload is challenging, making it difficult to capture sudden increases in workload; thirdly, the detection accuracy of these models is limited; fourthly, the models lack cross-pilot generalization. To address these challenges, this study proposes a large language model, WorkloadGPT, which utilizes low-interference indicators: eye movement and seat pressure. Specifically, features are extracted in 10 s time windows and input into WorkloadGPT for classification into low, medium, and high workload categories. Additionally, this article presents the design of an appropriate text template to serialize the tabular feature dataset into natural language, incorporating individual difference prompts during instance construction to enhance cross-pilot generalization. Finally, the LoRA algorithm was used to fine-tune the pre-trained large language model ChatGLM3-6B, resulting in WorkloadGPT. During the training process of WorkloadGPT, the GAN-Ensemble algorithm was employed to augment the experimental raw data, constructing a realistic and robust extended dataset for model training. The results show that WorkloadGPT achieved a classification accuracy of 87.3%, with a cross-pilot standard deviation of only 2.1% and a response time of just 1.76 s, overall outperforming existing studies in terms of accuracy, real-time performance, and cross-pilot generalization capability, thereby providing a solid foundation for enhancing flight safety. Full article

(This article belongs to the Special Issue Advances in Large Language Models: Techniques, Applications and Challenges)

► Show Figures

Figure 1

18 pages, 590 KiB

Open AccessArticle

Open Sesame! Universal Black-Box Jailbreaking of Large Language Models

by Raz Lapid, Ron Langberg and Moshe Sipper

Appl. Sci. 2024, 14(16), 7150; https://doi.org/10.3390/app14167150 - 14 Aug 2024

Cited by 10 | Viewed by 2810

Abstract

Large language models (LLMs), designed to provide helpful and safe responses, often rely on alignment techniques to align with user intent and social guidelines. Unfortunately, this alignment can be exploited by malicious actors seeking to manipulate an LLM’s outputs for unintended purposes. In [...] Read more.

Large language models (LLMs), designed to provide helpful and safe responses, often rely on alignment techniques to align with user intent and social guidelines. Unfortunately, this alignment can be exploited by malicious actors seeking to manipulate an LLM’s outputs for unintended purposes. In this paper, we introduce a novel approach that employs a genetic algorithm (GA) to manipulate LLMs when model architecture and parameters are inaccessible. The GA attack works by optimizing a universal adversarial prompt that—when combined with a user’s query—disrupts the attacked model’s alignment, resulting in unintended and potentially harmful outputs. Our novel approach systematically reveals a model’s limitations and vulnerabilities by uncovering instances where its responses deviate from expected behavior. Through extensive experiments, we demonstrate the efficacy of our technique, thus contributing to the ongoing discussion on responsible AI development by providing a diagnostic tool for evaluating and enhancing alignment of LLMs with human intent. To our knowledge, this is the first automated universal black-box jailbreak attack. Full article

(This article belongs to the Special Issue Advances in Large Language Models: Techniques, Applications and Challenges)

► Show Figures

Figure 1

Journal Menu

Journal Browser

Advances in Large Language Models: Techniques, Applications and Challenges

Share This Special Issue

Special Issue Editor

Special Issue Information

Keywords

Benefits of Publishing in a Special Issue

Published Papers (8 papers)

Research

Further Information

Guidelines

MDPI Initiatives

Follow MDPI