Machine Learning Advances and Applications on Natural Language Processing (NLP)

A special issue of Electronics (ISSN 2079-9292). This special issue belongs to the section "Computer Science & Engineering".

Deadline for manuscript submissions: 31 December 2024 | Viewed by 9500

Special Issue Editors


E-Mail Website
Guest Editor
Department of Science and Technology, International Hellenic University, 57001 Thessaloniki, Greece
Interests: machine learning; deep learning; data mining; parallel algorithms; data structures; information retrieval
Special Issues, Collections and Topics in MDPI journals

E-Mail Website
Guest Editor
Department of Science and Technology, International Hellenic University, 57001 Thessaloniki, Greece
Interests: algorithms; data structures; data mining; machine learning
Special Issues, Collections and Topics in MDPI journals

Special Issue Information

Dear Colleagues,

The impressive achievements of recent state-of-the-art machine learning models are revolutionizing the area of text and language applications. On the one hand, the so-called Large Language Models (LLMs) have demonstrated an enormous capability in automatic and reasonable text generation. On the other hand, the Transformer-based models have drastically improved the performance of other, highly important natural language tools, including traditional sentiment analyzers, machine (auto) translators, and multimedia annotators. Furthermore, such Natural Language Processing (NLP) techniques are heavily utilized in social networks, microblogs, eCommerce systems, and numerous other disciplines.

The goal of this Special Issue is to publish high-quality, peer-reviewed original research papers in the area of Natural Language Processing. It will also host high-quality review studies on the involved research fields. The aim of this Special Issue is to provide the research community with a rich collection of key papers that will open new directions in this area by introducing new models, methods, and techniques.

Indicative topics of interest include, but are not limited to:

  1. Large-language models;
  2. Sentiment analysis;
  3. Machine translation;
  4. Machine-generated/machine-assisted image and video annotation;
  5. Effective text/short-text representations;
  6. Messengers and social networks;
  7. Text and short-text classification;
  8. Text and short-text clustering;
  9. NLP applications for low-power devices.

Dr. Leonidas Akritidis
Prof. Dr. Panayiotis Bozanis
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Electronics is an international peer-reviewed open access semimonthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2400 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

  • LLM
  • machine translation
  • sentiment analysis
  • text representation
  • text classification
  • text clustering

Benefits of Publishing in a Special Issue

  • Ease of navigation: Grouping papers by topic helps scholars navigate broad scope journals more efficiently.
  • Greater discoverability: Special Issues support the reach and impact of scientific research. Articles in Special Issues are more discoverable and cited more frequently.
  • Expansion of research network: Special Issues facilitate connections among authors, fostering scientific collaborations.
  • External promotion: Articles in Special Issues are often promoted through the journal's social media, increasing their visibility.
  • e-Book format: Special Issues with more than 10 articles can be published as dedicated e-books, ensuring wide and rapid dissemination.

Further information on MDPI's Special Issue polices can be found here.

Published Papers (7 papers)

Order results
Result details
Select all
Export citation of selected articles as:

Research

Jump to: Review

19 pages, 1440 KiB  
Article
A Picture May Be Worth a Hundred Words for Visual Question Answering
by Yusuke Hirota, Noa Garcia, Mayu Otani, Chenhui Chu and Yuta Nakashima
Electronics 2024, 13(21), 4290; https://doi.org/10.3390/electronics13214290 - 31 Oct 2024
Abstract
How far can textual representations go in understanding images? In image understanding, effective representations are essential. Deep visual features from object recognition models currently dominate various tasks, especially Visual Question Answering (VQA). However, these conventional features often struggle to capture image details in [...] Read more.
How far can textual representations go in understanding images? In image understanding, effective representations are essential. Deep visual features from object recognition models currently dominate various tasks, especially Visual Question Answering (VQA). However, these conventional features often struggle to capture image details in ways that match human understanding, and their decision processes lack interpretability. Meanwhile, the recent progress in language models suggests that descriptive text could offer a viable alternative. This paper investigated the use of descriptive text as an alternative to deep visual features in VQA. We propose to process description–question pairs rather than visual features, utilizing a language-only Transformer model. We also explored data augmentation strategies to enhance training set diversity and mitigate statistical bias. Extensive evaluation shows that textual representations using approximately a hundred words can effectively compete with deep visual features on both the VQA 2.0 and VQA-CP v2 datasets. Our qualitative experiments further reveal that these textual representations enable clearer investigation of VQA model decision processes, thereby improving interpretability. Full article
15 pages, 2841 KiB  
Article
Named Entity Recognition for Equipment Fault Diagnosis Based on RoBERTa-wwm-ext and Deep Learning Integration
by Feifei Gao, Lin Zhang, Wenfeng Wang, Bo Zhang, Wei Liu, Jingyi Zhang and Le Xie
Electronics 2024, 13(19), 3935; https://doi.org/10.3390/electronics13193935 - 5 Oct 2024
Viewed by 613
Abstract
Equipment fault diagnosis NER is to extract specific entities from Chinese equipment fault diagnosis text, which is the premise of constructing an equipment fault diagnosis knowledge graph. Named entity recognition for equipment fault diagnosis can also provide important data support for equipment maintenance [...] Read more.
Equipment fault diagnosis NER is to extract specific entities from Chinese equipment fault diagnosis text, which is the premise of constructing an equipment fault diagnosis knowledge graph. Named entity recognition for equipment fault diagnosis can also provide important data support for equipment maintenance support. Equipment fault diagnosis text has complex semantics, fuzzy entity boundaries, and limited data size. In order to extract entities from the equipment fault diagnosis text, this paper presents an NER model for equipment fault diagnosis based on RoBERTa-wwm-ext and Deep Learning network integration. Firstly, this model uses the RoBERTa-wwm-ext to extract context-sensitive embeddings of text sequences. Secondly, the context feature information is obtained through the BiLSTM network. Thirdly, the CRF is combined to output the label sequence with a constraint relationship, improve the accuracy of sequence labeling task, and complete the entity recognition task. Finally, experiments and predictions are carried out on the constructed dataset. The results show that the model can effectively identify five types of equipment fault diagnosis entities and has higher evaluation indexes than the traditional model. Its precision, recall, and F1 value are 94.57%, 95.39%, and 94.98%, respectively. The case study proves that the model can accurately recognize the entity of the input text. Full article
Show Figures

Figure 1

18 pages, 1822 KiB  
Article
Self-HCL: Self-Supervised Multitask Learning with Hybrid Contrastive Learning Strategy for Multimodal Sentiment Analysis
by Youjia Fu, Junsong Fu, Huixia Xue and Zihao Xu
Electronics 2024, 13(14), 2835; https://doi.org/10.3390/electronics13142835 - 18 Jul 2024
Viewed by 705
Abstract
Multimodal Sentiment Analysis (MSA) plays a critical role in many applications, including customer service, personal assistants, and video understanding. Currently, the majority of research on MSA is focused on the development of multimodal representations, largely owing to the scarcity of unimodal annotations in [...] Read more.
Multimodal Sentiment Analysis (MSA) plays a critical role in many applications, including customer service, personal assistants, and video understanding. Currently, the majority of research on MSA is focused on the development of multimodal representations, largely owing to the scarcity of unimodal annotations in MSA benchmark datasets. However, the sole reliance on multimodal representations to train models results in suboptimal performance due to the insufficient learning of each unimodal representation. To this end, we propose Self-HCL, which initially optimizes the unimodal features extracted from a pretrained model through the Unimodal Feature Enhancement Module (UFEM), and then uses these optimized features to jointly train multimodal and unimodal tasks. Furthermore, we employ a Hybrid Contrastive Learning (HCL) strategy to facilitate the learned representation of multimodal data, enhance the representation ability of multimodal fusion through unsupervised contrastive learning, and improve the model’s performance in the absence of unimodal annotations through supervised contrastive learning. Finally, based on the characteristics of unsupervised contrastive learning, we propose a new Unimodal Label Generation Module (ULGM) that can stably generate unimodal labels in a short training period. Extensive experiments on the benchmark datasets CMU-MOSI and CMU-MOSEI demonstrate that our model outperforms state-of-the-art methods. Full article
Show Figures

Figure 1

12 pages, 356 KiB  
Article
Raising the Bar on Acceptability Judgments Classification: An Experiment on ItaCoLA Using ELECTRA
by Raffaele Guarasci, Aniello Minutolo, Giuseppe Buonaiuto, Giuseppe De Pietro and Massimo Esposito
Electronics 2024, 13(13), 2500; https://doi.org/10.3390/electronics13132500 - 26 Jun 2024
Viewed by 951
Abstract
The task of automatically evaluating acceptability judgments has relished increasing success in Natural Language Processing, starting from including the Corpus of Linguistic Acceptability (CoLa) in the GLUE benchmark dataset. CoLa spawned a thread that led to the development of several similar datasets in [...] Read more.
The task of automatically evaluating acceptability judgments has relished increasing success in Natural Language Processing, starting from including the Corpus of Linguistic Acceptability (CoLa) in the GLUE benchmark dataset. CoLa spawned a thread that led to the development of several similar datasets in different languages, broadening the investigation possibilities to many languages other than English. In this study, leveraging the Italian Corpus of Linguistic Acceptability (ItaCoLA), comprising nearly 10,000 sentences with acceptability judgments, we propose a new methodology that utilizes the neural language model ELECTRA. This approach exceeds the scores obtained from current baselines and demonstrates that it can overcome language-specific limitations in dealing with specific phenomena. Full article
Show Figures

Figure 1

23 pages, 3469 KiB  
Article
On Embedding Implementations in Text Ranking and Classification Employing Graphs
by Nikitas-Rigas Kalogeropoulos, Dimitris Ioannou, Dionysios Stathopoulos and Christos Makris
Electronics 2024, 13(10), 1897; https://doi.org/10.3390/electronics13101897 - 12 May 2024
Viewed by 973
Abstract
This paper aims to enhance the Graphical Set-based model (GSB) for ranking and classification tasks by incorporating node and word embeddings. The model integrates a textual graph representation with a set-based model for information retrieval. Initially, each document in a collection is transformed [...] Read more.
This paper aims to enhance the Graphical Set-based model (GSB) for ranking and classification tasks by incorporating node and word embeddings. The model integrates a textual graph representation with a set-based model for information retrieval. Initially, each document in a collection is transformed into a graph representation. The proposed enhancement involves augmenting the edges of these graphs with embeddings, which can be pretrained or generated using Word2Vec and GloVe models. Additionally, an alternative aspect of our proposed model consists of the Node2Vec embedding technique, which is applied to a graph created at the collection level through the extension of the set-based model, providing edges based on the graph’s structural information. Core decomposition is utilized as a method for pruning the graph. As a byproduct of our information retrieval model, we explore text classification techniques based on our approach. Node2Vec embeddings are generated by our graphs and are applied in order to represent the different documents in our collections that have undergone various preprocessing methods. We compare the graph-based embeddings with the Doc2Vec and Word2Vec representations to elaborate on whether our approach can be implemented on topic classification problems. For that reason, we then train popular classifiers on the document embeddings obtained from each model. Full article
Show Figures

Figure 1

28 pages, 4927 KiB  
Article
Enhancing Stock Market Forecasts with Double Deep Q-Network in Volatile Stock Market Environments
by George Papageorgiou, Dimitrios Gkaimanis and Christos Tjortjis
Electronics 2024, 13(9), 1629; https://doi.org/10.3390/electronics13091629 - 24 Apr 2024
Viewed by 1757
Abstract
Stock market prediction is a subject of great interest within the finance industry and beyond. In this context, our research investigates the use of reinforcement learning through implementing the double deep Q-network (DDQN) alongside technical indicators and sentiment analysis, utilizing data from Yahoo [...] Read more.
Stock market prediction is a subject of great interest within the finance industry and beyond. In this context, our research investigates the use of reinforcement learning through implementing the double deep Q-network (DDQN) alongside technical indicators and sentiment analysis, utilizing data from Yahoo Finance and StockTwits to forecast NVIDIA’s short-term stock movements over the dynamic and volatile period from 2 January 2020, to 21 September 2023. By incorporating financial data, the model’s effectiveness is assessed in three stages: initial reliance on closing prices, the introduction of technical indicators, and the integration of sentiment analysis. Early findings showed a dominant buy tendency (63.8%) in a basic model. Subsequent phases used technical indicators for balanced decisions and sentiment analysis to refine strategies and moderate rewards. Comparative analysis underscores a progressive increase in profitability, with average profits ranging from 57.41 to 119.98 with full data integration and greater outcome variability. These results reveal the significant impact of combining diverse data sources on the model’s predictive accuracy and profitability, suggesting that integrating sentiment analysis alongside traditional financial metrics can significantly enhance the sophistication and effectiveness of algorithmic trading strategies in fluctuating market environments. Full article
Show Figures

Figure 1

Review

Jump to: Research

27 pages, 2276 KiB  
Review
Sentiment Dimensions and Intentions in Scientific Analysis: Multilevel Classification in Text and Citations
by Aristotelis Kampatzis, Antonis Sidiropoulos, Konstantinos Diamantaras and Stefanos Ougiaroglou
Electronics 2024, 13(9), 1753; https://doi.org/10.3390/electronics13091753 - 2 May 2024
Cited by 1 | Viewed by 3372
Abstract
Sentiment Analysis in text, especially text containing scientific citations, is an emerging research field with important applications in the research community. This review explores the field of sentiment analysis by focusing on the interpretation of citations, presenting a detailed description of techniques and [...] Read more.
Sentiment Analysis in text, especially text containing scientific citations, is an emerging research field with important applications in the research community. This review explores the field of sentiment analysis by focusing on the interpretation of citations, presenting a detailed description of techniques and methods ranging from lexicon-based approaches to Machine and Deep Learning models. The importance of understanding both the emotion and the intention behind citations is emphasized, reflecting their critical role in scientific communication. In addition, this study presents the challenges faced by researchers (such as complex scientific terminology, multilingualism, and the abstract nature of scientific discourse), highlighting the need for specialized language processing techniques. Finally, future research directions include improving the quality of datasets as well as exploring architectures and models to improve the accuracy of sentiment detection. Full article
Show Figures

Figure 1

Back to TopTop