Applications of Deep Learning Techniques

1. Introduction

Since the emergence of “deep neural networks (DNNs)”, several deep learning [1] methods, including convolutional neural networks (CNNs), graph neural networks (GNNs), sequence-to-sequence models, generative models, deep reinforcement learning (DRL), and large language models (LLMs) [2], have been proposed [3,4]. These deep learning techniques are widely used in different fields and domains, for instance, in self-driving vehicles, virtual assistants, and healthcare devices, as well as for personalization, automatic game playing, chatbots, legal intelligence, etc. [3,5,6]. Meanwhile, remarkable deep learning applications such as AlphaGo [7], Alexa [8], AlphaFold 3 [9], and ChatGPT [6] have been developed that are changing daily lives.

Ensuring the continuous development and implementation of deep learning techniques in a wider range of fields, such as social science, business, agriculture, and smart cities, as well as promoting and enhancing the responsibility, fairness, and safety of use in deep learning applications are the goals of this Special Issue of Electronics, entitled “Applications of Deep Learning Techniques”.

2. The Present Issue

For this issue, 37 submissions were received. Each submission was initially assessed by one of the Guest Editors to evaluate its relevance to the topic of the Special Issue. If related, the submission was reviewed by at least two external reviewers; otherwise, it was rejected. Out of the 37 submissions, 15 were accepted after a rigorous peer-review process. The contributions included in this issue address the applications of different deep learning algorithms in a wide range of domains, such as car rental price prediction, automatic speech recognition, highway visibility prediction, and metro passenger flow prediction. A brief overview of each is presented below.

In the first contribution, Yang et al. review several machine learning methods for predicting car rental prices. Their study explores traditional forecasting methods, such as random forest regression and ARIMA, as well as deep learning-based methods, including multilayer perception, 1D CNN, and LSTM. Their results highlight ARIMA and LSTM as effective models for forecasting car rental prices.

In another study, Kumar et al. apply advanced AI models, especially the Swin transformer, to ASL and TSL to develop a single sign language translating platform. This study compares the efficiency of models such as Swin, DAT, ResNet-50, and VGG-16 for ASL recognition. The key notion involves the development of real-time ASL to English translation applications and an educational approach to training purposes. The balance between accuracy and speed of processing is discussed, and methods for incorporating MMM and LLMs to provide a more inclusive environment are outlined.

In the third contribution, Tran et al. present a new large-scale Vietnamese speech corpus (LSVSC). This corpus is very diverse regarding gender, topics, and regions, which might be helpful for automatic speech recognition (ASR) in the Vietnamese language. In contrast to previous Vietnamese corpora, which contain a limited number of words and phrases and are mostly only suitable for specific purposes, this new corpus is open to the public and comprises realistic models and is thus more relevant to the ASR field. Their study assesses the performance of the proposed LSVSC with cutting-edge end-to-end ASR models of LAS and Speech-Transformer. It also emphasizes the challenges of ASR system development in Vietnamese, namely because the language is tonal and monosyllabic with many dialects. The empirical analysis points toward the suitability of this corpus in ASR scenarios, as it offers a rich dataset that correctly models Vietnamese speech issues.

The study by Li et al. introduces the ATCNet model, which aims to forecast highway visibility and increase road safety, especially in regions that experience frequent cases of meteorological calamities, such as fog. The model includes transformer networks, capsule networks (CapsNet), and self-attention, which enhance temporal and spatial features for precise and accurate visibility predictions. A new dataset (WD13VIS) is developed from high-altitude highways in the Yunnan Province of China for evaluating the model. As evidenced by the experimental results, ATCNet achieves higher visibility forecast accuracy than the current models. Their research highlights the need for multidimensional data in addressing difficulties related to visibility prediction.

In another contribution, Lu et al. present a new model called MST-GRT that aims to predict short-term metro passenger flow using multi-time granularity data and multi-graph architecture. The model synthesizes data from different time levels through a deep convolutional neural network built with residual blocks, which facilitate the extraction of features with multi-time scales. Furthermore, the model incorporates multi-graph convolution modules to capture spatial features from diverse perspectives regarding the metro network and uses a reconstructed temporal convolution network layer to examine the temporal interaction of the data. This approach addresses the deficiencies in the indicated models, including their ineffectiveness in modeling long-term dependencies and the limited use of high-resolution data. The performance of the MST-GRT model is compared with the baseline models using the Hangzhou Metro smart-card dataset, and the results indicate that MST-GRT has superior performance in passenger flow prediction tasks compared with the baseline models, most significantly at the 60-min time grain size.

The article by Lu et al. presents an upgraded YOLOv8-MeY model that detects small foreign objects such as iron filings on bags of sugar in large industries. The YOLOv8 model was modified to address issues related to the identification of small objects in a given area that constitutes a minimal portion of the image area. The model was implemented in real time, by addressing the challenges encountered in a food factory, where automated sugar dispensing was carried out. The model achieved a 92% recognition rate. This modified model provides an optimal ratio of low weight, high speed of inference, and accuracy since one of the key challenges in the modern food industry is the detection of small objects that may pose a threat to consumers’ health in industrial food production.

On the topic of multi-target tracking in multi-agent systems, Han et al. address cooperative decision-making via deep reinforcement learning. In this paper, they present three multi-target allocation paradigms and analyze POLICY3, which, in its approach to agents and targets, outperforms other methods in simulations. To overcome the issues related to policy gaps in the MATD3 algorithm used by agents, a new approach called DAO-MATD3 is introduced as an optimized version of MATD3, which also involves integrating the dynamics of the actor network. Their study also confirms the objective improvement achieved in efficiency through adopting the proposed POLICY3-DAO-MATD3 architecture and highlights the ability of the model to avoid collision in pursuit tasks executed by multiple agents. The results of computer experiments reveal that all the considered strategies are efficient in different scenarios.

In another study, Yang et al. propose a method for improving the prediction accuracy of the initial production capacity of tight oil horizontal wells using deep neural networks (DNNs) trained on small datasets. The method addresses the limitations of traditional methods by employing a DNN model pretrained with sparse autoencoders (SAEs). The DNN was trained on a small dataset of 650 data points, covering 13 factors related to geology, development, and engineering. Improved L2 regularization techniques were used to enhance model robustness and prevent overfitting. The proposed method outperforms traditional shallow neural networks (SNNs) and support vector machines (SVMs).

The study by Zhao et al. involves the development of a robust and efficient system for recognizing wafer characters in complex industrial environments. This study proposes an improved YOLO v7-Tiny model with key enhancements, including an optimized spatial channel attention module (CBAM-L) for better feature extraction, an improved neck structure (BiFPN) for enhanced feature fusion, and an angle parameter for detecting tilted characters. The model outperforms other state-of-the-art models and demonstrates superior performance, especially in complex backgrounds and for rotated characters. The improved YOLO v7-Tiny model is both accurate and computationally efficient, making it suitable for deployment in embedded industrial devices with limited memory. Their study highlights the potential of this model for practical application in industrial wafer character recognition tasks.

In another contribution, Yahia et al. provide an overview of the research landscape of 3D object detection and orientation by identifying the key contributors and themes. Using a bibliometric analysis of publications from 2022 to 2023 in the Scopus database, the authors examined the conceptual, intellectual, and social structures of this research field. The analysis revealed that China, particularly Tsinghua University, leads in research volume and international collaborations, with Li Y being the top contributor. The key research themes include deep learning, autonomous driving, point clouds, and LiDAR technology, which are critical for advancements in robotics and autonomous vehicles. Deep learning is the most frequent keyword. Their study also highlighted China’s strong international collaborations, especially with the USA and Hong Kong.

The study by Kim et al. proposes a method for improving the accuracy and adaptability of knowledge tracing (KT) models by incorporating question difficulty into input features and outputs for better predictions of learners’ knowledge states in English proficiency assessments. Three experimental approaches are introduced: (i) reconstructing input features by incorporating difficulty, (ii) predicting difficulty with a multi-task learning objective, and (iii) enhancing the model’s output representations from (i) and (ii). Their study utilizes the EdNet-KT1 dataset, which includes over 224 million learning interactions from more than 780,000 students. The proposed methods exhibit significant performance improvements, especially when question difficulty is integrated into the model.

In another article, Li et al. improve quantitative investment strategies by developing a model that combines historical trading data and sentiment analysis from news articles. The authors propose a deep hybrid model that integrates technical indicators from three years of trading data with sentiment information extracted using a fine-tuned ALBERT model. The model employs LSTM and Transformer architectures to predict stock price movements, capturing both temporal trends and long-range dependencies. The proposed model outperformed traditional methods, achieving a 32.06% annualized return rate and a 5.81% maximum drawdown rate in backtesting. These results highlight the superiority of the model compared to the CSI 300 index, proving the practical value of the proposed quantitative investment approach in providing guidance for investors’ decision-making and yielding effective and stable returns.

The article by Guo and Mao addresses the challenge of accurately predicting long-term NOx emissions from rotary kilns by developing a hybrid model combining LSTM networks and Transformer architecture. This LSTM–Transformer model captures both short- and long-term dependencies, improving the accuracy of time series predictions. Tested on datasets from an alumina rotary kiln, the model outperformed other baseline models. The results demonstrate the model’s superior long-term prediction capabilities. However, the authors pointed to the model’s complexity and memory demands, which could limit its implementation.

In another contribution, Li et al. present a study to improve goat behavior recognition accuracy using an optimized XGBoost model, enhanced by the whale optimization algorithm (WOA) and social learning strategies. Data were collected from a three-axis acceleration sensor on the goats’ backs. The SL-WOA-XGBoost method outperformed the unoptimized XGBoost model by 3.12%. The highest accuracy was 97.14% for lying behavior, and the lowest was 91.47% for standing behavior. The results suggest that this approach offers a valuable tool for health monitoring and early disease detection in goats.

In the final contribution, Li et al. introduce a deconvolutional neural network (DNN) that significantly enhances the automation of the shoe sole gluing process, especially for irregular and unique sole designs. Unlike traditional methods, this approach avoids pooling layers, preserving spatial information and improving accuracy. The proposed method was trained on 3D point cloud data and exhibited high accuracy and precision. The method is suitable for real-time industrial use with a processing time of under two seconds. This research highlights the potential for integrating neural networks into more intelligent and automated manufacturing processes in the footwear industry.

3. Future Directions

The deep learning technique, as the most crucial breakthrough in machine learning history, has raised considerable interest among academics as well as professionals in different industries. It has been widely and successfully applied to different domains, as outlined by the research articles amassed in this Special Issue. The future directions and key topics on the applications and deep learning techniques include the following:

• The application of deep learning techniques for legal intelligence, such as legal text classification, argument mining, and judgment prediction;
• The application of deep learning techniques for natural language processing, such as information retrieval, text summarization, and sentiment analysis;
• The application of deep learning techniques for software engineering, including software development, software testing, and software maintenance;
• The application of deep learning techniques for healthcare and medical systems, such as precision medicine, drug discovery, molecular modeling, smart diagnostics, and medical imaging;
• The application of deep learning techniques for social media analysis, such as real-time violence detection, dis/misinformation, hate speech recognition, and country reputation monitoring;
• The application of deep learning techniques for academic data mining, such as information extraction from scientific text, innovation measurement, and citation analysis;
• The application of deep generative techniques for education, entertainment, finance, and materials science;
• The application of deep learning techniques in other special domains such as cybersecurity, business intelligence, Internet of Things, precious agriculture, and smart cities;
• Data quality evaluation, assurance, and improvement for deep learning in various applications;
• Responsibility, fairness, ethics, bias, trustworthiness, transparency, accountability, safety, and privacy in deep learning applications.

In addition, novel and effective approaches for constructing large-scale and high-quality datasets for domain-specific deep learning applications [10]; practical frameworks; and strategies for data quality evaluation, enhancement, and augmentation for deep learning applications [2,11] are also essential directions that deserve more attention.