Applied Machine Learning: New Methods, Applications, and Achievements

1. Introduction

The realm of machine learning (ML) is one of the most dynamic and compelling domains within the computing landscape today. Over the past few decades, ML has firmly embedded itself in our daily lives, offering effective solutions to real-world challenges. The scope of ML’s applications spans a multitude of sectors, encompassing engineering, industry, business, finance, medicine, and beyond. ML’s comprehensive spectrum of techniques embraces traditional algorithms such as linear regression, k-nearest neighbors, decision trees, support vector machines, and neural networks, while also incorporating cutting-edge innovations such as deep learning and boosted tree models.

Finding the optimal architecture and parameters for ML models presents a substantial challenge, but is essential for attaining strong performances in both learning and generalization. Furthermore, as ML is practically applied, it encounters the complexities of managing extensive, incomplete, distorted, and uncertain data. A key requirement for ML methods is interpretability, ensuring a clear understanding of how these models function and fostering confidence in their outcomes.

Based on the reviewers’ feedback, as well as the evaluations of the editor, 12 papers from 26 submissions have been selected for this Special Issue. These papers examine the conceptualization of problems, data representation, feature engineering, ML models employed, discerning comparisons against existing techniques, and the lucid interpretation of results. This Special Issue strives to not only showcase the prolific applications of ML, but to also provide insights into the methodologies and interpretations that underpin these advancements. The 12 papers, which cover a broad range of topics, are introduced briefly below.

2. Summary of the Contributions

Paper [1] addresses the challenge of recognizing physical activity in individuals with spinal cord injuries (SCI) using sensor-based approaches. SCI patients often experience health complications like obesity and muscle weakness, necessitating effective rehabilitation. Existing methods, relying on patient surveys, may not accurately capture actual activity levels. The advent of physical activity recognition systems presents a more reliable solution. The paper compares vision-based and sensor-based approaches, favoring wearable sensors for their affordability and ease of use. Sensor placement is crucial, with lower-limb sensors suited for locomotion and upper-limb activity requiring wrist and upper-arm sensors. This paper explores various applications of activity recognition, particularly in rehabilitation monitoring.

Segmenting continuous raw data before classification is a key challenge. While the sliding window approach is common, selecting an appropriate window size is vital to ensuring accuracy. Short windows may truncate activities, while long windows can merge them. Adaptive window techniques have been proposed to address this issue. The paper introduces a novel segmentation method tailored to dynamic activities in rehabilitation. An experiment compares this approach to the fixed sliding window method, demonstrating its effectiveness in boosting recognition accuracy.

This study, which employs wrist-worn accelerometers, reveals an accuracy improvement of over 5%, enhancing model robustness and successfully classifying similar activities. The method achieves recognition rates exceeding 91% with various ML classifiers, particularly the support vector machine. Wearable sensors, such as accelerometers, prove invaluable in rehabilitation assessment. The study acknowledges the potential limitations, while also highlighting the method’s potential for segmenting and recognizing physical activities, especially in rehabilitation scenarios.

In [2], a framework for analyzing consumer behavior in the rapidly growing over-the-top (OTT) media consumption market was introduced, considering factors such as pricing, service delivery, and infrastructure investments. This paper addresses the challenge of an imbalanced consumer distribution within the OTT market, and the need for accurate analysis reflecting changing market conditions due to factors like the COVID-19 pandemic. This paper highlights the rapid growth of the OTT market, driven by the availability of digital media content via the internet and IP-based paths. Platforms like YouTube, Netflix, and Amazon Prime are transforming the media landscape, threatening traditional markets. With a projected market value of over a trillion dollars by 2027, various service providers and telecom carriers are competing fiercely in this domain. The COVID-19 pandemic has further accelerated the growth of OTT consumption, making it vital to analyze consumer behavior effectively.

The proposed framework combines a conditional probability-based approach with machine learning techniques, such as support vector machines, k-nearest neighbors, and decision trees. This approach enhances classification performance, particularly for imbalanced consumer groups. The framework also adapts to changing consumer trends by dynamically retraining with incoming OTT consumer data. It yields improved classification accuracy, particularly for lower-number classes, showing a recall-based improvement of 5.3% to 19.2%. Unlike conventional methods, the proposed framework maintains consistently high performance, even as the OTT market environment changes. The study underscores the practical significance of the framework for companies participating in the OTT market, offering a stable performance in a dynamic environment.

Paper [3] investigates demand forecasting for the automotive original equipment manufacturer (OEM) sector, assessing 21 baseline, statistical, and ML algorithms. Utilizing real-world data from a European OEM, the study highlights the superiority of global ML models over local ones. The paper introduces a comprehensive set of metrics for evaluating demand forecasting models, emphasizing their practicality. The research demonstrates the effectiveness of pooling product data based on historical demand magnitude to mitigate forecast errors in global models.

The authors present two data pooling strategies for building global time series models. A novel approach is introduced to control forecast errors in global models. The integration of complementary data sources, such as world GDP, unemployment rates, and fuel prices, is discussed. The findings indicate that grouping products based on demand patterns and magnitude improves the performance of the ML models. The research reveals that certain models, such as SVR, voting ensemble, and random forest, outperform others, particularly when trained on product data of the same demand type. A comparison of batch and streaming ML models highlights the robustness of batch models. The potential of digital twins for accurate forecasts and scenario estimation is explored. The study identifies avenues for future research, focusing on refining error bounding strategies, addressing anomalous forecasts in global models, and enhancing model explainability for user trust. This paper concludes by providing insights into the potential applications of ML in demand forecasting for the automotive OEM sector.

In [4], a novel flight path planning algorithm for unmanned aerial vehicles (UAVs), based on ellipsoidal mapping, is introduced. The paper discusses the use of spheres and ellipsoids for geometric representations and mapping, emphasizing their advantages over other methods. The research addresses the challenge of efficiently calculating distances between ellipsoidal objects by a neural network. The algorithm utilizes teaching–learning based optimization (TLBO) and takes advantage of ellipsoidal representations of obstacles for UAV navigation. The method aims to provide collision-free, smooth flight paths, accommodating various environments, including indoor and outdoor settings.

The methodology used to calculate distances between ellipsoids using a neural network involves generating a training dataset and implementing a novel normalization method. The proposed fitness function for flight path planning considers several factors, including ensuring safe distances between the obstacle ellipsoids and the UAV ellipsoid to prevent collisions, minimizing the overall proximity range of the UAV throughout its entire flight path and other desired features. Results are presented, comparing the algorithm’s performance with other evolutionary algorithms and rapidly exploring random tree star algorithm (RRT*). While evolutionary techniques with the proposed fitness function generally outperformed RRT*, the latter demonstrated a better performance in terms of time. Future work is outlined, including developing a cost function for comparing different flight path planning algorithms, replacing the greedy strategy with reinforcement learning, and designing an intelligent low-level control algorithm for UAV navigation.

Paper [5] proposes a gesture recognition system integrated into visible light communication (VLC) systems for human–computer interaction applications. The GR technique utilizes light transitions between fingers, which are detected via a low-cost light-emitting diode (LED) and a photo-diode sensor at the receiver side. The system employs a long short-term memory (LSTM) neural network to classify finger movements based on interruptions in direct light transmission, making it suitable for high-speed communication. The accuracy of the proposed system in identifying gestures reaches 88%.

The authors present a solution that involves minimal additional cost, as it is integrated into a VLC-capable system. The LSTM-based approach offers effective gesture recognition with low computational complexity compared to traditional video processing techniques. The system’s performance is robust, achieving accurate recognition even under natural conditions with varying speeds and lighting conditions. Key contributions include the development of a practical gesture recognition methodology for VLC, utilizing off-the-shelf components, and demonstrating the effectiveness of a single photo-diode receiver setup. The system’s accuracy and efficiency are notable, allowing for gesture recognition within a communications-based VLC system. Possible future directions involve increasing the number of recognizable gestures, exploring gesture recognition from different aspects of sunlight and placements, and incorporating the system into automated VLC setups for more versatile applications. The study’s results highlight the potential of this approach for various domains such as healthcare, commerce, and home automation.

Paper [6] focuses on analyzing and modeling road traffic accidents (RTAs) using ML classifiers. The aim is to assist transportation authorities and policymakers by developing predictive models for RTAs. The research utilizes a real-life RTA dataset from Gauteng, South Africa, and evaluates the performance of various ML classifiers including naïve Bayes, logistic regression, k-nearest neighbor, AdaBoost, support vector machine, and random forest. The study also includes dimensionality reduction techniques and multiple missing data methods.

The findings of the study show that a random forest (RF) classifier combined with multiple imputations by chained equations (MICE) for handling missing data achieves the best overall performance. RF consistently outperformed other classifiers in terms of accuracy, precision, recall, and AUC. Additionally, the study found promising results with linear discriminant analysis (LDA) for dimensionality reduction. The study acknowledges its limitations, such as the use of a dataset from a specific region and the exclusion of certain features. Future work could involve hyperparameter tuning for specific classifiers, testing other ML algorithms like artificial neural networks and deep learning, and expanding the analysis to different datasets or regions.

In [7], a robust financial crisis early warning model for Chinese listed companies is proposed. The study’s premise stems from the economic challenges and risks that these companies face, driving the need for effective risk prediction and mitigation strategies. The authors propose a unique approach by incorporating textual data analysis, specifically the sentiment and tone analysis of financial news texts and the management discussion and analysis (MD&A) sections in annual reports. By leveraging web crawling and textual analysis techniques, the emotional tones of these texts are extracted. These tones serve as supplementary indicators alongside traditional financial indicators, providing insights into internal and external aspects of listed companies. The study includes 1082 Chinese A-share listed companies from 2012 to 2021 as its sample.

This research systematically evaluates the impact of emotional tone indicators on the accuracy of financial crisis early warning models. Thirteen ML models are employed to predict financial crises, and their performance is assessed using various evaluation metrics. The findings highlight several key points, such as textual data as indicators, model comparisons, external vs. internal information, and implications and future directions. The findings underscore the potential of emotional tone analysis of financial news in improving early warning models. This approach helps mitigate risks and increases the accuracy of predictions for listed companies. Furthermore, the study suggests avenues for future research, such as exploring the impact of linguistic features beyond emotional tone and developing specialized comprehensive emotional dictionaries for financial texts.

Paper [8] explores the field of DNA-based informatics, focusing on the construction and operation of computers using DNA as both hardware and software. This concept involves the utilization of DNA computers for intelligent and personalized diagnostics, particularly in the context of medical treatment. A new approach to designing diagnostic biochips that combines ML methods with the concept of biomolecular queue automata is introduced. This enables the scheduling of computational tasks at the molecular level by manipulating DNA molecules through cutting and ligating sequences. The authors stress the importance of ML methods in the design of biochips based on biomolecular computers, as accurate knowledge of unique DNA sequences is a fundamental aspect of these solutions.

The study highlights the potential of biomolecular computers in constructing biochips and emphasizes the significance of deterministic input-driven queue automata as a theoretical model for these systems. The use of specific restriction enzymes, particularly type IIB restriction endonucleases, is emphasized for the design of biomolecular computers with memory capabilities, such as queue automata. The paper introduces the concept of "Queue-PCR" as an innovative way to automate the polymerase chain reaction (PCR) method using biomolecular computers, thereby suggesting new avenues for the automation of molecular genetics techniques.

Paper [9] focuses on the application of ML algorithms to predict the clinical evolution of patients diagnosed with COVID-19. It aims to optimize the diagnostic process by utilizing predictive modeling to classify the clinical course of COVID-19 cases. The research involves a comparative analysis of various classification algorithms, including k-nearest neighbors, naïve Bayes, decision trees (DT), multilayer perceptrons (MLPs), and support vector machines. It analyzes 30,000 cases during the training and testing phases of the prediction models. The authors underscore the significance of accurate predictions for patients’ vital prognosis and the efficiency of initial consultations in hospitals.

The conclusion of this study highlights the achievement of predicting the clinical evolution of COVID-19 patients using optimized ML models. The MLP algorithm is identified as the most effective for this purpose, based on comparative benchmarks. The research suggests potential future work, including analyzing clinical data using different algorithms, applying ensemble learning, and exploring additional neural network algorithms. However, the study acknowledges limitations such as data quality concerns in medical applications of ML, which can impact diagnoses and introduce biases.

Paper [10] explores the use of generative adversarial networks (GANs) for regression tasks, focusing on multi-output regression in non-image data. The study introduces the concept of MOR-GANs, which employ Wasserstein GAN (WGAN) as a regression method. The paper compares the performance of MOR-GANs with Gaussian process regression (GPR) on various datasets and introduces a prediction algorithm for GANs to generate responses based on independent variables.

The authors emphasize that WGANs perform well in regression tasks, often surpassing GPR, and showcase their effectiveness across diverse datasets. MOR-GANs show notable performance in handling variable uncertainty, multi-modal distributions, and multi-output regression tasks. Importantly, this performance is achieved without requiring extensive modifications to the GAN architecture. The authors also speculate on potential applications in domains like image reconstruction and high-dimensional modeling, where MOR-GANs could provide invaluable insights and predictions.

In [11], a novel compressed convolutional neural network designed to address the challenges of human detection in computer vision, ReSTiNet, is introduced. Human detection is crucial for applications like public safety and security surveillance. ReSTiNet aims to incorporate a compact size, high detection speed, and accuracy in its design, inspired by recent advances in deep learning techniques. The main aim of ReSTiNet is to create a more capable human detection model suitable for portable devices with limited processing power. This lightweight model enhances the performance of intelligent surveillance systems without increasing hardware costs or processing demands. The paper emphasizes the importance of fire modules, their placement, and the integration of residual connections within the architecture.

The proposed ReSTiNet model is based on Tiny-YOLO architecture, and incorporates fire modules from SqueezeNet. It strategically adjusts the number and placement of these modules to reduce the model’s parameter count and overall size. Residual connections are integrated within the fire modules to enhance feature propagation and information flow, resulting in improved detection speed and accuracy. Performance of ReSTiNet surpasses other lightweight models, such as MobileNet and SqueezeNet, in terms of mean average precision. The study concludes that ReSTiNet can be adapted to various deep convolutional neural networks for compression purposes. The model’s performance will be further optimized for high-resolution images in future work, particularly for datasets like EuroCity Persons.

Finally, paper [12] introduces the natural language policy translator (NLPT) 2.0, an extended version of the NLPT 1.0 system, aimed at systematically translating data privacy policies from natural language to controlled natural language for data sharing agreement (CNL4DSA). With the surge in online social networks, user-generated content has led to data exploitation for various purposes. Privacy policies, often expressed in natural language, outline data handling, usage, and authorization details, but lack automatic control mechanisms. NLPT 2.0 addresses this by enabling translation for enhanced machine processing.

The proposed methodology combines natural language processing, logic programming, and ontologies. The system offers a user-friendly Graphical User Interface that allows non-expert users to input policies in natural language, which the system then translates into CNL4DSA. The study’s key aspects include the translation of social network data privacy policies and the effectiveness of the NLPT 2.0 system. Testing involved the use of privacy policies from popular social network platforms. The system demonstrated promising performance in components like ontology creation, fragment extraction, and context extraction, with results ranging from 70% to 95%. While human intervention is required for certain initial vocabulary and ontology definitions, the system aims to become increasingly automatic.

NLPT 2.0 addresses the complexity of parsing intricate phrases in original policies by introducing the role of a Policy Writer. Although some aspects still require human involvement, NLPT 2.0 is a valuable tool for translating privacy policies and enhancing machine analysis. Future work will focus on fully automating the process and addressing the remaining challenges.