Machine Learning and Knowledge Extraction

25 pages, 1242 KB

Open AccessArticle

Automated Event Detection and Classification in Soccer: The Potential of Using Multiple Modalities

by Olav Andre Nergård Rongved, Markus Stige, Steven Alexander Hicks, Vajira Lasantha Thambawita, Cise Midoglu, Evi Zouganeli, Dag Johansen, Michael Alexander Riegler and Pål Halvorsen

Mach. Learn. Knowl. Extr. 2021, 3(4), 1030-1054; https://doi.org/10.3390/make3040051 - 16 Dec 2021

Cited by 29 | Viewed by 8685

Abstract

Detecting events in videos is a complex task, and many different approaches, aimed at a large variety of use-cases, have been proposed in the literature. Most approaches, however, are unimodal and only consider the visual information in the videos. This paper presents and [...] Read more.

Detecting events in videos is a complex task, and many different approaches, aimed at a large variety of use-cases, have been proposed in the literature. Most approaches, however, are unimodal and only consider the visual information in the videos. This paper presents and evaluates different approaches based on neural networks where we combine visual features with audio features to detect (spot) and classify events in soccer videos. We employ model fusion to combine different modalities such as video and audio, and test these combinations against different state-of-the-art models on the SoccerNet dataset. The results show that a multimodal approach is beneficial. We also analyze how the tolerance for delays in classification and spotting time, and the tolerance for prediction accuracy, influence the results. Our experiments show that using multiple modalities improves event detection performance for certain types of events. Full article

(This article belongs to the Topic Applied Computer Vision and Pattern Recognition)

► Show Figures

Figure 1

21 pages, 5730 KB

Open AccessArticle

Detection and Classification of Knee Injuries from MR Images Using the MRNet Dataset with Progressively Operating Deep Learning Methods

by Ali Can Kara and Fırat Hardalaç

Mach. Learn. Knowl. Extr. 2021, 3(4), 1009-1029; https://doi.org/10.3390/make3040050 - 16 Dec 2021

Cited by 21 | Viewed by 11067

Abstract

This study aimed to build progressively operating deep learning models that could detect meniscus injuries, anterior cruciate ligament (ACL) tears and knee abnormalities in magnetic resonance imaging (MRI). The Stanford Machine Learning Group MRNet dataset was employed in the study, which included MRI [...] Read more.

This study aimed to build progressively operating deep learning models that could detect meniscus injuries, anterior cruciate ligament (ACL) tears and knee abnormalities in magnetic resonance imaging (MRI). The Stanford Machine Learning Group MRNet dataset was employed in the study, which included MRI image indexes in the coronal, sagittal, and axial axes, each having 1130 trains and 120 validation items. The study is divided into three sections. In the first section, suitable images are selected to determine the disease in the image index based on the disturbance under examination. It is also used to identify images that have been misclassified or are noisy and/or damaged to the degree that they cannot be utilised for diagnosis in the first section. The study employed the 50-layer residual networks (ResNet50) model in this section. The second part of the study involves locating the region to be focused on based on the disturbance that is targeted to be diagnosed in the image under examination. A novel model was built by integrating the convolutional neural networks (CNN) and the denoising autoencoder models in the second section. The third section is dedicated to making a diagnosis of the disease. In this section, a novel ResNet50 model is trained to identify disease diagnoses or abnormalities, independent of the ResNet50 model used in the first section. The images that each model selects as output after training are referred to as progressively operating deep learning methods since they are supplied as an input to the following model. Full article

(This article belongs to the Section Learning)

► Show Figures

Figure 1

19 pages, 4788 KB

Open AccessArticle

AI-Based Video Clipping of Soccer Events

by Joakim Olav Valand, Haris Kadragic, Steven Alexander Hicks, Vajira Lasantha Thambawita, Cise Midoglu, Tomas Kupka, Dag Johansen, Michael Alexander Riegler and Pål Halvorsen

Mach. Learn. Knowl. Extr. 2021, 3(4), 990-1008; https://doi.org/10.3390/make3040049 - 8 Dec 2021

Cited by 11 | Viewed by 8600

Abstract

The current gold standard for extracting highlight clips from soccer games is the use of manual annotations and clippings, where human operators define the start and end of an event and trim away the unwanted scenes. This is a tedious, time-consuming, and expensive [...] Read more.

The current gold standard for extracting highlight clips from soccer games is the use of manual annotations and clippings, where human operators define the start and end of an event and trim away the unwanted scenes. This is a tedious, time-consuming, and expensive task, to the extent of being rendered infeasible for use in lower league games. In this paper, we aim to automate the process of highlight generation using logo transition detection, scene boundary detection, and optional scene removal. We experiment with various approaches, using different neural network architectures on different datasets, and present two models that automatically find the appropriate time interval for extracting goal events. These models are evaluated both quantitatively and qualitatively, and the results show that we can detect logo and scene transitions with high accuracy and generate highlight clips that are highly acceptable for viewers. We conclude that there is considerable potential in automating the overall soccer video clipping process. Full article

(This article belongs to the Topic Applied Computer Vision and Pattern Recognition)

► Show Figures

Figure 1

24 pages, 14468 KB

Open AccessPerspective

Analysis of Explainers of Black Box Deep Neural Networks for Computer Vision: A Survey

by Vanessa Buhrmester, David Münch and Michael Arens

Mach. Learn. Knowl. Extr. 2021, 3(4), 966-989; https://doi.org/10.3390/make3040048 - 8 Dec 2021

Cited by 261 | Viewed by 17467

Abstract

Deep Learning is a state-of-the-art technique to make inference on extensive or complex data. As a black box model due to their multilayer nonlinear structure, Deep Neural Networks are often criticized as being non-transparent and their predictions not traceable by humans. Furthermore, the [...] Read more.

Deep Learning is a state-of-the-art technique to make inference on extensive or complex data. As a black box model due to their multilayer nonlinear structure, Deep Neural Networks are often criticized as being non-transparent and their predictions not traceable by humans. Furthermore, the models learn from artificially generated datasets, which often do not reflect reality. By basing decision-making algorithms on Deep Neural Networks, prejudice and unfairness may be promoted unknowingly due to a lack of transparency. Hence, several so-called explanators, or explainers, have been developed. Explainers try to give insight into the inner structure of machine learning black boxes by analyzing the connection between the input and output. In this survey, we present the mechanisms and properties of explaining systems for Deep Neural Networks for Computer Vision tasks. We give a comprehensive overview about the taxonomy of related studies and compare several survey papers that deal with explainability in general. We work out the drawbacks and gaps and summarize further research ideas. Full article

(This article belongs to the Section Thematic Reviews)

► Show Figures

Figure 1

20 pages, 2264 KB

Open AccessArticle

A Novel Feature Representation for Prediction of Global Horizontal Irradiance Using a Bidirectional Model

by Sourav Malakar, Saptarsi Goswami, Bhaswati Ganguli, Amlan Chakrabarti, Sugata Sen Roy, K. Boopathi and A. G. Rangaraj

Mach. Learn. Knowl. Extr. 2021, 3(4), 946-965; https://doi.org/10.3390/make3040047 - 25 Nov 2021

Cited by 1 | Viewed by 3851

Abstract

Complex weather conditions—in particular clouds—leads to uncertainty in photovoltaic (PV) systems, which makes solar energy prediction very difficult. Currently, in the renewable energy domain, deep-learning-based sequence models have reported better results compared to state-of-the-art machine-learning models. There are quite a few choices of [...] Read more.

Complex weather conditions—in particular clouds—leads to uncertainty in photovoltaic (PV) systems, which makes solar energy prediction very difficult. Currently, in the renewable energy domain, deep-learning-based sequence models have reported better results compared to state-of-the-art machine-learning models. There are quite a few choices of deep-learning architectures, among which Bidirectional Gated Recurrent Unit (BGRU) has apparently not been used earlier in the solar energy domain. In this paper, BGRU was used with a new augmented and bidirectional feature representation. The used BGRU network is more generalized as it can handle unequal lengths of forward and backward context. The proposed model produced 59.21%, 37.47%, and 76.80% better prediction accuracy compared to traditional sequence-based, bidirectional models, and some of the established states-of-the-art models. The testbed considered for evaluation of the model is far more comprehensive and reliable considering the variability in the climatic zones and seasons, as compared to some of the recent studies in India. Full article

(This article belongs to the Special Issue Recent Advances in Feature Selection)

► Show Figures

Figure 1

24 pages, 789 KB

Open AccessArticle

Language Semantics Interpretation with an Interaction-Based Recurrent Neural Network

by Shaw-Hwa Lo and Yiqiao Yin

Mach. Learn. Knowl. Extr. 2021, 3(4), 922-945; https://doi.org/10.3390/make3040046 - 19 Nov 2021

Cited by 2 | Viewed by 3421

Abstract

Text classification is a fundamental language task in Natural Language Processing. A variety of sequential models are capable of making good predictions, yet there is a lack of connection between language semantics and prediction results. This paper proposes a novel influence score (I-score), [...] Read more.

Text classification is a fundamental language task in Natural Language Processing. A variety of sequential models are capable of making good predictions, yet there is a lack of connection between language semantics and prediction results. This paper proposes a novel influence score (I-score), a greedy search algorithm, called Backward Dropping Algorithm (BDA), and a novel feature engineering technique called the “dagger technique”. First, the paper proposes to use the novel influence score (I-score) to detect and search for the important language semantics in text documents that are useful for making good predictions in text classification tasks. Next, a greedy search algorithm, called the Backward Dropping Algorithm, is proposed to handle long-term dependencies in the dataset. Moreover, the paper proposes a novel engineering technique called the “dagger technique” that fully preserves the relationship between the explanatory variable and the response variable. The proposed techniques can be further generalized into any feed-forward Artificial Neural Networks (ANNs) and Convolutional Neural Networks (CNNs), and any neural network. A real-world application on the Internet Movie Database (IMDB) is used and the proposed methods are applied to improve prediction performance with an 81% error reduction compared to other popular peers if I-score and “dagger technique” are not implemented. Full article

(This article belongs to the Section Learning)

► Show Figures

Figure 1

22 pages, 923 KB

Open AccessArticle

A Multi-Component Framework for the Analysis and Design of Explainable Artificial Intelligence

by Mi-Young Kim, Shahin Atakishiyev, Housam Khalifa Bashier Babiker, Nawshad Farruque, Randy Goebel, Osmar R. Zaïane, Mohammad-Hossein Motallebi, Juliano Rabelo, Talat Syed, Hengshuai Yao and Peter Chun

Mach. Learn. Knowl. Extr. 2021, 3(4), 900-921; https://doi.org/10.3390/make3040045 - 18 Nov 2021

Cited by 35 | Viewed by 7962

Abstract

The rapid growth of research in explainable artificial intelligence (XAI) follows on two substantial developments. First, the enormous application success of modern machine learning methods, especially deep and reinforcement learning, have created high expectations for industrial, commercial, and social value. Second, the emerging [...] Read more.

The rapid growth of research in explainable artificial intelligence (XAI) follows on two substantial developments. First, the enormous application success of modern machine learning methods, especially deep and reinforcement learning, have created high expectations for industrial, commercial, and social value. Second, the emerging and growing concern for creating ethical and trusted AI systems, including compliance with regulatory principles to ensure transparency and trust. These two threads have created a kind of “perfect storm” of research activity, all motivated to create and deliver any set of tools and techniques to address the XAI demand. As some surveys of current XAI suggest, there is yet to appear a principled framework that respects the literature of explainability in the history of science and which provides a basis for the development of a framework for transparent XAI. We identify four foundational components, including the requirements for (1) explicit explanation knowledge representation, (2) delivery of alternative explanations, (3) adjusting explanations based on knowledge of the explainee, and (4) exploiting the advantage of interactive explanation. With those four components in mind, we intend to provide a strategic inventory of XAI requirements, demonstrate their connection to a basic history of XAI ideas, and then synthesize those ideas into a simple framework that can guide the design of AI systems that require XAI. Full article

(This article belongs to the Special Issue Advances in Explainable Artificial Intelligence (XAI))

► Show Figures

Figure 1

21 pages, 9680 KB

Open AccessArticle

Deep Self-Organizing Map of Convolutional Layers for Clustering and Visualizing Image Data

by Christos Ferles, Yannis Papanikolaou, Stylianos P. Savaidis and Stelios A. Mitilineos

Mach. Learn. Knowl. Extr. 2021, 3(4), 879-899; https://doi.org/10.3390/make3040044 - 14 Nov 2021

Cited by 5 | Viewed by 7483

Abstract

The self-organizing convolutional map (SOCOM) hybridizes convolutional neural networks, self-organizing maps, and gradient backpropagation optimization into a novel integrated unsupervised deep learning model. SOCOM structurally combines, architecturally stacks, and algorithmically fuses its deep/unsupervised learning components. The higher-level representations produced by its underlying convolutional [...] Read more.

The self-organizing convolutional map (SOCOM) hybridizes convolutional neural networks, self-organizing maps, and gradient backpropagation optimization into a novel integrated unsupervised deep learning model. SOCOM structurally combines, architecturally stacks, and algorithmically fuses its deep/unsupervised learning components. The higher-level representations produced by its underlying convolutional deep architecture are embedded in its topologically ordered neural map output. The ensuing unsupervised clustering and visualization operations reflect the model’s degree of synergy between its building blocks and synopsize its range of applications. Clustering results are reported on the STL-10 benchmark dataset coupled with the devised neural map visualizations. The series of conducted experiments utilize a deep VGG-based SOCOM model. Full article

(This article belongs to the Section Visualization)

► Show Figures

Figure 1

16 pages, 759 KB

Open AccessReview

Recent Advances in Deep Reinforcement Learning Applications for Solving Partially Observable Markov Decision Processes (POMDP) Problems Part 2—Applications in Transportation, Industries, Communications and Networking and More Topics

by Xuanchen Xiang, Simon Foo and Huanyu Zang

Mach. Learn. Knowl. Extr. 2021, 3(4), 863-878; https://doi.org/10.3390/make3040043 - 28 Oct 2021

Cited by 7 | Viewed by 8043

Abstract

The two-part series of papers provides a survey on recent advances in Deep Reinforcement Learning (DRL) for solving partially observable Markov decision processes (POMDP) problems. Reinforcement Learning (RL) is an approach to simulate the human’s natural learning process, whose key is to let [...] Read more.

The two-part series of papers provides a survey on recent advances in Deep Reinforcement Learning (DRL) for solving partially observable Markov decision processes (POMDP) problems. Reinforcement Learning (RL) is an approach to simulate the human’s natural learning process, whose key is to let the agent learn by interacting with the stochastic environment. The fact that the agent has limited access to the information of the environment enables AI to be applied efficiently in most fields that require self-learning. It’s essential to have an organized investigation—we can make good comparisons and choose the best structures or algorithms when applying DRL in various applications. The first part of the overview introduces Markov Decision Processes (MDP) problems and Reinforcement Learning and applications of DRL for solving POMDP problems in games, robotics, and natural language processing. In part two, we continue to introduce applications in transportation, industries, communications and networking, etc. and discuss the limitations of DRL. Full article

(This article belongs to the Section Thematic Reviews)

► Show Figures

Figure 1

28 pages, 8130 KB

Open AccessReview

A Review of the Role of Machine Learning Techniques towards Brain–Computer Interface Applications

by Saim Rasheed

Mach. Learn. Knowl. Extr. 2021, 3(4), 835-862; https://doi.org/10.3390/make3040042 - 26 Oct 2021

Cited by 47 | Viewed by 8460

Abstract

This review article provides a deep insight into the Brain–Computer Interface (BCI) and the application of Machine Learning (ML) technology in BCIs. It investigates the various types of research undertaken in this realm and discusses the role played by ML in performing different [...] Read more.

This review article provides a deep insight into the Brain–Computer Interface (BCI) and the application of Machine Learning (ML) technology in BCIs. It investigates the various types of research undertaken in this realm and discusses the role played by ML in performing different BCI tasks. It also reviews the ML methods used for mental state detection, mental task categorization, emotion classification, electroencephalogram (EEG) signal classification, event-related potential (ERP) signal classification, motor imagery categorization, and limb movement classification. This work explores the various methods employed in BCI mechanisms for feature extraction, selection, and classification and provides a comparative study of reviewed methods. This paper assists the readers to gain information regarding the developments made in BCI and ML domains and future improvements needed for improving and designing better BCI applications. Full article

(This article belongs to the Section Thematic Reviews)

► Show Figures

Figure 1

16 pages, 879 KB

Open AccessArticle

Fully Homomorphically Encrypted Deep Learning as a Service

by George Onoufriou, Paul Mayfield and Georgios Leontidis

Mach. Learn. Knowl. Extr. 2021, 3(4), 819-834; https://doi.org/10.3390/make3040041 - 13 Oct 2021

Cited by 15 | Viewed by 4979

Abstract

Fully Homomorphic Encryption (FHE) is a relatively recent advancement in the field of privacy-preserving technologies. FHE allows for the arbitrary depth computation of both addition and multiplication, and thus the application of abelian/polynomial equations, like those found in deep learning algorithms. This project [...] Read more.

Fully Homomorphic Encryption (FHE) is a relatively recent advancement in the field of privacy-preserving technologies. FHE allows for the arbitrary depth computation of both addition and multiplication, and thus the application of abelian/polynomial equations, like those found in deep learning algorithms. This project investigates how FHE with deep learning can be used at scale toward accurate sequence prediction, with a relatively low time complexity, the problems that such a system incurs, and mitigations/solutions for such problems. In addition, we discuss how this could have an impact on the future of data privacy and how it can enable data sharing across various actors in the agri-food supply chain, hence allowing the development of machine learning-based systems. Finally, we find that although FHE incurs a high spatial complexity cost, the run time is within expected reasonable bounds, while allowing for absolutely private predictions to be made, in our case for milk yield prediction with a Mean Absolute Percentage Error (MAPE) of 12.4% and an accuracy of 87.6% on average. Full article

(This article belongs to the Section Privacy)

► Show Figures

Figure 1

17 pages, 2351 KB

Open AccessArticle

Knowledge Graphs Representation for Event-Related E-News Articles

by M.V.P.T. Lakshika and H.A. Caldera

Mach. Learn. Knowl. Extr. 2021, 3(4), 802-818; https://doi.org/10.3390/make3040040 - 26 Sep 2021

Cited by 6 | Viewed by 5936

Abstract

E-newspaper readers are overloaded with massive texts on e-news articles, and they usually mislead the reader who reads and understands information. Thus, there is an urgent need for a technology that can automatically represent the gist of these e-news articles more quickly. Currently, [...] Read more.

E-newspaper readers are overloaded with massive texts on e-news articles, and they usually mislead the reader who reads and understands information. Thus, there is an urgent need for a technology that can automatically represent the gist of these e-news articles more quickly. Currently, popular machine learning approaches have greatly improved presentation accuracy compared to traditional methods, but they cannot be accommodated with the contextual information to acquire higher-level abstraction. Recent research efforts in knowledge representation using graph approaches are neither user-driven nor flexible to deviations in the data. Thus, there is a striking concentration on constructing knowledge graphs by combining the background information related to the subjects in text documents. We propose an enhanced representation of a scalable knowledge graph by automatically extracting the information from the corpus of e-news articles and determine whether a knowledge graph can be used as an efficient application in analyzing and generating knowledge representation from the extracted e-news corpus. This knowledge graph consists of a knowledge base built using triples that automatically produce knowledge representation from e-news articles. Inclusively, it has been observed that the proposed knowledge graph generates a comprehensive and precise knowledge representation for the corpus of e-news articles. Full article

(This article belongs to the Section Visualization)

► Show Figures

Figure 1

14 pages, 1106 KB

Open AccessArticle

An Assessment of the Application of Private Aggregation of Ensemble Models to Sensible Data

by Sergio Yovine, Franz Mayr, Sebastián Sosa and Ramiro Visca

Mach. Learn. Knowl. Extr. 2021, 3(4), 788-801; https://doi.org/10.3390/make3040039 - 25 Sep 2021

Cited by 2 | Viewed by 4706

Abstract

This paper explores the use of Private Aggregation of Teacher Ensembles (PATE) in a setting where students have their own private data that cannot be revealed as is to the ensemble. We propose a privacy model that introduces a local differentially private mechanism [...] Read more.

This paper explores the use of Private Aggregation of Teacher Ensembles (PATE) in a setting where students have their own private data that cannot be revealed as is to the ensemble. We propose a privacy model that introduces a local differentially private mechanism to protect student data. We implemented and analyzed it in case studies from security and health domains, and the result of the experiment was twofold. First, this model does not significantly affecs predictive capabilities, and second, it unveiled interesting issues with the so-called data dependency privacy loss metric, namely, high variance and values. Full article

(This article belongs to the Section Privacy)

► Show Figures

Figure 1

17 pages, 1044 KB

Open AccessArticle

A Critical Study on Stability Measures of Feature Selection with a Novel Extension of Lustgarten Index

by Rikta Sen, Ashis Kumar Mandal and Basabi Chakraborty

Mach. Learn. Knowl. Extr. 2021, 3(4), 771-787; https://doi.org/10.3390/make3040038 - 24 Sep 2021

Cited by 3 | Viewed by 4575

Abstract

Stability of feature selection algorithm refers to its robustness to the perturbations of the training set, parameter settings or initialization. A stable feature selection algorithm is crucial for identifying the relevant feature subset of meaningful and interpretable features which is extremely important in [...] Read more.

Stability of feature selection algorithm refers to its robustness to the perturbations of the training set, parameter settings or initialization. A stable feature selection algorithm is crucial for identifying the relevant feature subset of meaningful and interpretable features which is extremely important in the task of knowledge discovery. Though there are many stability measures reported in the literature for evaluating the stability of feature selection, none of them follows all the requisite properties of a stability measure. Among them, the Kuncheva index and its modifications, are widely used in practical problems. In this work, the merits and limitations of the Kuncheva index and its existing modifications (Lustgarten, Wald, nPOG/nPOGR, Nogueira) are studied and analysed with respect to the requisite properties of stability measure. One more limitation of the most recent modified similarity measure, Nogueira’s measure, has been pointed out. Finally, corrections to Lustgarten’s measure have been proposed to define a new modified stability measure that satisfies the desired properties and overcomes the limitations of existing popular similarity based stability measures. The effectiveness of the newly modified Lustgarten’s measure has been evaluated with simple toy experiments. Full article

(This article belongs to the Special Issue Recent Advances in Feature Selection)

► Show Figures

Figure 1

Journal Menu

Journal Browser

Mach. Learn. Knowl. Extr., Volume 3, Issue 4 (December 2021) – 14 articles

Further Information

Guidelines

MDPI Initiatives

Follow MDPI