Submit to Entropy Review for Entropy Propose a Special Issue

Journal Browser

► Journal Browser

Machine Learning and Entropy: Discover Unknown Unknowns in Complex Data Sets

Print Special Issue Flyer
Special Issue Editors
Special Issue Information
Keywords
Published Papers

A special issue of Entropy (ISSN 1099-4300).

Deadline for manuscript submissions: closed (30 January 2016) | Viewed by 89438

Share This Special Issue

Special Issue Editor

Prof. Dr. Andreas Holzinger

E-Mail Website1 Website2
Guest Editor

1. Human-Centered AI Lab, Institute of Forest Engineering, Department of Forest and Soil Sciences, University of Natural Resources and Life Sciences, 1190 Vienna, Austria
2. xAI Lab, Alberta Machine Intelligence Institute, University of Alberta, Edmonton, AB T5J 3B1, Canada
Interests: artificial intelligence (AI); machine learning (ML); explainable AI (xAI); causability; decision support systems; medical AI; health informatics
Special Issues, Collections and Topics in MDPI journals

Special Issue Information

Dear Colleagues,

In the real world, we are confronted, not only with complex and high-dimensional data sets, but also usually with noisy, incomplete, and uncertain data, where the application of traditional methods of knowledge discovery and data mining always entail the danger of modeling artifacts. Originally, information entropy was introduced by Shannon (1949), as a measure of uncertainty in data. Up to the present, many different types of entropy methods with a large number of different purposes and possible application areas have emerged. In this Special Issue we are seeking papers discussing advances in the application of learning algorithms and entropy for use in knowledge discovery and data mining, to discover unknowns in complex data sets, e.g., for biomarker discovery in biomedical data sets.

Prof. Dr. Andreas Holzinger
Guest Editor

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Entropy is an international peer-reviewed open access monthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2600 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

Machine Learning
Knowledge Discovery
Entropy-based Data Mining

Published Papers (13 papers)

Download All Papers

Research

1057 KiB

Open AccessArticle

Distribution Entropy Boosted VLAD for Image Retrieval

by Qiuzhan Zhou, Cheng Wang, Pingping Liu, Qingliang Li, Yeran Wang and Shuozhang Chen

Entropy 2016, 18(8), 311; https://doi.org/10.3390/e18080311 - 24 Aug 2016

Cited by 11 | Viewed by 4815

Abstract

Several recent works have shown that aggregating local descriptors to generate global image representation results in great efficiency for retrieval and classification tasks. The most popular method following this approach is VLAD (Vector of Locally Aggregated Descriptors). We present a novel image presentation called Distribution Entropy Boosted VLAD (EVLAD), which extends the original vector of locally aggregated descriptors. The original VLAD adopts only residuals to depict the distribution information of every visual word and neglects other statistical clues, so its discriminative power is limited. To address this issue, this paper proposes the use of the distribution entropy of each cluster as supplementary information to enhance the search accuracy. To fuse two feature sources organically, two fusion methods after a new normalization stage meeting power law are also investigated, which generate identically sized and double-sized vectors as the original VLAD. We validate our approach in image retrieval and image classification experiments. Experimental results demonstrate the effectiveness of our algorithm. Full article

(This article belongs to the Special Issue Machine Learning and Entropy: Discover Unknown Unknowns in Complex Data Sets)

► Show Figures

Figure 1

4576 KiB

Open AccessArticle

Voice Activity Detection Using Fuzzy Entropy and Support Vector Machine

by R. Johny Elton, P. Vasuki and J. Mohanalin

Entropy 2016, 18(8), 298; https://doi.org/10.3390/e18080298 - 12 Aug 2016

Cited by 20 | Viewed by 6865

Abstract

This paper proposes support vector machine (SVM) based voice activity detection using FuzzyEn to improve detection performance under noisy conditions. The proposed voice activity detection (VAD) uses fuzzy entropy (FuzzyEn) as a feature extracted from noise-reduced speech signals to train an SVM model for speech/non-speech classification. The proposed VAD method was tested by conducting various experiments by adding real background noises of different signal-to-noise ratios (SNR) ranging from −10 dB to 10 dB to actual speech signals collected from the TIMIT database. The analysis proves that FuzzyEn feature shows better results in discriminating noise and corrupted noisy speech. The efficacy of the SVM classifier was validated using 10-fold cross validation. Furthermore, the results obtained by the proposed method was compared with those of previous standardized VAD algorithms as well as recently developed methods. Performance comparison suggests that the proposed method is proven to be more efficient in detecting speech under various noisy environments with an accuracy of 93.29%, and the FuzzyEn feature detects speech efficiently even at low SNR levels. Full article

(This article belongs to the Special Issue Machine Learning and Entropy: Discover Unknown Unknowns in Complex Data Sets)

► Show Figures

Graphical abstract

2969 KiB

Open AccessArticle

How Is a Data-Driven Approach Better than Random Choice in Label Space Division for Multi-Label Classification?

by Piotr Szymański, Tomasz Kajdanowicz and Kristian Kersting

Entropy 2016, 18(8), 282; https://doi.org/10.3390/e18080282 - 30 Jul 2016

Cited by 38 | Viewed by 11695

Abstract

We propose using five data-driven community detection approaches from social networks to partition the label space in the task of multi-label classification as an alternative to random partitioning into equal subsets as performed by RAkELd. We evaluate modularity-maximizing using fast greedy and leading eigenvector approximations, infomap, walktrap and label propagation algorithms. For this purpose, we propose to construct a label co-occurrence graph (both weighted and unweighted versions) based on training data and perform community detection to partition the label set. Then, each partition constitutes a label space for separate multi-label classification sub-problems. As a result, we obtain an ensemble of multi-label classifiers that jointly covers the whole label space. Based on the binary relevance and label powerset classification methods, we compare community detection methods to label space divisions against random baselines on 12 benchmark datasets over five evaluation measures. We discover that data-driven approaches are more efficient and more likely to outperform RAkELd than binary relevance or label powerset is, in every evaluated measure. For all measures, apart from Hamming loss, data-driven approaches are significantly better than RAkELd (

α = 0.05

), and at least one data-driven approach is more likely to outperform RAkELd than a priori methods in the case of RAkELd’s best performance. This is the largest RAkELd evaluation published to date with 250 samplings per value for 10 values of RAkELd parameter k on 12 datasets published to date. Full article

(This article belongs to the Special Issue Machine Learning and Entropy: Discover Unknown Unknowns in Complex Data Sets)

► Show Figures

Graphical abstract

604 KiB

Open AccessArticle

A PUT-Based Approach to Automatically Extracting Quantities and Generating Final Answers for Numerical Attributes

by Yaqing Liu, Lidong Wang, Rong Chen, Yingjie Song and Yalin Cai

Entropy 2016, 18(6), 235; https://doi.org/10.3390/e18060235 - 22 Jun 2016

Cited by 5 | Viewed by 4333

Abstract

Automatically extracting quantities and generating final answers for numerical attributes is very useful in many occasions, including question answering, image processing, human-computer interaction, etc. A common approach is to learn linguistics templates or wrappers and employ some algorithm or model to generate a final answer. However, building linguistics templates or wrappers is a tough task for builders. In addition, linguistics templates or wrappers are domain-dependent. To make the builder escape from building linguistics templates or wrappers, we propose a new approach to final answer generation based on Predicates-Units Table (PUT), a mini domain-independent knowledge base. It is deserved to point out that, in the following cases, quantities are not represented well. Quantities are absent of units. Quantities are perhaps wrong for a given question. Even if all of them are represented well, their units are perhaps inconsistent. These cases have a strong impact on final answer solving. One thousand nine hundred twenty-six real queries are employed to test the proposed method, and the experimental results show that the average correctness ratio of our approach is 87.1%. Full article

(This article belongs to the Special Issue Machine Learning and Entropy: Discover Unknown Unknowns in Complex Data Sets)

► Show Figures

Graphical abstract

1897 KiB

Open AccessArticle

Stimuli-Magnitude-Adaptive Sample Selection for Data-Driven Haptic Modeling

by Arsen Abdulali, Waseem Hassan and Seokhee Jeon

Entropy 2016, 18(6), 222; https://doi.org/10.3390/e18060222 - 07 Jun 2016

Cited by 9 | Viewed by 5238

Abstract

Data-driven haptic modeling is an emerging technique where contact dynamics are simulated and interpolated based on a generic input-output matching model identified by data sensed from interaction with target physical objects. In data-driven modeling, selecting representative samples from a large set of data in a way that they can efficiently and accurately describe the whole dataset has been a long standing problem. This paper presents a new algorithm for the sample selection where the variances of output are observed for selecting representative input-output samples in order to ensure the quality of output prediction. The main idea is that representative pairs of input-output are chosen so that the ratio of the standard deviation to the mean of the corresponding output group does not exceed an application-dependent threshold. This output- and standard deviation-based sample selection is very effective in applications where the variance or relative error of the output should be kept within a certain threshold. This threshold is used for partitioning the input space using Binary Space Partitioning-tree (BSP-tree) and k-means algorithms. We apply the new approach to data-driven haptic modeling scenario where the relative error of the output prediction result should be less than a perceptual threshold. For evaluation, the proposed algorithm is compared to two state-of-the-art sample selection algorithms for regression tasks. Four kinds of haptic related behavior–force datasets are tested. The results showed that the proposed algorithm outperformed the others in terms of output-approximation quality and computational complexity. Full article

(This article belongs to the Special Issue Machine Learning and Entropy: Discover Unknown Unknowns in Complex Data Sets)

► Show Figures

Graphical abstract

700 KiB

Open AccessArticle

A Conjecture Regarding the Extremal Values of Graph Entropy Based on Degree Powers

by Kinkar Chandra Das and Matthias Dehmer

Entropy 2016, 18(5), 183; https://doi.org/10.3390/e18050183 - 13 May 2016

Cited by 6 | Viewed by 4079

Abstract

Many graph invariants have been used for the construction of entropy-based measures to characterize the structure of complex networks. The starting point has been always based on assigning a probability distribution to a network when using Shannon’s entropy. In particular, Cao et al. (2014 and 2015) defined special graph entropy measures which are based on degrees powers. In this paper, we obtain some lower and upper bounds for these measures and characterize extremal graphs. Moreover we resolve one part of a conjecture stated by Cao et al. Full article

(This article belongs to the Special Issue Machine Learning and Entropy: Discover Unknown Unknowns in Complex Data Sets)

4680 KiB

Open AccessArticle

Estimation of Tsunami Bore Forces on a Coastal Bridge Using an Extreme Learning Machine

by Iman Mazinani, Zubaidah Binti Ismail, Shahaboddin Shamshirband, Ahmad Mustafa Hashim, Marjan Mansourvar and Erfan Zalnezhad

Entropy 2016, 18(5), 167; https://doi.org/10.3390/e18050167 - 28 Apr 2016

Cited by 18 | Viewed by 5934

Abstract

This paper proposes a procedure to estimate tsunami wave forces on coastal bridges through a novel method based on Extreme Learning Machine (ELM) and laboratory experiments. This research included three water depths, ten wave heights, and four bridge models with a variety of girders providing a total of 120 cases. The research was designed and adapted to estimate tsunami bore forces including horizontal force, vertical uplift and overturning moment on a coastal bridge. The experiments were carried out on 1:40 scaled concrete bridge models in a wave flume with dimensions of 24 m × 1.5 m × 2 m. Two six-axis load cells and four pressure sensors were installed to the base plate to measure forces. In the numerical procedure, estimation and prediction results of the ELM model were compared with Genetic Programming (GP) and Artificial Neural Networks (ANNs) models. The experimental results showed an improvement in predictive accuracy, and capability of generalization could be achieved by the ELM approach in comparison with GP and ANN. Moreover, results indicated that the ELM models developed could be used with confidence for further work on formulating novel model predictive strategy for tsunami bore forces on a coastal bridge. The experimental results indicated that the new algorithm could produce good generalization performance in most cases and could learn thousands of times faster than conventional popular learning algorithms. Therefore, it can be conclusively obtained that utilization of ELM is certainly developing as an alternative approach to estimate the tsunami bore forces on a coastal bridge. Full article

(This article belongs to the Special Issue Machine Learning and Entropy: Discover Unknown Unknowns in Complex Data Sets)

► Show Figures

Figure 1

396 KiB

Open AccessArticle

Finding Influential Users in Social Media Using Association Rule Learning

by Fredrik Erlandsson, Piotr Bródka, Anton Borg and Henric Johnson

Entropy 2016, 18(5), 164; https://doi.org/10.3390/e18050164 - 27 Apr 2016

Cited by 71 | Viewed by 8282

Abstract

Influential users play an important role in online social networks since users tend to have an impact on one other. Therefore, the proposed work analyzes users and their behavior in order to identify influential users and predict user participation. Normally, the success of a social media site is dependent on the activity level of the participating users. For both online social networking sites and individual users, it is of interest to find out if a topic will be interesting or not. In this article, we propose association learning to detect relationships between users. In order to verify the findings, several experiments were executed based on social network analysis, in which the most influential users identified from association rule learning were compared to the results from Degree Centrality and Page Rank Centrality. The results clearly indicate that it is possible to identify the most influential users using association rule learning. In addition, the results also indicate a lower execution time compared to state-of-the-art methods. Full article

(This article belongs to the Special Issue Machine Learning and Entropy: Discover Unknown Unknowns in Complex Data Sets)

► Show Figures

Figure 1

6825 KiB

Open AccessArticle

An Informed Framework for Training Classifiers from Social Media

by Dong Seon Cheng and Sami Abduljalil Abdulhak

Entropy 2016, 18(4), 130; https://doi.org/10.3390/e18040130 - 09 Apr 2016

Cited by 1 | Viewed by 4322

Abstract

Extracting information from social media has become a major focus of companies and researchers in recent years. Aside from the study of the social aspects, it has also been found feasible to exploit the collaborative strength of crowds to help solve classical machine learning problems like object recognition. In this work, we focus on the generally underappreciated problem of building effective datasets for training classifiers by automatically assembling data from social media. We detail some of the challenges of this approach and outline a framework that uses expanded search queries to retrieve more qualified data. In particular, we concentrate on collaboratively tagged media on the social platform Flickr, and on the problem of image classification to evaluate our approach. Finally, we describe a novel entropy-based method to incorporate an information-theoretic principle to guide our framework. Experimental validation against well-known public datasets shows the viability of this approach and marks an improvement over the state of the art in terms of simplicity and performance. Full article

(This article belongs to the Special Issue Machine Learning and Entropy: Discover Unknown Unknowns in Complex Data Sets)

► Show Figures

Figure 1

683 KiB

Open AccessArticle

The Effect of Threshold Values and Weighting Factors on the Association between Entropy Measures and Mortality after Myocardial Infarction in the Cardiac Arrhythmia Suppression Trial (CAST)

by Christopher Mayer, Martin Bachler, Andreas Holzinger, Phyllis K. Stein and Siegfried Wassertheurer

Entropy 2016, 18(4), 129; https://doi.org/10.3390/e18040129 - 08 Apr 2016

Cited by 19 | Viewed by 5600

Abstract

Heart rate variability (HRV) is a non-invasive measurement based on the intervals between normal heart beats that characterize cardiac autonomic function. Decreased HRV is associated with increased risk of cardiovascular events. Characterizing HRV using only moment statistics fails to capture abnormalities in regulatory function that are important aspects of disease risk. Thus, entropy measures are a promising approach to quantify HRV for risk stratification. The purpose of this study was to investigate this potential for approximate, corrected approximate, sample, fuzzy, and fuzzy measure entropy and its dependency on the parameter selection. Recently, published parameter sets and further parameter combinations were investigated. Heart rate data were obtained from the "Cardiac Arrhythmia Suppression Trial (CAST) RR Interval Sub-Study Database" (Physionet). Corresponding outcomes and clinical data were provided by one of the investigators. The use of previously-reported parameter sets on the pre-treatment data did not significantly add to the identification of patients at risk for cardiovascular death on follow-up. After arrhythmia suppression treatment, several parameter sets predicted outcomes for all patients and patients without coronary artery bypass grafting (CABG). The strongest results were seen using the threshold parameter as a multiple of the data’s standard deviation (

r = 0.2 \cdot σ

). Approximate and sample entropy provided significant hazard ratios for patients without CABG and without diabetes for an entropy maximizing threshold approximation. Additional parameter combinations did not improve the results for pre-treatment data. The results of this study illustrate the influence of parameter selection on entropy measures’ potential for cardiovascular risk stratification and support the potential use of entropy measures in future studies. Full article

(This article belongs to the Special Issue Machine Learning and Entropy: Discover Unknown Unknowns in Complex Data Sets)

► Show Figures

Figure 1

2212 KiB

Open AccessArticle

Feature Selection of Power Quality Disturbance Signals with an Entropy-Importance-Based Random Forest

by Nantian Huang, Guobo Lu, Guowei Cai, Dianguo Xu, Jiafeng Xu, Fuqing Li and Liying Zhang

Entropy 2016, 18(2), 44; https://doi.org/10.3390/e18020044 - 28 Jan 2016

Cited by 37 | Viewed by 6340

Abstract

Power quality signal feature selection is an effective method to improve the accuracy and efficiency of power quality (PQ) disturbance classification. In this paper, an entropy-importance (EnI)-based random forest (RF) model for PQ feature selection and disturbance classification is proposed. Firstly, 35 kinds of signal features extracted from S-transform (ST) with random noise are used as the original input feature vector of RF classifier to recognize 15 kinds of PQ signals with six kinds of complex disturbance. During the RF training process, the classification ability of different features is quantified by EnI. Secondly, without considering the features with zero EnI, the optimal perturbation feature subset is obtained by applying the sequential forward search (SFS) method which considers the classification accuracy and feature dimension. Then, the reconstructed RF classifier is applied to identify disturbances. According to the simulation results, the classification accuracy is higher than that of other classifiers, and the feature selection effect of the new approach is better than SFS and sequential backward search (SBS) without EnI. With the same feature subset, the new method can maintain a classification accuracy above 99.7% under the condition of 30 dB or above, and the accuracy under 20 dB is 96.8%. Full article

(This article belongs to the Special Issue Machine Learning and Entropy: Discover Unknown Unknowns in Complex Data Sets)

► Show Figures

Figure 1

2211 KiB

Open AccessArticle

Using Multidimensional ADTPE and SVM for Optical Modulation Real-Time Recognition

by Junyu Wei, Zhiping Huang, Shaojing Su and Zhen Zuo

Entropy 2016, 18(1), 30; https://doi.org/10.3390/e18010030 - 16 Jan 2016

Cited by 6 | Viewed by 5230

Abstract

Based on the feature extraction of multidimensional asynchronous delay-tap plot entropy (ADTPE) and multiclass classification of support vector machine (SVM), we propose a method for recognition of multiple optical modulation formats and various data rates. We firstly present the algorithm of multidimensional ADTPE, which is extracted from asynchronous delay sampling pairs of modulated optical signal. Then, a multiclass SVM is utilized for fast and accurate classification of several widely-used optical modulation formats. In addition, a simple real-time recognition scheme is designed to reduce the computation time. Compared to the existing method based on asynchronous delay-tap plot (ADTP), the theoretical analysis and simulation results show that our recognition method can effectively enhance the tolerance of transmission impairments, obtaining relatively high accuracy. Finally, it is further demonstrated that the proposed method can be integrated in an optical transport network (OTN) with flexible expansion. Through simply adding the corresponding sub-SVM module in the digital signal processer (DSP), arbitrary new modulation formats can be recognized with high recognition accuracy in a short response time. Full article

(This article belongs to the Special Issue Machine Learning and Entropy: Discover Unknown Unknowns in Complex Data Sets)

► Show Figures

Figure 1

1374 KiB

Open AccessArticle

Fruit Classification by Wavelet-Entropy and Feedforward Neural Network Trained by Fitness-Scaled Chaotic ABC and Biogeography-Based Optimization

by Shuihua Wang, Yudong Zhang, Genlin Ji, Jiquan Yang, Jianguo Wu and Ling Wei

Entropy 2015, 17(8), 5711-5728; https://doi.org/10.3390/e17085711 - 07 Aug 2015

Cited by 131 | Viewed by 14635

Abstract

Fruit classification is quite difficult because of the various categories and similar shapes and features of fruit. In this work, we proposed two novel machine-learning based classification methods. The developed system consists of wavelet entropy (WE), principal component analysis (PCA), feedforward neural network (FNN) trained by fitness-scaled chaotic artificial bee colony (FSCABC) and biogeography-based optimization (BBO), respectively. The K-fold stratified cross validation (SCV) was utilized for statistical analysis. The classification performance for 1653 fruit images from 18 categories showed that the proposed “WE + PCA + FSCABC-FNN” and “WE + PCA + BBO-FNN” methods achieve the same accuracy of 89.5%, higher than state-of-the-art approaches: “(CH + MP + US) + PCA + GA-FNN ” of 84.8%, “(CH + MP + US) + PCA + PSO-FNN” of 87.9%, “(CH + MP + US) + PCA + ABC-FNN” of 85.4%, “(CH + MP + US) + PCA + kSVM” of 88.2%, and “(CH + MP + US) + PCA + FSCABC-FNN” of 89.1%. Besides, our methods used only 12 features, less than the number of features used by other methods. Therefore, the proposed methods are effective for fruit classification. Full article

(This article belongs to the Special Issue Machine Learning and Entropy: Discover Unknown Unknowns in Complex Data Sets)

► Show Figures

Figure 1

Journal Menu

Journal Browser

Machine Learning and Entropy: Discover Unknown Unknowns in Complex Data Sets

Share This Special Issue

Special Issue Editor

Special Issue Information

Keywords

Published Papers (13 papers)

Research

Further Information

Guidelines

MDPI Initiatives

Follow MDPI