entropy-logo

Journal Browser

Journal Browser

Information Theory Based Error and Regularization Functions in Artificial Intelligence Algorithms in the Big Data Era

A special issue of Entropy (ISSN 1099-4300). This special issue belongs to the section "Information Theory, Probability and Statistics".

Deadline for manuscript submissions: closed (15 December 2019) | Viewed by 8581

Special Issue Editors


E-Mail Website
Guest Editor
Department of Informatics, Warsaw University of Life Sciences, 02-776 Warsaw, Poland
Interests: machine learning; computational intelligence; data science; data mining; smart metering
Special Issues, Collections and Topics in MDPI journals

E-Mail Website
Guest Editor
UCLA–Computer Science Department, Engineering VI Room 368A, 404 Westwood Plaza Los Angeles, CA 90095-1596, USA
Interests: machine learning (statistical relational learning, tractable learning), knowledge representation and reasoning (graphical models, lifted probabilistic inference, knowledge compilation), applications of probabilistic reasoning and learning (probabilistic programming, probabilistic databases), and artificial intelligence

Special Issue Information

Dear Colleagues,

We all live in the era of emerging technological advancements. The days when almost everything was done manually are gone, and now we live in the time in which a lot of activities have been taken over by machines, software, and automatic processes. In this context, artificial intelligence (AI) has a special place in all the advancements that have been made. AI is applying science to computers and machines to develop intelligence like humans have. With this technology, machines are able to do some of the simple to complex tasks that humans do on a regular basis.

Today, the amount of data is exploding at an unprecedented rate as a result of developments in Web technologies, social media, and mobile and sensing devices. The concept of big data is defined by Gartner as high volume, high velocity, high variety, and high veracity data that require new processing paradigms to enable insight discovery, improved decision making, and process optimization. The potential of big data is highlighted by their definition; however, the realization of this potential depends on improving traditional approaches or developing new ones capable of handling such data. Because of their potential, big data have been referred to as a revolution that will transform how we live, work, and think. The main purpose of this revolution is to make use of large amounts of data to enable knowledge discovery and better decision making.

The ability to extract value from big data depends on data analytics, which can be done using AI systems. While AI provides significant support in various areas such as time series forecasting, fraud detection, and image recognition, the road to the excellence is long. This is because AI has not been able to overcome a number of challenges—especially in the big data era—that still stand in the way of progress.

The main scope of this Special Issue is to propose new methods that are applicable for various artificial intelligence algorithms, thus improving their quality, robustness, and handling of big data. This would be possible as a result of application of the novel and nonstandard error and regularization information theory-based functions in artificial intelligence algorithms.

Dr. Krzysztof Gajowniczek
Prof. Dr. Guy Van den Broeck
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Entropy is an international peer-reviewed open access monthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2600 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

  • Artificial intelligence
  • Big data
  • Computational intelligence
  • Data mining
  • Entopy
  • Information theory
  • Learning theory

Published Papers (2 papers)

Order results
Result details
Select all
Export citation of selected articles as:

Research

17 pages, 2084 KiB  
Article
Semantic and Generalized Entropy Loss Functions for Semi-Supervised Deep Learning
by Krzysztof Gajowniczek, Yitao Liang, Tal Friedman, Tomasz Ząbkowski and Guy Van den Broeck
Entropy 2020, 22(3), 334; https://doi.org/10.3390/e22030334 - 14 Mar 2020
Cited by 15 | Viewed by 4046
Abstract
The increasing size of modern datasets combined with the difficulty of obtaining real label information (e.g., class) has made semi-supervised learning a problem of considerable practical importance in modern data analysis. Semi-supervised learning is supervised learning with additional information on the distribution of [...] Read more.
The increasing size of modern datasets combined with the difficulty of obtaining real label information (e.g., class) has made semi-supervised learning a problem of considerable practical importance in modern data analysis. Semi-supervised learning is supervised learning with additional information on the distribution of the examples or, simultaneously, an extension of unsupervised learning guided by some constraints. In this article we present a methodology that bridges between artificial neural network output vectors and logical constraints. In order to do this, we present a semantic loss function and a generalized entropy loss function (Rényi entropy) that capture how close the neural network is to satisfying the constraints on its output. Our methods are intended to be generally applicable and compatible with any feedforward neural network. Therefore, the semantic loss and generalized entropy loss are simply a regularization term that can be directly plugged into an existing loss function. We evaluate our methodology over an artificially simulated dataset and two commonly used benchmark datasets which are MNIST and Fashion-MNIST to assess the relation between the analyzed loss functions and the influence of the various input and tuning parameters on the classification accuracy. The experimental evaluation shows that both losses effectively guide the learner to achieve (near-) state-of-the-art results on semi-supervised multiclass classification. Full article
Show Figures

Figure 1

36 pages, 1308 KiB  
Article
Spherical Fuzzy Logarithmic Aggregation Operators Based on Entropy and Their Application in Decision Support Systems
by Yun Jin, Shahzaib Ashraf and Saleem Abdullah
Entropy 2019, 21(7), 628; https://doi.org/10.3390/e21070628 - 26 Jun 2019
Cited by 97 | Viewed by 4180
Abstract
Keeping in view the importance of new defined and well growing spherical fuzzy sets, in this study, we proposed a novel method to handle the spherical fuzzy multi-criteria group decision-making (MCGDM) problems. Firstly, we presented some novel logarithmic operations of spherical fuzzy sets [...] Read more.
Keeping in view the importance of new defined and well growing spherical fuzzy sets, in this study, we proposed a novel method to handle the spherical fuzzy multi-criteria group decision-making (MCGDM) problems. Firstly, we presented some novel logarithmic operations of spherical fuzzy sets (SFSs). Then, we proposed series of novel logarithmic operators, namely spherical fuzzy weighted average operators and spherical fuzzy weighted geometric operators. We proposed the spherical fuzzy entropy to find the unknown weights information of the criteria. We study some of its desirable properties such as idempotency, boundary and monotonicity in detail. Finally, the detailed steps for the spherical fuzzy decision-making problems were developed, and a practical case was given to check the created approach and to illustrate its validity and superiority. Besides this, a systematic comparison analysis with other existent methods is conducted to reveal the advantages of our proposed method. Results indicate that the proposed method is suitable and effective for the decision process to evaluate their best alternative. Full article
Show Figures

Figure 1

Back to TopTop