Reprint
Information Bottleneck
Theory and Applications in Deep Learning
Edited by
June 2021
274 pages
- ISBN978-3-0365-0802-3 (Hardback)
- ISBN978-3-0365-0803-0 (PDF)
This book is a reprint of the Special Issue Information Bottleneck: Theory and Applications in Deep Learning that was published in
Chemistry & Materials Science
Computer Science & Mathematics
Physical Sciences
Summary
The celebrated information bottleneck (IB) principle of Tishby et al. has recently enjoyed renewed attention due to its application in the area of deep learning. This collection investigates the IB principle in this new context. The individual chapters in this collection: • provide novel insights into the functional properties of the IB; • discuss the IB principle (and its derivates) as an objective for training multi-layer machine learning structures such as neural networks and decision trees; and • offer a new perspective on neural network learning via the lens of the IB framework. Our collection thus contributes to a better understanding of the IB principle specifically for deep learning and, more generally, of information–theoretic cost functions in machine learning. This paves the way toward explainable artificial intelligence.
Format
- Hardback
License and Copyright
© 2022 by the authors; CC BY-NC-ND license
Keywords
information theory; variational inference; machine learning; learnability; information bottleneck; representation learning; conspicuous subset; information bottleneck; stochastic neural networks; variational inference; machine learning; information bottleneck; mutual information; representation learning; neural networks; information; bottleneck; compression; classification; information bottleneck; representation learning; mutual information; optimization; information theory; information bottleneck; classifier; decision tree; ensemble; deep neural networks; information bottleneck; regularization methods; information bottleneck principle; deep networks; semi-supervised classification; latent space representation; hand crafted priors; learnable priors; regularization; information theory; information bottleneck; machine learning; information theory; information bottleneck; machine learning; information bottleneck; deep learning; neural networks; information bottleneck; deep learning; neural networks