Next Article in Journal
Conditional Values in Quantum Mechanics
Previous Article in Journal
Information Bottleneck Driven Deep Video Compression—IBOpenDVCW
Previous Article in Special Issue
Inferring Dealer Networks in the Foreign Exchange Market Using Conditional Transfer Entropy: Analysis of a Central Bank Announcement
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
This is an early access version, the complete PDF, HTML, and XML versions will be available soon.
Article

Bias in O-Information Estimation

by
Johanna Gehlen
1,*,
Jie Li
1,
Cillian Hourican
1,
Stavroula Tassi
2,3,
Pashupati P. Mishra
4,5,6,
Terho Lehtimäki
4,5,6,
Mika Kähönen
5,7,
Olli Raitakari
8,9,10,11,
Jos A. Bosch
12 and
Rick Quax
1,13
1
Computational Science Lab, Informatics Institute, University of Amsterdam, 1105 Amsterdam, The Netherlands
2
Unit of Medical Technology and Intelligent Information Systems (MEDLAB), Department of Material Science and Engineering, University of Ioannina, 45110 Ioannina, Greece
3
Department of Mechanical and Aeronautics Engineering, University of Patras, 26500 Patras, Greece
4
Department of Clinical Chemistry, Faculty of Medicine and Health Technology, Tampere University, 33720 Tampere, Finland
5
Finnish Cardiovascular Research Center Tampere, Faculty of Medicine and Health Technology, Tampere University, 33720 Tampere, Finland
6
Fimlab Laboratories, Department of Clinical Chemistry, 33520 Tampere, Finland
7
Department of Clinical Physiology, Tampere University Hospital, 33520 Tampere, Finland
8
Centre for Population Health Research, University of Turku and Turku University Hospital, 20014 Turku, Finland
9
Research Centre of Applied and Preventive Cardiovascular Medicine, University of Turku, 20520 Turku, Finland
10
Department of Clinical Physiology and Nuclear Medicine, Turku University Hospital, 20520 Turku, Finland
11
InFLAMES Research Flagship, University of Turku, 20520 Turku, Finland
12
Clinical Psychology, Faculty of Social and Behavioural Sciences, University of Amsterdam, 1105 Amsterdam, The Netherlands
13
Institute for Advanced Study, 1012 Amsterdam, The Netherlands
*
Author to whom correspondence should be addressed.
Entropy 2024, 26(10), 837; https://doi.org/10.3390/e26100837
Submission received: 5 August 2024 / Revised: 18 September 2024 / Accepted: 24 September 2024 / Published: 30 September 2024

Abstract

Higher-order relationships are a central concept in the science of complex systems. A popular method of attempting to estimate the higher-order relationships of synergy and redundancy from data is through O-information. It is an information–theoretic measure composed of Shannon entropy terms that quantifies the balance between redundancy and synergy in a system. However, bias is not yet taken into account in the estimation of the O-information of discrete variables. In this paper, we explain where this bias comes from and explore it for fully synergistic, fully redundant, and fully independent simulated systems of n=3 variables. Specifically, we explore how the sample size and number of bins affect the bias in the O-information estimation. The main finding is that the O-information of independent systems is severely biased towards synergy if the sample size is smaller than the number of jointly possible observations. This could mean that triplets identified as highly synergistic may in fact be close to independent. A bias approximation based on the Miller–Maddow method is derived for O-information. We find that for systems of n=3 variables the bias approximation can partially correct for the bias. However, simulations of fully independent systems are still required as null models to provide a benchmark of the bias of O-information.
Keywords: higher-order relationships; O-information; information synergy; bias; complex systems higher-order relationships; O-information; information synergy; bias; complex systems

Share and Cite

MDPI and ACS Style

Gehlen, J.; Li, J.; Hourican, C.; Tassi, S.; Mishra, P.P.; Lehtimäki, T.; Kähönen, M.; Raitakari, O.; Bosch, J.A.; Quax, R. Bias in O-Information Estimation. Entropy 2024, 26, 837. https://doi.org/10.3390/e26100837

AMA Style

Gehlen J, Li J, Hourican C, Tassi S, Mishra PP, Lehtimäki T, Kähönen M, Raitakari O, Bosch JA, Quax R. Bias in O-Information Estimation. Entropy. 2024; 26(10):837. https://doi.org/10.3390/e26100837

Chicago/Turabian Style

Gehlen, Johanna, Jie Li, Cillian Hourican, Stavroula Tassi, Pashupati P. Mishra, Terho Lehtimäki, Mika Kähönen, Olli Raitakari, Jos A. Bosch, and Rick Quax. 2024. "Bias in O-Information Estimation" Entropy 26, no. 10: 837. https://doi.org/10.3390/e26100837

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop