Information and Self-Organization

The process of “self-organization” takes place in open and complex systems that acquire spatio-temporal or functional structures without specific ordering instructions from the outside. In domains such as physics, chemistry or biology, the phrase, “far from equilibrium”, refers to systems that are “far from thermal equilibrium”, while in other disciplines, the term refers to the property of being “away from the resting state”. Such systems are “complex” in the sense that they are composed of many interacting components, parts, elements, etc., and “open” in the sense that they exchange with their environment matter, energy, and information. Here, “information” may imply Shannon information [1], as a measure of the capacity of a channel through which a message passes, pragmatic information, as the impact of a message on recipients, or semantic information, as the meaning conveyed by a message.

An attempt to bring these lines of thought together was made by Hermann Haken in his 1988 book Information and Self-Organization [2]. In the meantime, a number of authors have studied the interplay between information and self-organization in a variety of fields. Though the selection of the relevant authors and topics is surely not complete, we believe that this special issue mirrors the state of these interdisciplinary approaches fairly well. In fact, the various papers of this Special Issue expose the different ways processes of self-organization are linked with the various forms of information. As will be seen below, a study of such links has consequences on a number of research domains, ranging from physics and chemistry, through the life sciences and cognitive science, including human behavior and action, to our understanding of society, economics, and the dynamics of cities and urbanization.

As will be seen below, the contributions to this Special Issue shed light on the various facets of information and self-organization. And since these various facets do not lend themselves to a topic-oriented order, and since a reader may prefer one over another, we present the papers in an alphabetic order that follows the family name of the first author of each article.

The Contributions

A central theme in the theories of complexity is the self-organized bottom-up transition from the local micro-scale of a system’s elementary parts to the global macro-scale of the system as a whole. In a way, the various theories of complexity differ in their conceptualization of this scale-dependent transition. Harald Atmanspacher [3] approaches this issue from the perspective of his notions of contextual emergence and the associated process of partitioning; both notions are related to the property that complex systems are open in the sense that they exchange matter, energy and information with their environment. The implication, according to Atmanspacher, is that any definition or measure of complexity requires, in addition to generality, also contextuality, that is, taking into consideration the role of the actual system’s environment (including measuring instruments). This contextuality in its turn emerges by the utilization of lower level features as necessary (but not sufficient) conditions for the description of higher-level features—a process based on suitably generated partitions of microstates. Meaningful macro-states thus emerge from partitions of the space of microstates. But then, what is a meaningful macro-state and how is meaning related to complexity?

Atmanspacher responds to the above questions by reference to Grassberger’s and Atlan’s views that the concept of complexity is essentially subjective and can only be defined by reference to an observer. From this epistemological view of complexity follows the challenge “to understand the relation between the complexity of a pattern or a symbol sequence and the meaning” it conveys to the observer (or put alternatively, the meaning a subjective observer extracts from the pattern or sequence). Atmanspacher suggests meeting this challenge by reference to Shannon’s purely syntactic information and its required extension by Weaver into semantic and pragmatic forms of information—in particular into pragmatic information as elaborated by Weizsäcker [4]. The rational: “If the meaning of some input into a system is understood, then it triggers action and changes the structure or behavior of the system.” From this view follows an intimate connection between complexity and meaning mediated as it is by the meaningful pragmatic information. Atmanspacher closes the article by a suggestion that although the notion of meaning is “usually employed in the study of cognitive systems, also physical systems allow (though not require) a description in terms of pragmatic information.” He illustrates this claim by reference to the case study of laser systems far from thermal equilibrium.

Information is a confusing notion: on the one hand, it is commonly used in everyday language as well as in scientific discussions, on the other, as Rainer Feistel and Werner Ebeling write in their contribution [5], “to the present day, the information specialists … have not agreed yet on a generally accepted, comprehensive and rigorous definition of what ‘information’ actually is”. As a consequence, studies on information—from Shannon’s seminal theory [1] and onward—tend to start with what their authors define as information. Feistel and Ebeling’s contribution is of no exception and starts by introducing a distinction between two new notions of information—structural information, “associated with arbitrary physical processes or structures” and symbolic information, the self-organized emergent property of socio-culturally agreed conventions. As an example, they refer to the evolution of human spoken language in which structural information in the form of “sound waves produced involuntarily by human babies…” are being transformed, by what they describe as ritualization, into the conventions of symbolic information of spoken languages. By “ritualization” they refer to a process of self-organization that gives rise to the emergence of symbolic information.

Feistel and Ebeling then relate these two forms of information to the notions of entropy and value. Due to Shannon, the concept of entropy has from the start been used in connection with information theory, while the notion of value has not. Commencing from Rudolf Clausius’ entropy they suggest that in the domain of matter, entropy can be interpreted as the value of energy. Then in the context of biological evolutionary theory, they interpret fitness as selective value which in simple cases is “a value of the structural information” while as a consequence of “…ritualization and the emergence of symbolic genetic information, fitness becomes a semantic value of the symbolic information…”. Finally, they relate their structural-symbolic forms of information to the distinction between use value and exchange value that was dominant in the classical political economy from Adam Smith to Karl Marx. While use value is subjective, exchange value can be seen as an emergent property of two levels of symbolic information: “The first level is the exchange value of a commodity which is the value of the symbolic information that the owner or producer of the commodity had exploited in order to make the good available for sale. The second level is fiat money which, after a ritualization transition in the course of social evolution, replaced commodity money.”

As is well recorded, the “classical” distinction between use and exchange values, with its entailed “labor theory of value”, has been replaced by the theory of marginal utility and has thus lost popularity in mainstream economics. This raises an interesting question for further research: Can marginal utility theory be related to information theory?

In their conclusions, Feistel and Ebeling establish a bridge to some other contributions to this Special Issue by stating: “values express the essence of biological, ecological, economic or social properties and relations with respect to the dynamics of the system, they represent fundamental order parameters of self-organized cooperative processes in complex multi-particle systems in the sense of Haken’s synergetics [6].”

In their paper, Haken and Portugali [7] stress a prerequisite for self-organization: the spontaneous formation of structures and functions by a system requires an exchange of energy, information and/or matter with its surrounding. In other words, the system must be open. This requirement holds both for the animate world (e.g., a flower) and the inanimate world (e.g., a fluid forming movement patterns). With respect to information, Haken and Portugali discuss the concepts of Shannon information, pragmatic information and semantic information as well as their interrelation. The latter is highlighted by the new concept of information adaptation, according to which the various forms of information condition each other [8]. To simultaneously deal with information and self-organization, Haken and Portugali base their approach on Jaynes’ maximum (information) entropy principle and establish connections with basic concepts of synergetics, i.e., order parameters and the slaving principle. Order parameters are both macroscopic descriptors of a system and determine the behavior of the numerous individual elements by means of the “slaving principle”. This implies a considerable reduction of complexity. In this approach, order parameters move in an “attractor landscape”. Since information is processed by the human brain, it appears natural to include neuronal self-organization and its perceptual correlates in the Haken and Portugali approach. The best studied process is surely visual perception, i.e., pattern recognition. This allows the authors to exemplify the concepts of order parameters, and the slaving principle. As a byproduct, the equations of the Synergetic Computer for pattern recognition are derived in a new way. The authors deal also with the invariance problem: how can we (or an advanced computer) recognize objects irrespective of their position in space, illumination etc.? Using their concept of “quasi-attractors”, the authors deal with the recognition of ambivalent figures, hybrid images and scenes. A treatment of saccadic eye movements is sketched. Their paper closes with a discussion of exploratory behavior of rats, some applications to urbanism (“synchronisation urge”) and hints at ties to consciousness research.

Gregoire and Catharina Nicolis [9] treat a one-variable nonlinear system. In our interpretation the behavior of its state variable can be visualized as the position of a particle that undergoes an overdamped motion in a one-dimensional periodic potential under the impact of a time-periodic and stochastic force. Their model allows them to elucidate the non-equilibrium dynamics of information, such as information entropy production, information transfer, Kolmogorov-Sinai entropy, and stochastic resonance. Their paper fits nicely not only into the domain of this Special Issue on Information and Self-organization, but also into the general concept of synergetics. There it is shown that close to instabilities the behavior of complex, multi-component systems can be described by few variables or even a single one, called order parameter(s). These state variables may obey rather simple stochastic differential equations, e.g., the one treated so beautifully by G. and C. Nicolis.

Emergence, self-organization, and complexity are among the basic properties of complex systems. In several previous studies Gershenson and Fernandez have proposed measures of emergence, self-organization, and complexity based on Shannon’s information theory. They interpreted Shannon’s information as a measure of emergence, the inverse of Shannon’s information as a measure of self-organization, while the balance between the two is interpreted as a measure of complexity. They developed these measures using discrete Shannon information with disadvantages such that in the continuous domain, Shannon’s information entropy is “a proxy of the average uncertainty for a probability distribution with a given parameter set, rather than a proxy of the system’s average uncertainty”.

In their contribution, Santamaría-Bonfil, Fernández and Gershenson [10] start by reminding the reader of the above noted measures based as they were on discrete Shannon information. They then suggest a novel approach and measures for the above basic notions of complexity based on differential entropy. This approach allows them to develop continuous version of their measures of emergence, self-organization, and complexity. Finally, they present results that relate their approach to Haken and Portugali’s notion of information adaptation (see above [7]) suggesting that their measure of complexity as a balance between emergence and self-organization corresponds to the processes of inflation and deflation through which, according to Haken and Portugali, information adaptation comes into being.

Santamaría-Bonfil’s et al. paper raises several philosophical questions that concern the nature of emergence, self-organization and complexity. With respect to emergence and self-organization, one way to interpret them is as processes, while a measure implies more/less—what does it mean more/less emergence or self-organization? Similarly, with respect to complexity: what does it mean more/less complexity and then when a low complexity system turns into a simple system?

Self-organization, as noted above, is implemented by the exchange of information. But how exactly the process of self-organization evolves? How the parts of a system exchange information and how out of this exchange an order emerges spontaneously, that is, by self-organization? In their paper, Santos, Encarnação, Santos, Portugali and Pacheco [11] suggest that by means of evolutionary game theory (EGT) one is able to expose and follow in detail, the intricacies of self-organization processes, that is, how by means of information exchange, one or few populations composed of individuals self-organize and coordinate a certain behavior. Starting from a traditional coordination game model that encompasses information exchange within a single class of individuals, they move to a two-sector scenario, representing a co-evolutionary process, where, for example, the success or failure of a given strategy in a population of individuals of type A depends on the success/failure of another population of individuals of type B and vice-versa. Such co-evolutionary games are typical of many social problems ranging from the economic relations between consumers and producers to the relations between local or national government and the private sector.

Finally, Santos et al. introduce an innovative approach that investigates a three-sectors co-evolutionary game. Such a game might refer, for instance, to the relations between producers, consumers and a third sector—the public sector, or between the public, private and a third sector—the civil society sector. They explore the changes in the self-organization process of all sectors, given the feedback that a third sector might imparts on the other two. As concluded by Santos et al. recognizing and understating the above self-organized dynamics of information exchange and cooperation is of specific importance in the context of policy making as means to “anticipate the effectiveness of new policies or to identify possible drawbacks...”.

Peter Schuster’s contribution [12] commences from the notion that processes of self-organization are ubiquitous in biology. An outstanding example is the evolution of species. In his pioneering work, Schuster has dealt with evolution at the level of molecules carrying genetic information, i.e., DNA and RNA. He takes three mechanisms into account: competition, cooperation, and mutation, and studies, what he called, a “minimal model”. It consists of chemical reaction equations that represent in particular the kinetics of replication. Schuster studies the impact of stochastic effects on the competition—cooperation system and discusses the important concepts of quasispecies and fitness. When the latter is defined as number of offsprings, this concept coincides with that of the order parameter central to synergetics (cf. the contribution by Haken and Portugali, this issue [7]). In his discussion, Schuster includes the role both of Shannon information and semantic/pragmatic information. It is noteworthy to mention that molecular evolution has been directly observed by experiments with viruses, bacteria, various cell-free in vitro essays, and has found important applications in evolutionary biotechnology.

Thompson and Quian [13] base their axiomatic approach to stochastic non-equilibrium thermodynamics on abstract mathematical concepts. In particular, they assume the existence of a well-defined stationary probability distribution on a countable state space and illustrate their procedure concerning criticality by several examples. Central to their approach is the relation between a potential H(x) of a state x and the stationary probability p^eq(x):

$$H\left(x\right)=-\ln p^{eq}\left(x\right)$$

(1)

The authors relate this equation to Boltzmann’s distribution function of systems in thermal equilibrium, his entropy formula, and eventually $p^{eq}$ to a distribution function using the definition of free energy. Based on their foregoing work, Graham and Haken [14] have shown that—under rather general conditions—the relation (1) holds also for systems far from thermal equilibrium and can be applied, e.g., to non-equilibrium phase transitions. Relations of the type (1) can also be derived by means of Jaynes’ maximum (information) entropy principle, that is utilized in the contribution by Haken and Portugali (this issue). The relation (1) is basic to Friston’s [15] free energy principle for biological systems.

From our point of view, a little caveat should be observed with respect to calling H(x) “energy”. Actually, in nonequilibrium systems, p^eq(x) is determined by rate constants, whereas in equilibrium states, constants determining energies are involved.

Tsuda, Yamaguti, and Watanabe [16] deal with the development of the human brain as self-organizing process leading to functional differentiation of neurons and cortical modules. To this end they present numerical treatments of specific models such as one-dimensional maps, e.g., a time-discrete version of the well-known Kuramoto model. The general frame is provided by deterministic dynamical systems subjected to constraints as previously applied to neural systems by Tsuda and co-authors. In its development, the structural and functional differentiation of an individual brain is promoted in particular by the intentions and actions of surrounding people, which according to Tsuda et al. become constraints of the self-organization of neural dynamics. The authors derive and discuss also mutual information as well as transfer entropies. Their results imply a hierarchy between two modules 1 and 2 in accordance with synergetics where slaving modes of module 1 behave cooperatively forming few order parameters, whereas slaved modes of module 2 show more varieties of interaction. The authors conclude that a developed system may express a conscious mind for module 1 and a more unconscious mind for module 2. In our opinion, this fits nicely—at least in principle, with Leopold’s experimental finding [17], on vision.