Naturalizing Information

Abstract

Certain definitions of information can be seen to be compatible with each other if their relationships are properly understood as referring to different levels of organization in a subsumptive hierarchy. The resulting hierarchy, with thermodynamics subsuming information theory, and that in turn subsuming semiotics, amounts to a naturalizing of the information concept.

1. Introduction

In this paper I forge a naturalistic, or naturalized, concept of information, by relating it to energy dissipative structures [1]. This gives the concept a definable physical and material substrate. The conceptual bases for this exercise will be nothing more than two commonly recognized definitions of information—Shannon's, and Bateson's—together with my own, thermodynamic, definition. Arguing that the “real” definition of information is an amalgam of all three, I find it impossible to say in a single sentence what that definition is. This, however, does not necessarily deconstruct the idea of information. Instead, it can be enriched by way of enfolding these separate definitions within a subsumptive hierarchy of three integrative levels. This is possible because the three definitions mentioned above are operative at different levels of generality.

2. Hierarchy

I will be using the subsumption hierarchy throughout this paper to represent logical relations between levels of organization. This form of hierarchy [2] (there labeled “specification hierarchy”) is a nesting of classes and subclasses. One version of its logic would be:

$$\{\text{most general}\{\text{more specific}\{\text{most particalar}\}\}\}$$

where the most general, or generally present, class subsumes the others. So the more particular classes inherit the properties of the more general ones. I use “classes” in plural here because the system is conceptually a tree, with its trunk in the most general class. In this paper I will need to refer only to a single branch, but this system would allow for multiple definitions of information at any given level, if that should sometime become a useful project. The most general influence at any time in a given locale would be the weakest but most persistent, while the most particular influence would locally be the strongest, but also the most ephemeral. The “more specific” level shown in the above hierarchy would be more abstract than the “most particular” level. Thus, if “grasping” were the most particular event, then “capturing” would be less specific and more general. Explanations and definitions can be pitched at one or other of these levels (using the kind of argument in [3]). I believe that the three definitions of information dealt with herein occupy different levels in a subsumptive hierarchy, and therefore would all be in effect at once in any local example.

3. Modes of Information

The three definitions of information I refer to are the Shannonian [4], Bateson's definition [5], and, as I call it, the “thermodynamic definition”, which I have recently been fielding on Internet discussions, and am here presenting in more detail. I conceive these perspectives on information as “modes”, not merely definitions, because they refer to different kinds of embodiments of the information. The Shannon concept could be most generally stated as: “information is a reduction in uncertainty”. This implies an observer, or, more generally, some system that makes choices. Bateson's definition: “a difference (or distinction) that makes a difference” implies further that the informed system is capable of classifying inputs or sensations as indications of external objects.

My own “any constraint on entropy production” derives from Pattee's [6] treatment of nonholonomic constraints. In my version it does not imply a distinct observer, but does point to the ability for classifications to emerge at material locales, and so any such locale has the potentiality for behaving as if it had a viewpoint. I refer to entropy production in this definition rather than energy flows because the former is more general (as seen in processes like undirected mass wasting or diffusion), and refers as well to the forward-looking aspect of events in an out-of-thermodynamic-equilibrium world. My perspective is that the evident distance from thermodynamic equilibrium of our universe is a fact that contextualizes, and subsumes everything else. That is, entropy production is imminent everywhere as being called for by the universe's tendency toward thermodynamic equilibrium. Thus, all energy gradients are metastable. This cues the philosophical stance of this paper.

The “real” meaning of information might perhaps be supposed to found in the “intersection” of these definitions. However, since the modes of information cited here do not exist at a same level, any simple intersection could not actually be stated as such, and would instead have to be taken as being vague at best. Rather than being content with this culturally uncongenial approach, we can parse the three modes as meanings at different levels in a subsumptive hierarchy, thus:

$$\{\text{themodynamic search}\{\text{reduction of uncertainty}\{\text{emergence of meaning}\}\}\}$$

Here, all three senses of information would be in effect in a given system or at a given locale at any moment, with the definition at the most general level subsuming the others. Thus, meaning may emerge attendant upon a reduction of uncertainty engendered by choice during a search among potential energy flow pathways differentiated by different accessible rates. It may be noted that the more general, or generally-present-in-the-world classes, function as contexts for the more particular ones. This contextual organization shows the formulation to be an exercise in pragmaticism, as a manner of clarifying ideas [7,8], with each more inner subclass qualifying a concept further.

4. Thermodynamic Searching: Information from Inquiry

Here I propose that the most fundamental aspect of information is its physical/material source in actions. This observation would be trivial unless informational constraints do arise already at this most basic (physical) integrative level, with consequences for the higher levels. We encounter a major constraint on information even as we consider searching for it—searching, or sorting, is a physical activity, as can be illustrated with a very general example. Consider a large, well-watered plateau in a rain belt, or it may be a plateau topped by a melting glacier. It has several possible sites for water runoff in waterfalls. These vary in the amount of water they can drain according to their locations, substrata and configurations. Some will drain faster than others, flushing more water per unit of time than other routes. As the water drains off, some outlets will increase in relative importance. Eventually only these will remain as drainage avenues. The search for physical flows has located a sequence of them as ports of “least action” [9] during the drainage history of the locale. The characteristics of these ports were, and are, serving as transient nonholonomic constraints on the flow dynamics.

These nonholonomic constraints (configurations assembled by history) were the relevant information in the history of dynamical flows in this case. Clearly any locale in a material world will have, or be composed of, such constraints with respect to multiple energy gradients. These constraints are the embodiments of history at any locale. Without history there would be no information, and, of course, no informational constraints would be needed to understand the world—as for example would reputedly be the case in a quark-gluon plasma. History appears (often unmarked as such) in physical analyses as constant parameters constraining whatever actions are being modeled mathematically. Most generally, in Y = aX^b, a and b are nonholonomic informational constraints. With respect to any particular potential energy dispersion, a locale will be organized with respect to that flow, but the flow will also change that organization as it proceeds [10].

We see that material locales, however conceptually jumbled they may appear, are in fact implicitly organized with respect to multiple possible energy flows. This organization, which is realized—or, from the information perspective, is “perceived”—only after the fact of thermodynamic processes, argues well for the idea that thermodynamics is the most general of scientific perspectives about the natural world. Even a randomly assembled junk pile will be implicitly organized thermodynamically. Energy gradients will be elicited to disperse in any landscape at all, and will discover those routes that equalize them fastest. The energy flows themselves tend to narrow the further choice of possible flow pathways [11], although some of the newly uncovered possible flow paths might actually open up new directions. So the “inquiry” of thermodynamic searching is characterized by “path dependency” as it constructs an evolutionary trajectory [12]. The consequence of this basic thermodynamic constraint on information will be that, whatever information is constructed in the higher integrative levels, it will be consistent with those constructions reflecting avenues for searching that dissipated the available energies through the fastest available routes—the characteristic constraint on information at the lowest integrative level.

5. Material Realization: Information as the Chosen

The above framework for choice implies local materiality, it may be liquid. Purely physical systems would reassemble spontaneously, obliterating any results of local choices that may have been made. Such systems would be imagined to change only globally. Materiality implies some degree of global preordination (given a Laplacean demonic intelligence!) but choices must nevertheless be made locally, therefore in at least partial ignorance. As a sequence of choices is made, a stream of information—as a concatenation of reductions of uncertainty about the next step—is created by way of historical adventures. Any choice also changes downstream adjacent possibilities for further change. In this way arise drainage systems, biological species, and, it seems reasonable to me, most universal physical constants (like the gravitational constant, g) as well. Each of these, then, represents the current result of a string of choices, with the latest surmounting, and in some cases even obliterating, the results of previous choices. Choices are expedient; materiality is history-laden. We may note here as well the view that semiosis emerges from interactions [13], resulting in a Peircean “habit taking” (e.g., [14]).

The process of choosing requires (in the physical sense) “actions” [9] and delivers change, and is thus subsumed by thermodynamic searching, as discussed above. Thermodynamic searching for the fastest way to dissipate supporting energy gradients, being everywhere necessary, underlies all more specific searches, as will be discussed in connection with the emergence of meaning below. Each choice, therefore, acts under the guidance of a kind of underlying physical palimpsest at any locale. Where dissipations are mediated by dissipative structures, each of these is itself an energy gradient, and will survive only so long as it does not become a site along the route for the fastest potential local dissipation not blocked by an energy barrier. It can successfully avoid this only by acting/working as vigorously as possible in the erection of defensive barriers.

The “Shannonian” process of choosing in a natural setting would be “evolutionary” (sensu [12]) because in principle no downstream choices can be foreseen. Choosing thus partakes in a process of discovery, or construction, with the elected information being emergent. An unguided material system could be “developmental” (sensu [12]) only in the narrow sense that each choice would have to be launched upon the latest prior one. It would thus be mediating an “epigenetic” process, usually creating an at least partially preserved informational “path”. Fully developmental (therefore guided) material systems would not generate new configurational information during their constitutive processes except via fluctuations, or by surviving disruptions. Following upon either of these, such a developing system would individuate—that is to say, evolve. Information is historical, and is therefore a product of evolution. Thus, an individual specimen of a particular species, insofar as it is viewed as embodying only the definitive forms of that species, would carry no anatomical information. An instructive example of the accrual of information would be the development of human personality as an historical process. (Based on this, one might postulate the “personality” of any material locale.)

The general pattern of a natural development is describable—given that there is guidance along the way (or entrainment by an end point)—as a process of changing from a vaguer embodiment toward a more definite one. The assembly of a machine is neither evolutionary (hopefully!) nor developmental. It is a definite, “crisp” procedure all the way through. Nothing “natural” has this character. An interesting natural example regarding the biological synthesis of protein enzymes is found in Dunker and Kriwacki [15]. Until recently such cellular processes have been described in very mechanistic terms (e.g., [16]). The newer picture is that the “native” enzyme is very flexible and continually altering its shape according to conditions and its current activity.

Another aspect of developmental processes can be seen in the positive feedback of autocatalytic cycles, described by Ulanowicz [17] as the development of “ascendency”. This is developmental, not because it follows a script (which it does not), but because the empirical tendency is generally toward a more definite embodiment, gradually entraining more and more of the ambient possibilities of its locale as the cycle incorporates more members, becoming more locally dominant. During this process there is loss of unentrained activities that could give the cycle a measure of flexibility and “requisite variety”, thus leading the system toward a more senescent condition [12] as it grows in membership while becoming more hegemonic locally. Outside of highly controlled systems like the living, such cycles tend not to get so ankylosed, being continually perturbed by various environmental fluctuations—that is to say, being frequently confounded by new informational entropy not readily incorporable into the system.

To here, we have the picture of material locales with metastable energy gradients that have the tendency to dissipate explosively. But the material world has placed energy barriers everywhere, so that dispersion of energies can actually happen only relatively gradually even along the more favorable (fastest dissipating) pathways. Living systems are also energy barriers in this sense [18]. And so we can model the situation as:

$$\{\text{explosive energy dispersion tendency}\{\text{material informational constraints}\{\text{including the biological}\}\}\}$$

6. Interpretation: Information Can Generate Meaning

The term “generate” in this heading should not be interpreted in a diachronic sense. Searching, choosing and interpreting most generally occur simultaneously, as in the Peircean triad, Firstness, Secondness, Thirdness, which I would hierarchize as {First {Seconds {Thirds}}}. No single, fully explicit rendering of these very general categories is canonical; for our purposes here, {unentrained possible actions {tendency {incorporation}}} would be sufficient. Especially in abiotic situations (e.g., [19]), the three categories would emerge together, with Thirds being fragile and fugitive. In biotic systems there may be visualized a more sequential, but still overlapping, semiosic process localized within a relatively more stable system.

In Newtonian interaction there is no interpretation; we have action and, simultaneously, equal and opposite reaction. Interpretation requires that context have a significant effect on the outcome of interactions. In Peircean semiotics the context becomes embodied in an interpretive system as the Sign, which represents a relevant context of a semiosic system in situ [19,20]. Signs are representations of contexts erected on the boundary between a semiosic system's internal and external dimensions. For example, words exist external to the user, in the language of a social system, but also have become “implanted” in the minds of individuals, where they guide thinking. Signs get constructed in a semiosic system during repeated experiences in given contexts.

Bateson's view of information as a detected difference, or distinction, that makes a difference to a responsive system, requires that there be an actual material system (however fragile and fugitive), which I have referred to as a “system of interpretance” [21]. This, as material, will necessarily be embedded in many potential contexts, only some of which could be significant to it [22]. These relevant contexts will have played a part in the historical construction of the system, and so will have left traces in the form of receptors. In living systems some of these historical events will have predated existing individuals, and so receptors related to these will have been passed on to the current living indirectly by way of evolutionary processes. Others will have been discovered during an individual's development. Both exemplify the Peircean “taking of habits”, but at different time scales—evolutionary time for receptors inherited from the species, experienced time for signs constructed during an individual's ontogeny and adventures. Interpretation develops along with habits built upon successful choices of reactions to relevant contextualized impacts.

“Making a difference” means that the system (of interpretance) has, or can generate, a sign that stands for the significance of an indication received upon its receptors. Its interpretation may take some time to emerge, by way of a developmental process involving several to many subordinate processes (the “interpretants” in Peircean terms—see examples in Figure 1). Since these processes usually would occur at a smaller scale than that of the system itself, the actual “perceived” time at the level of the system itself may be short [2].

Definitive meanings can emerge in a semiosic system when it has the stability of a living system, where signs can be embodied with some degree of permanence and reliability. Otherwise, signs would be vague, fugitive and impermanent, and, practically, meaningless—which does not mean that they would be non-existent [19]. Meaning generally is expressed in the activities of a system as a result of its guidance by signs. In the simplest material cases, if an interaction generates a different result in altered contexts, we would have a primitive “protosemiosis”, as when, e.g., some physical result is temperature sensitive involving a threshold type of effect. The fullest expression of meaning, however, emerges with biological systems, as related to their ecological niches and roles.

Biological examples allow us to reinforce the synchronic aspect of the formulation:

$$\{\text{thermodynamic potentials for energy dispersion}\{\text{reduction of uncertainty of action}\{\text{emergence of meaning}\}\}\}$$

because here we can see the temporal initiation of an episode as being located in the highest level, associated with meaning. Consider a predator that has fixed upon a prey organism—it strikes. The uncertainty of its next motion has been eliminated by having chosen a particular line of attack. And this choice of behavior will exemplify a particularly rapid means of utilizing its ATP energy stores contextualizing the strike. The predator was not directly driven to attack by the metastability of the prey's energy gradient (its susceptibility), or even by its own surplus energy stores. Rather, both of these could be said to have “invited” the attack, which then reduced the uncertainty of what would happen next, as guided by signs and receptors in the predator's organization. The predator's strike served as a constraint upon the entropy production of the event. If it had been unguided by inherited or habitually constructed signs, the predator would have clumsily spent more energies upon the kill. Thus:

$$\{\text{imminent entropy production potentials}\{\text{constrained by choice of attack}\{\text{strike}\}\}\}$$

So, search is physical, pulled by the Second Law of thermodynamics in an out-of-equilibrium universe. Then choice would be biased by least action/fastest dispersal of locally available energy gradient [9], as local meaning emerges pragmatically, by way of immediate adaptability. Meaning resides in finality [21], which in the predator case is its requirement for nutriment. But we must note that there is a more general finalism involved here as well—the Second Law of thermodynamics, acting here in the lowest, most general level of organization—the physical—and which would be operative in both searching and making a choice. We can note as well that the burgeoning of autocatalytic cycles, and their increasing ascendency, is also “pulled” by positive feedback loops plausibly viewed as being invited by the Second Law.

7. Summary

The general argument above is that information and meaning, when viewed from a physical perspective, emerge in a landscape of metastable energy potentials when an action decides a choice of path. The action may be informed (as in living systems) by prior inherited, or previously acquired, tendencies. In a Shannonian perspective this would be a reduction of global informational entropy, but in a natural system there is no reduction because each choice may open up new, previously unavailable possibilities that sometimes temporarily even increases the informational entropy of the field [23]. At any moment, a field of possibilities, choice, and meaningful action are intimately entwined in a systemic relationship that can be succinctly modeled using a subsumption hierarchy, as in:

$$\begin{array}{c}\{\text{imminent search}\{\text{choice}\{\text{meaning}\}\}\}\\ \{\text{implicit seeking}\{\text{choosing}\{\text{assimilating}\}\}\}\end{array}$$

or

$$\{\text{informational entropy}\{\text{reducted to information}\{\text{by meaning}\}\}\}$$

These hierarchies are simultaneously both synchronic and diachronic. “Synchronic” because in general in the natural world choosing does not necessarily decrease the field of further possible choices. The outer (lower) levels subsume everything found in the higher levels, while the higher levels (here guided by meaning) integrate/harness all the lower level powers occurring locally under their emergent rules. Meaning harnesses a choice among possibilities that was invited by the fastest route to equalize local energy gradients, and that meaning is then further reinforced in the event.