*Learning in the Evolutionary Perspective*

A recent trend in the design is learning from nature, biomimetics. Deep learning is one of the technologies developed within the biomimetic paradigm. In the case of intelligence, we still have a lot to learn from nature about how our own brains, intelligence, and learning function. One of the strategies is to start with learning in simplest organisms, in order to uncover basic mechanisms of the process. Evolution can be seen as a process of problem-solving [34]. "From the amoeba to Einstein, the growth of knowledge is always the same: We try to solve our problems, and to obtain, by a process of elimination something approaching adequacy in our tentative solutions" [62] (p. 261). All acquired knowledge—whether it is acquired in the process of genetic evolution or in the process of individual learning—consists, this is Popper's central claim, in the modification "of some form of knowledge, or disposition, which was there previously, and in the last instance of inborn expectations" [62] (p. 71).

Popper's theory of the growth of knowledge through trial-and-error conjecture-based problem-solving shares basic approach with evolutionary epistemology. According to Campbell [63], all knowledge processes can be seen as the "variation and selective retention process of evolutionary adaptation" [64]. Thagard [65] criticizes Popper, Campbell, Toulmin, and others who proposed Darwinian models of the growth of (scientific) knowledge. Evolutionary epistemology emphasizes analogy between the development of biological species and scientific knowledge, based on variation, selection, and transmission. Thagard, on the other hand, holds that di fferences are more important than similarities, and that scientific knowledge is guided by "intentional, abductive theory construction in scientific discovery, the selection of theories according to general criteria, the achievement of progress by sustained application of criteria, and the transmission of selected theories in highly organized scientific communities". Even though scientific knowledge is a specific, formal kind of knowledge, it is nevertheless knowledge.

This criticism of evolutionary epistemology is addressing a specific understanding of evolution, through Darwinism in the narrow sense. However, the contemporary extended evolutionary synthesis provides mechanisms beyond blind variation of narrow Darwinism, and can accommodate for learning, anticipation, and intentionality [66–69]. In a similar, broader evolutionary approach, Watson and Szathmáry ask "Can evolution learn?" in [70], and sugges<sup>t</sup> that "evolution can learn in more sophisticated ways than previously realized". Here "A system exhibits learning if its performance at some task improves with experience". They propose new theoretical approaches to the evolution of evolvability, and the evolution of ecological organizations, among others. They refer to Turing, who made an algorithmic model of computation (Turing machine) and established the connection between learning and intelligence through an algorithmic approach [71]. The relationship between learning and evolution is established through the notion of reinforcement learning, as "reusing behaviors that have been successful in the past (reinforcement learning) is intuitively similar to the way selection increases the proportion of fit phenotypes in a population". Watson and Szathmáry's paper list number of di fferent types of learning, including diverse machine learning approaches, ended with the claim that there is a clear analogy between evolution and the process of learning, and that we can better understand evolution if we see it as learning.

In spite of mentioning Turing's pioneering work on the topic of algorithmic learning, Watson and Szathmáry assume "incremental adaptation (e.g., from positive and/or negative reinforcement)".

Critics of the evolutionary approach argue for the impossibility of such incremental process to produce highly complex structures such as intelligent living organisms. Monkeys typing Shakespeare are often used as illustration. As an counterargument, Chaitin [72] pointed out that typing monkeys' argumen<sup>t</sup> does not take into account physical laws of the universe, which dramatically limit what can be typed. Moreover, the universe is not a typewriter, but a computer, so a monkey types random input into a computer. The computer interprets the strings as programs. Or, in the words of Gershenfeld: "Your genome doesn't store anywhere that you have five fingers. It stores a developmental program, and when you run it, you ge<sup>t</sup> five fingers" [73].

Sloman argued that "many of the developments in biological evolution that are so far not understood, and in some cases have gone unnoticed, were concerned with changes in information processing. The same is true of changes in individual development and learning: They often produce new forms of information processing". He addressed this phenomenon through computational ideas

about morphogenesis and meta-morphogenesis [74]. His approach o ffers new insight, that variation is algorithmic. To Sloman's computational approach, I would add that steps in variation are morphological computation, which means physical computation, capable of randomly modifying genes, and executing morphological programs which do not present smooth incremental changes, but considerable jumps in properties of structures and processes. Morphological computation acts also through gene regulation, which is one more process that was unknown to both Darwin and to proponents of evolution as Modern Synthesis. Originally, genes were considered as coding for specific proteins, and it was believed that all genes were active. Gene regulation involves a mechanism that can repress or induce the expression of a gene. According to Nature [75], "These include structural and chemical changes to the genetic material, binding of proteins to specific DNA elements to regulate transcription, or mechanisms that modulate translation of mRNA".

#### **4. Learning as Computation in Networks of Agents**

In what follows, we will focus on info-computational framework of learning. Informational structures constituting the fabric of physical nature are networks of networks, which represent semantic relations between data for an agen<sup>t</sup> [18]. Information is organized in levels or layers, from quantum level to atomic, molecular, cellular, organismic, social, and so on. Computation/information processing involves data structure exchanges within informational networks, which are instructively represented by Carl Hewitt's actor model [76]. Di fferent types of computation emerge at di fferent levels of organization in nature as exchanges of informational structures between the nodes (computational agents) in the network [17].

The research in computing nature/natural computing is characterized by bi-directional knowledge exchanges, through the interactions between computing and natural sciences [54]. While natural sciences are adopting tools, methodologies, and ideas of information processing, computing is broadening the notion of computation, taking information processing found in nature as computation [2,77]. Based on that, Denning argues that computing today is a natural science, the fourth grea<sup>t</sup> domain of science [78,79]. Computation found in nature is a physical process, where nature computes with physical bodies as objects. Physical laws govern processes of computation, which appear on many different levels of organization in nature.

With its layered computational architecture, natural computation provides a basis for a unified understanding of phenomena of embodied cognition, intelligence, and learning (knowledge generation), including meta-learning (learning to learn) [47,80]. Natural computation can be modelled as a process of exchange of information in a network of informational agents [76], i.e., entities capable of acting on their own behalf, which is Hewitt's actor model applied to natural agents.

One sort of computation is found on the quantum-mechanical level, where agents are elementary particles, and messages (information carriers) are exchanged by force carriers, while di fferent types of computation can be found on other levels of organization in nature. In biology, information processing is going on in cells, tissues, organs, organisms, and eco-systems, with corresponding agents and message types. In biological computing, the message carriers are chunks of information such as molecules, while in social computing, they are sentences while the computational nodes (agents) are molecules, cells, and organisms in biological computing or groups/societies in social computing [19].

#### **5. Info-Computational Learning by Morphological Computation**

The notion of computation in this framework refers to the most general concept of intrinsic computation, that is spontaneous computation processes in the nature [2,77], and which is used as a basis of designed computation found in computing machinery [81]. Intrinsic natural computation includes quantum computation [81,82], processes of self-organization, self-assembly, developmental processes, gene regulation networks, gene assembly, protein–protein interaction networks, biological transport networks, and similar. It is both analog (such as found in dynamic systems) and digital. The majority of info-computational processes are sub-symbolic and some of them are symbolic (like reasoning and languages).

Within info-computational framework, or computing nature [18], computation on a given level of organization of information presents a realization/actualization of the laws that govern interactions between its constituent parts. On the basic level, computation is manifestation of causation in the physical substrate [83]. In every next layer of organization, a set of rules governing the system switch to the new emergen<sup>t</sup> regime. It remains ye<sup>t</sup> to be established how this process exactly goes on in nature, and how emergen<sup>t</sup> properties occur [84]. Research on natural computing is expected to uncover those mechanisms. In the words of Rozenberg and Kari: "(O)ur task is nothing less than to discover a new, broader, notion of computation, and to understand the world around us in terms of information processing" [2]. From the research in complex dynamical systems, biology, neuroscience, cognitive science, networks, concurrency etc., new insights essential for the info-computational nature are steadily coming. Here it should be mentioned that the computing nature with "bold" physical computation [85] is the maximal physicalist approach to computing. There are less radical approaches, such as taken by Horsman, Stepney, and co-authors [86–88], known as Abstraction/Representation theory (AR theory), where "physical computing is the use of a physical system to predict the outcome of an abstract evolution", where computation defines the relationship between physical systems and abstract concepts/representations. Unlike AR theory, info-computationalism also embraces computation without representation, in the sense of Brooks [89] or Pfeifer [90]. Even though it is already established that the original Turing model of computation is specific and represents a human performing calculation, as pointed out by Copeland [91], even Turing himself started exploring computation beyond the Turing Machine model.

Turing's 1952 paper [92] may be considered as a predecessor of natural computing. It addressed the process of morphogenesis by proposing a chemical model as the explanation of the development of biological patterns such as the spots and stripes on animal skin. Turing did not claim that a physical system producing patterns actually performed computation. From the perspective of computing nature, we can now argue that morphogenesis is a process of morphological computation. Informational structure (as representation of embodied physical structure) presents a program that governs computational process [93], which in its turn changes that original informational structure following/implementing/realizing physical laws.

Morphology is the central idea in our understanding of the connection between computation and information. Morphological/morphogenetic computing on that informational structure leads to new informational structures via processes of self-organization of information. Evolution itself is a process of morphological computation on a long-term scale. It is also important to take into account the second order process of morphogenesis of morphogenesis (meta-morphogenesis) as done by Sloman [74].

A closely related idea to natural computing is Valiant's [94] view of evolution by "ecorithms"—learning algorithms that perform "probably approximately correct" (PAC) computation. Unlike the classical model of Turing machine, the "ecorithmic" computation does not give perfect results, but good enough (for an agent). That is the case for natural computing in biological agents who always act under resource constraints, especially time, energy, and material limitations, unlike Turing machine model of computation that by definition operates with unlimited resources. An older term for PAC due to Simon is "satisficing" [95] (p. 129): "Evidently, organisms adapt well enough to 'satisfice'; they do not, in general, 'optimize'".

#### **6. Learning to Learn from Raw Data and up—Agency from System 1 to System 2**

Cognition is a result of a processes of morphological computation on informational structures of a cognitive agen<sup>t</sup> in the interaction with the physical world, with processes going on at both sub-symbolic and symbolic levels [4]. This morphological computation establishes connections between an agent's body, its nervous system (control), and its environment [49]. Through the embodied interaction with the informational structures of the environment, via sensory-motor coordination, information structures are induced (stimulated, produced) in the sensory data of a cognitive agent, thus establishing perception, categorization, and learning. Those processes result in constant updates of memory and other structures that support behavior, particularly anticipation. Embodied and corresponding induced informational structures (in the Sloman's sense of virtual machine) [96] are the basis of all cognitive activities, including consciousness and language as a means of maintenance of "reality" or the representation of the world in the agent.

From the simplest cognizing agents such as bacteria to the complex biological organisms with nervous systems and brains, the basic informational structures undergo transformations through morphological computation as developmental and evolutionary form-generation—morphogenesis. Living organisms as complex agents inherit bodily structures resulting from a long evolutionary development of species. Those structures are the embodied memory of the evolutionary past [54]. They present the means for agents to interact with the world, ge<sup>t</sup> new information that induces embodied memories, learn new patterns of behavior, and learn/construct new knowledge. By Hebbian learning in the brain (where neurons that wire together, fire together, and habits increase probability of firing), world shapes humans' (or an animals') informational structures. Neural networks that "self-organize stable pattern recognition code in real-time in response to arbitrary sequences of input patterns" are an illustrative example [97].

On the fundamental level of quantum mechanical substrate, information processes represent actions of laws of physics. Physicists are already working on reformulating physics in terms of information [98–103]. This development can be related to the Wheeler's idea "it from bit" [104] and von Weizsäcker's ur-alternatives [105].

In the computing nature approach, nature is consisting of physical structures that form levels of organization, on which computation processes develop. It has been argued that on the lower levels of organization, finite automata or Turing machines might be an adequate model of computation, while in the case of human cognition on the level of the whole-brain, non-Turing computation is necessary, see Ehresmann [106] and Ghosh et al. [107]. Symbols on the higher levels of abstraction (System 2) are related with several possible sub-symbolic realizations, which they generalize, as Ehresmann's models show. Zenil et al.'s work on causality by algorithmic generative models to "decompose an observation into its most likely algorithmic generative models" [108] presents one of the recent attempts to computationally/algorithmically approach causality. Algorithmic computation is a very important part of computational models defined by Turing, based on symbol manipulation. The connection to sub-symbolic is done through algorithmic information theory.

Apart from the *Handbook of Natural Computing* [77] that presents concrete models of natural computation, interesting work on computational modelling of biochemistry and reaction networks have been done by Cardelli [109–112], including the study of morphisms of reaction networks that link structure to function. On the side of cognitive computing, Fresco addresses the physical computation and its role in cognition [113].

Principles of morphological computing and data self-organization from biology have been applied in robotics as well. In recent years, morphological computing emerged as a new idea in robotics, see [3,4] and references therein. Initially, robotics treated separately the body as a machine, and its control as a program. Meanwhile it has become apparent that embodiment itself is fundamental for cognition, generation of behavior, intelligence, and learning. Embodiment is central because cognition arises from the interaction of brain, body, and environment [90]. Agents' behavior develops through embodied interaction with the environment, in particular through sensory-motor coordination, when information structure is induced in the sensory data, thus leading to perception, learning, and categorization [48]. Morphological computing has also been applied in soft robotics, self-assembly systems, and molecular robotics, embodied robotics, and more. Even though the use of morphological computing in robotics is slightly di fferent from the one in computing nature, there are common grounds and possibilities to learn from each other on the multidisciplinary level. Similar goes for the research being done in the fields of cognitive informatics and cognitive computing. There are also important connections to

computational mechanics, algorithmic information dynamics (probabilistic framework of algorithmic information dynamics used for causal analysis), and neuro-symbolic computation, combining symbolic and neural processing, all of which are in di fferent ways relevant to the topic. Those connections remain to explore in the future work.
