Evolutionary Implications of Self-Assembling Cybernetic Materials with Collective Problem-Solving Intelligence at Multiple Scales

Abstract

In recent years, the scientific community has increasingly recognized the complex multi-scale competency architecture (MCA) of biology, comprising nested layers of active homeostatic agents, each forming the self-orchestrated substrate for the layer above, and, in turn, relying on the structural and functional plasticity of the layer(s) below. The question of how natural selection could give rise to this MCA has been the focus of intense research. Here, we instead investigate the effects of such decision-making competencies of MCA agential components on the process of evolution itself, using in silico neuroevolution experiments of simulated, minimal developmental biology. We specifically model the process of morphogenesis with neural cellular automata (NCAs) and utilize an evolutionary algorithm to optimize the corresponding model parameters with the objective of collectively self-assembling a two-dimensional spatial target pattern (reliable morphogenesis). Furthermore, we systematically vary the accuracy with which the uni-cellular agents of an NCA can regulate their cell states (simulating stochastic processes and noise during development). This allows us to continuously scale the agents’ competency levels from a direct encoding scheme (no competency) to an MCA (with perfect reliability in cell decision executions). We demonstrate that an evolutionary process proceeds much more rapidly when evolving the functional parameters of an MCA compared to evolving the target pattern directly. Moreover, the evolved MCAs generalize well toward system parameter changes and even modified objective functions of the evolutionary process. Thus, the adaptive problem-solving competencies of the agential parts in our NCA-based in silico morphogenesis model strongly affect the evolutionary process, suggesting significant functional implications of the near-ubiquitous competency seen in living matter.

1. Introduction

Biological systems are organized in an exquisite architecture of layers, including molecular networks, organelles, cells, tissues, organs, organisms, swarms, and ecosystems. It is well recognized that life exhibits complexity at every scale. Increasingly realized, however, is the fact that those layers are not merely complex but actually active “agential matter”, which has agendas and competencies of its own [1,2]. Elsewhere, we have discussed examples of problem-solving in unconventional spaces, including transcriptional, physiological, metabolic, and anatomical space [3].

Especially interesting is the ability of these ubiquitous biological agents to deal with novel situations on the fly, which is not limited to brainy animals navigating 3D space but also occurs with respect to injury, mutations, and other kinds of external and internal perturbations (reviewed in [4]). One example of such problem-solving capabilities is the regenerative properties of some species that can regrow limbs, organs, or entire parts of their bodies when amputated, and—remarkably—stop when the precisely correct target morphology is complete [5,6,7]. This can be understood as cellular collectives navigating morphospace until the desired target shape—or the goal—is reached again. Other examples include the ability of scrambled tadpole faces to reorganize in novel ways to result in normal frog faces [8], and the normal shape and size of structures in amphibia despite drastic changes in cell number [9] and cell size [10], which are handled by exploiting different molecular mechanisms to reach correct target morphologies despite novel changes in internal components. Behavioral and morphological plasticity intersect in cases such as tadpoles made with eyes on their tails, which nevertheless can see and learn in visual assays without needing rounds of evolutionary adaptation [11].

The ability to navigate transcriptional and anatomical spaces, using perception–action loops and homeostatic setpoints, is now being increasingly targeted by biomedical and bioengineering efforts [12,13]. A fascinating body of work exists around the question of how neural and non-neural problem-solving capacities evolved, and how neuro-behavioral intelligence affects evolution [14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31]. However, we and others have previously suggested that somatic competency pre-dates neural intelligence [32,33,34], and has a bi-directional interaction with the evolutionary and developmental process [1,3,35]. Thus, here, we address the second half of the evolution–intelligence spiral: how are evolutionary processes affected by the competency of the material? Especially important is the inclusion of the middle layer between the genotype and phenotype. Mutation operates on genomes, and selection operates on phenotypic performance, but in most organisms, the connection between them is not linear or shallow—instead, developmental physiology provides a deep reservoir of dynamics that strongly alter the process. As a contribution to the study of evolvability and developmental mechanisms potentiating it [36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53], we established a virtual embryogeny [54] system focused on anatomical morphogenesis by cells. In this minimal model of morphogenesis, we were able to study the effects of different degrees of cellular competency on the evolutionary process.

The standard understanding of (Neo-Darwinian) evolution is schematized in Figure 1A: The genome of an organism encodes aspects of the organism’s cellular hardware, which together define the phenotypic traits. Given a competitive environment, natural selection then favors organisms with advantageous traits, and thus, on average, the corresponding genes tend to get passed on to the next generations more frequently. Random mutations may occur, consequently changing traits in the offspring phenotype. This affects the offspring’s reproductive success during the selection stage and, in that way, good traits prevail, and bad ones perish over time.

This view has been revised by Waddington [55,56], and more recent works [57,58,59,60,61,62,63,64,65,66], and has been the subject of vigorous debate [40,63,67,68,69,70,71,72] with respect to its capabilities for discovery, its optimal locus of control, and the degree to which various aspects are random (uncorrelated to the probability of future fitness improvements). Important open questions concern ways in which the properties of development—the layer between the mutated genotype and the selected phenotype—are evolved and in turn affect the evolutionary process [36,39,45,46,73,74,75,76,77,78]. Specifically, significant work has been conducted at the interface of evolution and learning—selectionist accounts of change and variational accounts of change respectively [30,61,62,66,79,80,81,82,83,84,85]. Significant progress has been made on the question of how evolution produces agents with behavioral competency in diverse problem spaces [17,86,87,88]. We have focused on a particular kind of competency—that of navigating anatomical morphospace [3,12,89,90]. More specifically, we here investigate in silico the evolutionary implications of the self-orchestrated process of morphogenesis, where local actions of single cells need to be aligned with a global policy of a multi-cellular collective to guide the formation of a large-scale tissue, in turn affecting the underlying evolutionary process. Work on developmental plasticity, chimeras, synthetic biobots, and the ability to overcome novel stressors has highlighted ways in which evolution seems to give rise to problem-solving machines, not fixed solutions to specific environments [91].

Thus, the problem-solving capacities of development, regeneration, and remodeling ensure that in many (perhaps most) kinds of organisms, the mapping from genotype to phenotype is not merely complex and indirect [92] (as schematized in Figure 1B) but actually enables evolution to search the space of behavior-shaping signals, not microstates, and exploit the modularity and triggers of complex downstream responses (cf. Figure 1C). We have previously argued that both evolution and human bioengineers face a range of unique problems and opportunities when dealing with the agential material of life—not passive or even just active matter but a substrate that has problem-solving competencies and agendas at many scales [93,94]. What selection sees is not the actual quality of the genome but the quality of the form and function of the flexible physiological “software” that runs on the genomically specified molecular hardware as schematically illustrated in Figure 2. This in turn suggests that the actual progress of evolution should be significantly impacted by the degree and kind of competency in the developmental architecture. Prior work has suggested a powerful feedback loop between the evolution of morphogenetic problem-solving and the effects of these competencies on the ability of evolutionary search to produce adaptive complexity [1,35,95]. Here, we construct and analyze a new model of evolving morphogenesis to study how different competency architectures within and among cells impact evolutionary metrics such as rate, robustness to noise, and transferability to new environmental challenges.

To quantitatively study the effects that different levels of competency of the decision-making centers in a multi-scale competency architecture have on the process of evolution, we here rely on tools from the research field of Artificial Life [96], which furthers computational and cybernetic models that mimic life-like behavior based on ideas taken from biology; a simple example is cellular automata (CAs) [97]. In such CAs, the (numerical) states of localized cells, organized on a discrete spatial grid, change in time via local update rules. Although typically rather simple “hardcoded” update rules are employed, CAs often display complex dynamics (cf. Conway’s Game of Life [98] or Lenia [99]) but are not known to exhibit homeostatic (closed-loop) activity. An extension of CAs, termed neural cellular automata (NCAs) [100], utilize artificial neural networks (ANNs) as more flexible trainable update rules, aiming to model the internal decision-making machinery of biological cells. Employing machine learning methods, such NCAs have been trained to perform self-orchestrated pattern formation (notably, of images from a single “seed” cell) [101] and even the co-evolution of a rigid robot’s morphology, and its controller has been demonstrated with such NCAs [102].

NCAs exhibit a striking resemblance to the genome-based multi-scale competency architecture of biological life [102] as illustrated in Figure 2: an organism’s entire building plan is encoded in its genome (corresponding to the NCA parameters), while its cells collectively run the self-orchestrated developmental program of morphogenesis (realized by the NCA layout and ANN architecture) via perception–action cycles at the uni-cellular level (cell state updates in the NCA, cf. Figure 2A). Starting from an initial cell state configuration of the NCA, the details of a virtual organism are then, step by step, “refined” in a collective self-organizing growth phase on the cellular level, and maintained against cell state errors later on in the virtual organism’s lifetime. Thus, a single NCA, once trained, guides the growth and integrity of a virtual organism’s tissue via intracellular information processing and intercellular communication, imitating in silico the multi-scale competency-based process of morphogenesis and morphostasis. (Notably, although an NCA update function—the cells’ ANN—is trainable in principle, current approaches pre-train (or evolve, as in our case) the ANN parameters to subsequently study the NCA behavior (such as simulated morphogenesis). Thus, an NCA’s self-orchestrated (developmental) program is defined by a particular set of ANN parameters rather than being acquired during the lifetime of the NCA. Here, we investigate the efficiency at which an evolutionary process arrives at satisfying parameters under various conditions.)

Here, we deploy a swarm of virtual uni-cellular agents on the spatial grid of an NCA. As illustrated in Figure 2A, each uni-cellular agent’s internal decision-making machinery is modeled by an ANN that allows each agent to independently perceive the cell states of its adjacent neighbors on the grid and propose cell state update actions to regulate its own cell state over time. The collective of cells thereby forms a spatial pattern or tissue of cell states on the NCA via local communication rules.

We utilize evolutionary algorithms (EAs) [103] as a simulated evolutionary process to optimize the parameters of such NCAs, so the uni-cellular agents evolve to collectively self-assemble a predefined target pattern of cell states in a fixed number of developmental steps; see Figure 2B for a flow-chart of the evolutionary process. We explicitly separate the NCA parameters into a structural and a functional part. The structural parameters describe the initial cell state, and the functional parameters the weights and biases of the ANN of each agent as illustrated by the “Genome” in Figure 2B. Both the structural and functional parts of the genome are compiled into a swarm of uni-cellular phenotypes on the grid of the NCA. Thus, starting from an initial cell state configuration, given by the structural part of the genome, the uni-cellular agents of the NCA run the developmental program of morphogenesis via successive perception–action cycles (see Figure 2A) to self-assemble in successive developmental steps a system-level phenotype, i.e., a two-dimensional pattern of cell states on the NCA. The deviation of these final cell state configurations from a desired target pattern—here, a Czech flag pattern or smiley-face pattern reminiscent of that of the amphibian craniofacial pre-pattern [104]—defines the phenotypic fitness score of a particular NCA realization. Based on an entire population of NCAs, and on the corresponding fitness scores, the EA successively samples the genomes of the next generation of NCAs, which, over time, evolve to reliably self-assemble the target pattern.

The conceptually simple process of cell state updates of NCAs and the ANN-based modeling of the uni-cellular decision-making allow us to interfere with (I) the reliability of the cell state update executions, and (II) with the computational capacity of the ANNs that guide each cell’s decision-making. To vary the former (I), we introduce a decision-making probability $P_{\mathrm{D}}$ that specifies the probability at which a proposed update of each individual cell is executed in the environment (or omitted otherwise). Thus, by tuning the decision-making probability from $P_{\mathrm{D}}=0$ to $P_{\mathrm{D}}=1$, we can continuously vary the behavior of the NCA from a direct-encoding scheme without competency to a multi-scale competency architecture with perfect reliability in cell state update executions.

To systematically vary the computational capacity of the involved ANNs (II), we introduce independent copies of a particular sub-module of the uni-cellular agents’ ANNs, i.e., of the policy module illustrated in Figure 2A (see Section 3.1 and Section 4.2 and Appendix A for details on the ANN architectures). This increases the number of evolvable parameters of the ANNs, which are responsible for performing the same operation, namely, interpreting the cell’s local environment and proposing a cell state update action. Thus, by taking the average output of all redundant policy modules of a single agent, a cell’s decision-making can be biased by the several redundant paths through which signals are transmitted in the ANN, inspired by error-correcting codes [105,106,107]. We explicitly define a redundancy number R that specifies how many redundant copies of the policy module are used in the ANNs of the cells of the NCA.

The decision-making probability (I) and the redundancy number (II) represent two levels of competency in our system (schematically illustrated by the red arrowin Figure 1C and Figure 2B), which we can scale continuously (I) or discretely (II) to systematically tune the behavior of an NCA. Throughout the manuscript, we refer to these two parameters as “competency levels”, but we would like to stress that many more options would have been possible to vary the competency in our system. For instance, the particular ANN architecture can have large effects on the competency of the uni-cellular agents; a systematic investigation thereof is out of the scope of this work. Here, we utilize two particular ANN architectures, one based on a Feedforward (FF) and one based on a recurrent ANN architecture [108] that is inspired by gene regulatory networks (GRNs) [109], which we thus term recurrent gene regulatory network (RGRN), see Appendix A for details.

To study the effects of different competency levels of the decision-making centers in a multi-scale competency architecture on the underlying evolutionary process of a morphogenesis task, we systematically vary in large-scale simulations the decision-making probability (I) and the redundancy number (II) of NCAs with FF and RGRN ANN architectures. Furthermore, we expose the corresponding NCAs to different noise conditions during cell state updates (III) and perform several statistically independent evolutionary searches at each parameter combination (I–III) to investigate the performance of the evolutionary process of finding solutions to such noisy pattern formation tasks.

The manuscript is organized as follows: In Section 3, we describe the numerical and computational methods applied herein. More specifically, we introduce NCAs in Section 3.1, and describe the neuroevolution approach used to optimize the NCAs ANN parameters based on ideas of evolution and natural selection via EAs in Section 3.2. We specify the particular morphogenetic problem we primarily focused on—the $8\times 8$ Czech flag task—in Section 4.1, and compare in Section 4.2 the efficiency of evolving the target pattern via a direct encoding scheme and a multi-scale competency architecture. In Section 4.3, we functionally define and systematically vary the different tunable competency levels in our system to illustrate the evolutionary implications of utilizing a multi-scale competency architecture rather than a direct encoding scheme for morphogenesis tasks. We then study the effects of allowing the evolutionary process to afford competency as a gene during optimization in Section 4.4, and eventually investigate our multi-scale competency approach for robustness and generalizability regarding system parameter changes in Section 4.5, and for transferability to modified target patterns in Section 4.6. We conclude in Section 5, and attach an appendix.

2. General Summary

Biological systems are composed of layers of organization, each level providing the foundation for the next higher level of abstraction: membranes, DNA, and proteins form cells, which then collectively organize into tissue and, in further hierarchical steps, into tissues, organs, bodies, swarms, ecosystems, etc. Each of these layers has a degree of ability to adapt in real-time to new conditions to establish and maintain specific outcomes in terms of physiological, metabolic, transcriptional, and anatomical spaces. In other words, evolution works with material that is not passive matter but rather has a degree of competency—an agential material that forms the layer between the genotype and the phenotype. Many scientific studies have been dedicated to investigating how evolution gives rise to such intriguing problem-solving machines we call organisms. In this study, we ask the reverse question: what is it like to evolve over such a material vs. one that passively maps genotypes into the form and function that selection operates over—how does it affect the process of evolution itself? We test this in silico by utilizing evolutionary algorithms to adapt the behavior of a swarm of virtual uni-cellular agents in large-scale simulations of virtual embryos. In our minimal model, the cells collectively self-assemble a predefined target tissue on a neural cellular automaton. We find that competency at the cellular level of our multi-scale model system strongly affects the resulting evolutionary process, as well as the generalizability, evolvability, and transferability of the evolved solutions, suggesting the profound evolutionary implications of the highly intricate multi-scale competency architecture of biological life.

3. Methods

3.1. Neural Cellular Automaton: A Multi-Agent Model for Morphogenesis

Cellular automata (CAs) have been introduced by von Neumann to study self-replicating machines [97] and are simple models for Artificial Life [96]. In CAs, a discrete spatial grid of cells is maintained over time, each cell i being attributed a binary, integer, real, or even vector-valued state ${\mathbf{c}}_i\left(t_k\right)$ at each step in time $t_k$. The cell states evolve over time via local updated rules ${\mathbf{c}}_i\left(t_{k+1}\right)=f_u\left({\mathcal{N}}_i\left(t_k\right)\right)$ as a function of its own ${\mathbf{c}}_i\left(t_k\right)$ and the numerical states ${\mathbf{c}}_{i_{\nu }}\left(t_k\right)$ of its $i_{\nu =1,\cdots ,N}$ neighboring cells on the grid that we collect in the matrix ${\mathcal{N}}_i\left(t_k\right)=\left({\mathbf{c}}_{i_0}\left(t_k\right),\cdots ,{\mathbf{c}}_{i_N}\left(t_k\right)\right)$ with $i_0\equiv i$. Although typically rather simple “hardcoded” (i.e., predefined) update rules $f_u(·)$ are employed, CAs often display complex dynamics and can even be utilized for universal computation (cf. Conway’s Game of Life [98] or Wolfram’s rule 110 [110,111]).

Neural cellular automata (NCAs) [100] extend CAs by replacing the local update rule with more flexible [112] artificial neural networks (ANNs) $f_u(·)\to f_{\theta }(·)$, where $\theta$ denotes the set of trainable parameters of the ANN (see Appendix A for details). Employing Machine Learning, such NCAs have been trained to perform self-orchestrated pattern formation [101] (notably, of RGB images from a single “seed” pixel) and even the co-evolution of a rigid robot’s morphology, and its controller has been demonstrated recently with NCAs in silico [102]. Such self-orchestrated pattern formation is reminiscent of the self-regulated development of a biological organism, from a single fertilized egg cell to a complex anatomical form. Thus, NCAs have been proposed as toy models for morphogenesis [101].

An NCA basically represents a grid of cells that are equipped with identical ANNs, each perceiving the numerical cell states of its host’s local environment, ${\mathcal{N}}_i\left(t_k\right)$, and proposing actions, ${\mathbf{a}}_i\left(t_k\right)=f_{\theta }\left({\mathcal{N}}_i\left(t_k\right)\right)$, to regulate its own cell state

$${\mathbf{c}}_i\left(t_{k+1}\right)={\mathbf{c}}_i\left(t_k\right)+{\mathbf{a}}_i\left(t_k\right)+{\xi }_{\mathbf{c}},$$

(1)

and, in turn, the cell states of its neighbors—where we also account for potential noise ${\xi }_{\mathbf{c}}$ in the environment during the process of morphogenesis. Thus, each cellular agent can only perceive the numerical states of its direct neighbors ${\mathcal{N}}_i\left(t_k\right)$ at an instant of time $t_k$ and, in turn, communicate with these neighbors via cell state updates ${\mathbf{c}}_i\left(t_{k+1}\right)$, following a policy $\pi \left({\mathcal{N}}_i\left(t_k\right)\right)\approx f_{\theta }\left({\mathcal{N}}_i\left(t_k\right)\right)$ that is approximated by an ANN with parameters $\theta$. Through the lens of Reinforcement Learning [113], an NCA can thus be understood as a trainable, locally communicating multi-agent system that can be utilized such that the collective of cells achieves a target system-level outcome (see Appendix B for details).

In contrast to previous contributions of in silico morphogenesis experiments in NCAs [101], we here do not use standard convolutional filters in our ANN architectures but utilize permutation-invariant ANNs with respect to a cell’s neighbors, ${\mathcal{N}}_i\left(t_k\right)$ (see Figure 2A for an illustration). Inspired by Ref. [114], this is achieved by partitioning a cell’s ANN into (i) a sensory part $f_{\theta }^{\left(s\right)}(·)$, preprocessing its own, and the state of each neighboring cell separately into a respective sensor embedding ${\epsilon }_{i_{\nu }}\left(t_k\right)=f_{\theta }^{\left(s\right)}\left({\mathbf{c}}_{i_{\nu }}\left(t_k\right)\right)\in {\mathbb{R}}^s$, for $i_{\nu =0,\cdots ,N}$. These neighbor-wise sensor embeddings are (ii) averaged into a cell’s context vector ${\mathbf{s}}_i\left(t_k\right)=\left[\frac{1}{N+1}{\sum }_{\nu =0}^N{\epsilon }_{i_{\nu }}\left(t_k\right)\right]\in {\mathbb{R}}^s$ of fixed size s, which is then used as the input of (iii) a controller ANN $f_{\theta }^{\left(c\right)}(·)$, potentially with recurrent feedback connections, that eventually outputs the cell’s action ${\mathbf{a}}_i\left(t_k\right)=f_{\theta }^{\left(c\right)}\left({\mathbf{s}}_i\left(t_k\right)\right)$; for details we refer to Appendix A.

Due to the mean aggregation of a cell’s sensory embedding, each cell completely loses its ability to spatially distinguish between neighboring (and even its own) state inputs and thus fully integrates into the tissue locally. We would like to stress the close relation of our approach to the concept of breaking down the computational boundaries of a cell’s “Self” via forgetting [93] and to the scaling of goals from a single agent’s to a system-level objective [95].

To model the developmental process of morphogenesis, we here employ NCAs on a two-dimensional $N_x\times N_y$ square grid with the objective that all cells of the grid assume their correct, predefined target cell type ${\widehat{g}}_i$ after a fixed number of $t_{\mathrm{D}}$ developmental time steps, starting from an initial cell state configuration ${\mathbf{c}}_i\left(0\right)$. We attribute a number of $N_{\mathrm{G}}$ elements ${\mathbf{g}}_i\left(t_k\right)\in {\mathbb{R}}^{N_{\mathrm{G}}}$ of the $N_{\mathrm{C}}$-dimensional cell state ${\mathbf{c}}_i\left(t_k\right)\in {\mathbb{R}}^{N_{\mathrm{C}}}$ of an NCA as indicators for expressing one of $1,\cdots ,N_{\mathrm{G}}$ discrete cell types such that ${\mathbf{c}}_i\left(t_k\right)={\mathbf{g}}_i\left(t_k\right)\cup {\mathbf{h}}_i\left(t_k\right)$; the remaining $N_{\mathrm{H}}=(N_{\mathrm{C}}-N_{\mathrm{G}})$ elements of the cell state represent hidden states ${\mathbf{h}}_i\left(t_k\right)\in {\mathbb{R}}^{N_{\mathrm{H}}}$ of a cell that can be utilized by the NCA for intercellular communication. We explicitly define each cell’s type $g_i\left(t_k\right)$ as the argument (i.e., the index) of the maximum element of the $N_{\mathrm{G}}$-dimensional indicator vector ${\mathbf{g}}_i\left(t_k\right)$:

$$g_i\left(t_k\right)=\underset{\mathbf{g}\in {\mathbb{R}}^{N_{\mathrm{G}}}}{argmax}\left({\mathbf{g}}_i\left(t_k\right)\right).$$

(2)

Training an NCA to assemble a predefined target pattern (realized by a set of $N_j=N_x\times N_y$ target cell types $\{{\widehat{g}}_1,\cdots ,{\widehat{g}}_{N_j}\}$ for the entire grid) thus boils down to finding a suitable set of NCA parameters (cf. “Genotype” in Figure 2B) that minimizes the deviation of each cell i’s type $g_i\left(t_{\mathrm{D}}\right)$ from ${\widehat{g}}_i$ after $t_{\mathrm{D}}$ developmental time steps, i.e., after the developmental stage of the virtual organism (cf. “System-level Phenotype” in Figure 2B, from left to right, and details below). Here, we are interested in the evolutionary implications of biologically inspired multi-scale competency architectures, the latter being modeled by our morphogenetic NCA implementation. We thus introduce in Section 3.2, and utilize in Section 4, evolutionary algorithms to evolve suitable sets of NCA parameters that maximize the fitness score based on comparing the “final” cell types of the NCA, $g_i\left(t_{\mathrm{D}}\right)$, with the predefined target cell types ${\widehat{g}}_i$.

3.2. Neuroevolution of NCAs: An Evolutionary Algorithm Approach to Morphogenesis

Evolutionary algorithms (EAs) are heuristic optimization algorithms that maintain and optimize a set, i.e., a population $\mathcal{X}=\{{\mathbf{x}}_1,\cdots ,{\mathbf{x}}_{N_{\mathrm{P}}}\}$ of parameters ${\mathbf{x}}_j\in {\mathbb{R}}^{\mathrm{X}}$, also termed individuals, over successive generations to maximize an objective function, or a fitness score $r\left({\mathbf{x}}_j\right):{\mathbb{R}}^{\mathrm{X}}\to \mathbb{R}$. Inspired by the ideas of natural selection and the DNA-based reproduction machinery of biological life, EAs (i) predominantly select the high-fitness individuals of a given population for reproduction, and utilize (ii) crossover and (iii) mutation operations to generate new offspring by (ii) merging the genomic material of two high-quality individuals from the current population ${\mathbf{x}}_o={\mathbf{x}}_j⨁{\mathbf{x}}_k$, and (iii) occasionally mutating the offspring genomes ${\mathbf{x}}_o\to {\mathbf{x}}_o+{\xi }_{\mathbf{x}}$ by adding (typically Gaussian) noise to the parameters; the ⨁ symbol indicates a genuine merging operation of two genomes, which may depend on the particular EA implementation. In that way, a population $\mathcal{X}$ of individuals is guided towards high-fitness regions in the parameter space ${\mathbb{R}}^{\mathrm{X}}$, typically over many generations of successive selection and reproduction cycles (i)–(iii).

In contrast to biological life, many use cases of EAs do not require a distinction between individuals in the parameter space, i.e., genotypes ${\mathbf{x}}_j$, and the corresponding organisms in their natural environment, i.e., phenotypes, ${\mathbf{p}}_j$: while the genetic crossover and mutation operations of biological reproduction rely on bio-molecular mechanisms at the level of RNA and DNA, i.e., are performed in the genotype space, selection typically happens at the much more abstract level of an organism’s natural environment, i.e., in the phenotype space. Carrying this through computationally can be resource-demanding, depending on the complexity of a simulated environment. Nevertheless, to address the asymmetry between genotypes and phenotypes in multi-scale competency architectures, it is essential to evaluate the fitness score of the EA in the phenotype space instead of the genotype space $r\left({\mathbf{x}}_j\right)\to r\left({\mathbf{p}}_j\right)$.

We explicitly separate the genotype and phenotype representations of individuals by introducing a biologically inspired developmental layer [1] in between genotypes and phenotypes ${\mathbf{x}}_j\underset{Layer}{\overset{Dev.}{\to }}{\mathbf{p}}_j$ as illustrated in Figure 2. More precisely, we follow Section 3.1 and model the developmental process of morphogenesis in silico by utilizing NCAs: we treat an NCA j’s parameters, such as the set of $i=1,\cdots ,N_j$ initial cell states ${\mathbf{x}}_j^{\left(\mathrm{S}\right)}={\left\{{\mathbf{c}}_i\left(0\right)\right\}}_j$ and the corresponding ANN parameters ${\mathbf{x}}_j^{\left(\mathrm{F}\right)}={\theta }_j$, as the (virtual) organism’s genome,

$${\mathbf{x}}_j={\mathbf{x}}_j^{\left(\mathrm{S}\right)}\cup {\mathbf{x}}_j^{\left(\mathrm{F}\right)}=\left({\left\{{\mathbf{c}}_i\left(0\right)\right\}}_j,{\theta }_j\right),$$

(3)

explicitly partitioning the genome into a structural (S) and a functional (F) part, as indicated by the superscripts. We then perform a fixed number of $t_{\mathrm{D}}$ developmental steps employing Equation (1) and interpret the corresponding set of “final” cell types ${\left\{g_i\left(t_{\mathrm{D}}\right)\right\}}_j$ of the entire NCA, cf. Equation (2), as the mature phenotype,

$${\mathbf{p}}_j={\left\{g_i\left(t_{\mathrm{D}}\right)\right\}}_j,$$

(4)

representing a two-dimensional tissue of cells.

In an effort to evolve the parameters ${\mathbf{x}}_j$ of an NCA j to achieve the morphogenesis of a two-dimensional spatial pattern of cell types ${\mathbf{p}}_j$ that resembles a pattern of predefined target cell types $\{{\widehat{g}}_1,\cdots ,{\widehat{g}}_{N_{\mathrm{j}}}\}$ of a total of $N_j$ cells on an $N_x\times N_y$ square grid (see Section 3.1), we define the phenotype-based fitness score $r\left({\mathbf{p}}_j\right)$ as

$$r\left({\mathbf{p}}_j\right)=(2n_j^{\left(\mathrm{G}\right)}-N_j)+r_T{n}_j^{\left(\mathrm{T}\right)}-r_S{n}_j^{\left(\mathrm{S}\right)},$$

(5)

where (i) $n_j^{\left(\mathrm{G}\right)}$ is the number of correctly assumed cell types $g_i\left(t_{\mathrm{D}}\right)={\widehat{g}}_i$ after $t_{\mathrm{D}}$ developmental steps, (ii) $n_j^{\left(\mathrm{T}\right)}$ is the number of time steps at which the entire target cell type pattern is correctly assumed, i.e., whenever $g_i(t_k\le t_{\mathrm{D}})={\widehat{g}}_i$ for all i, and (iii) $n_j^{\left(\mathrm{S}\right)}$ is the number of successive time steps $t_s$ and $t_{s+1}\le t_{\mathrm{D}}$, where all cell types stagnate, i.e., where $g_i\left(t_{s+1}\right)=g_i\left(t_s\right)$ for all i. With Equation (5), we thus reward the entire NCA j by counting all correctly assumed cell types after $t_D$ developmental steps (while discounting all incorrect cell types $g_i\left(t_{\mathrm{D}}\right)\ne {\widehat{g}}_i$), we reward maintaining the target pattern over time with a factor of $r_T$, and discount a stagnation of a suboptimal pattern over time by a factor of $r_S$. We consider the problem solved if a final fitness score of $N_j=N_x\times N_y$ is reached. Notably, there is no explicit fitness or reward feedback at the level of the uni-cellular agents in our system; the fitness score is solely used as the selection criterion for sampling the next evolutionary generations, so the cellular collective needs to evolve an intrinsic signaling mechanism to successfully perform the requested morphogenesis task.

The here proposed setting of genotypes ${\mathbf{x}}_j$, corresponding phenotypes ${\mathbf{p}}_j$, and associated fitness scores $r\left({\mathbf{p}}_j\right)$, given by Equations (3)–(5), respectively, can be used in combination with any black-box evolutionary or genetic algorithm. We rely on the well-established Covariance Matrix Adaptation Evolutionary Strategy (CMA-ES) [103] to simultaneously evolve the set of initial cell state configurations (i.e., structural genes, ${\mathbf{x}}_j^{\left(\mathrm{S}\right)}$) and the set of corresponding ANN parameters of an NCA (i.e., functional genes, ${\mathbf{x}}_j^{\left(\mathrm{F}\right)}$) with the objective of the purely self-orchestrated formation of two-dimensional spatial tissue as illustrated by Figure 2A and described by Equation (1).

4. Results

4.1. The System: An Agential Substrate Evolves to Self-Assemble the Czech Flag

Evolution works with an active rather than a passive substrate, i.e., with biological cells with agendas of their own [1]. Thus, at every stage of development during morphogenesis, collective decisions are made at vastly different length- and time scales within an organism, guiding the formation of the mature phenotype. We aim to model exactly this process via the neural cellular automata (NCAs) described in Section 3.1 and employ evolutionary algorithms (EAs) to evolve the parameters of such NCAs so the latter perform well on a target morphogenesis task, see Section 3.2. Without loss of generality, we consider an $N_x\times N_y=8\times 8$ Czech flag pattern (as a more complex version of the classic French flag problem of morphogenesis [115,116]) as the target pattern for our in silico morphogenesis experiments, with a fixed number of $N_j=64$ cells in total, $N_{\mathrm{G}}=3$ distinct cell types (colored blue, white, and red, respectively) and $N_{\mathrm{H}}=1$ hidden state, which renders the dimension of the NCA cell state $N_{\mathrm{C}}=4$. We use a square grid of cells with $N=8$ neighbors per cell and with fixed boundary conditions (see Appendix C for details).

Starting from a genotype ${\mathbf{x}}_j$ defined in Equation (3), we perform a number of $t_D=25$ developmental steps per morphogenesis experiment to “grow” a phenotype ${\mathbf{p}}_j$, described by Equation (4), based on which the fitness score $r\left({\mathbf{p}}_j\right)$ is evaluated following Equation (5) (see Figure 2B for an illustration of this process). During this entire process, we limit the magnitude of the numerical cell state values ${\mathbf{c}}_i\left(t_k\right)$ at all time steps $t_k$ to the interval $l_{\mathbf{c}}=[-3,3]$, and, analogously, limit the magnitude of the proposed actions ${\mathbf{a}}_i\left(t\right)$ of each uni-cellular agent to the interval $l_{\mathbf{a}}=[-1,1]$. This is achieved by clipping the numerical values of ${\mathbf{c}}_i\left(t_{k+1}\right)$ after a cell state update described by Equation (1), and the ANN outputs ${\mathbf{a}}_i\left(t\right)$ to the respective limits $l_{\mathbf{c}}$ and $l_{\mathbf{a}}$. The noise level ${\xi }_{\mathrm{c}}$ defined in Equation (1) is counted in units of the action limits $max\left(l_{\mathbf{a}}\right)$ and is thus sampled from a Gaussian distribution with zero mean and standard deviation ${\xi }_{\mathrm{c}}$ independently for each of the $N_{\mathrm{C}}=4$ cell state elements, thus affecting the cell state updates during development; the actual numerical values for the hyperparameters above turned out to be well suited for the problem at hand, especially to reasonably compare and discuss simulation results for the means of this contribution but are not crucial for the more general aspects of the evolutionary implications of multi-scale intelligence discussed here.

To study the effects of different types of decision-making machinery within a cell, we utilize two different architectures for the NCA artificial neural networks (ANNs), a Feedforward (FF) and a recurrent ANN inspired by gene regulatory networks [117,118,119,120] (RGRNs). (The terminology FF and RGRN stems from the respective agents’ Feedforward and Recurrent Gene Regulatory Network ANN controller layers (see Appendix A for details)). Notably, the RGRN-agent architecture augments cells with an internal memory that is independent of their states in the NCA and thus can not be accessed by the cells’ neighbors. To balance the length of the structural genome ${\mathbf{x}}^{\left(\mathrm{S}\right)}$ and functional genome ${\mathbf{x}}^{\left(\mathrm{F}\right)}$ defined in Equation (3), the two ANN architectures, FF and RGRN, are chosen such that the number of parameters $N_{\mathrm{FF}}=192$ and $N_{\mathrm{RGRN}}=164$ is roughly the same as the number of initial cell states $N_j\times N_C=64\times 4=256$. Thus, the ANNs utilized here—and detailed in

Table A1. Architecture and number of parameters in sensory and controller ANNs of the two different agent architectures used in this contribution. The total number of parameters depends on the redundancy number R of the controller ANN (cf. Section 4.1).

	Sensory ANN (Num. Param.)	Controller ANN (Num. Param.)	Total Num. Param.
FF agent	Feedforward (112)	Feedforward (82)	$112+R\times 82$
RGRN agent	Feedforward (112)	RGRN (52)	$112+R\times 52$

of Appendix A—are tiny compared to Ref. [101].

For each experiment of evolving the parameters of an NCA, i.e., for each independent run of the EA, we typically utilize a population $\mathcal{X}$ of $N_{\mathrm{P}}=96$ individuals and a maximum number of $N_{\mathrm{M}}=2000$ generations. As the EAs ultimate fitness criterion, we consider the average $F_j={\langle r\left({\mathbf{p}}_j\right)\rangle }_{N_{\mathrm{E}}}$ of $N_{\mathrm{E}}=8$ statistically independent fitness scores $r\left({\mathbf{p}}_j\right)$ of corresponding morphogenesis simulations starting from an individual j’s genotype ${\mathbf{x}}_j$ and resulting in a corresponding phenotype ${\mathbf{p}}_j$ after $t_D$ developmental steps; the developmental program described via Equation (1) is imperfect due to the developmental noise applied to the cell state updates and can thus lead to different, noise-induced phenotypic realizations. Typical values used here for the corresponding reward factors defined in Equation (5) are $r_T=0.25$ and $r_S=0.5$. We consider the problem solved if a fitness of $F_j=max\left(n_j^{\left(\mathrm{C}\right)}\right)=N_j=64$ is reached, but since we reward individuals to maintain the target pattern over time (via $r_T$), the maximum possible fitness score after $t_{\mathrm{D}}$ developmental time steps is $max\left(r_j\left({\mathbf{p}}_j\right)\right)=70.25$ in this example. Further details about the hyper-parameters of the EA and afforded computational resources can be found in Appendix D.

4.2. Direct vs. Multi-Scale Encoding: Cellular Competencies Affect System Level Evolvability

We aim in this contribution to investigate the evolutionary implications of biologically inspired multi-scale competency architectures [1,94]. Thus, we compare two qualitatively different evolutionary processes, both with the objective of morphogenetic pattern formation but whose genomes either (i) directly encode the phenotypic features of a two-dimensional target pattern (cf. Figure 1A), or (ii) encode the cellular competencies of a multi-scale architecture that gives rise to the same phenotypic features (cf. Figure 1C). Notably, different definitions of direct and indirect encodings in multi-agent systems have been used in the literature [54]. Here, we specifically distinguish between structural parameters ${\mathbf{x}}_j^{\left(\mathrm{S}\right)}={\left\{{\mathbf{c}}_i\left(0\right)\right\}}_j$ in the search space that directly encode features of the phenotype, i.e., specific initial cell types $g_i\left(0\right)\approx {\widehat{g}}_i$ and functional parameters ${\mathbf{x}}_j^{\left(\mathrm{F}\right)}={\theta }_j$ that indirectly, or rather functionally, encode the target pattern by parametrizing the intercellular communication and intracellular information processing competencies of the NCA that facilitate the self-orchestrated pattern formation process.

If no ANN at all were present in our model, i.e., ${\theta }_j=\left\{\right\}$, and in the absence of noise ${\xi }_{\mathbf{c}}=0$, we would re-establish a direct mapping between the genotype and phenotype as ${\mathbf{c}}_i\left(0\right)={\mathbf{c}}_i\left(t_{\mathrm{D}}\right)$, and thus a direct encoding of the target cell type pattern could be achieved $g_i\left(0\right)=g_i\left(t_{\mathrm{D}}\right)$. However, by default, we allow each cell to successively regulate its own cell state towards a target homeostatic value via an iterative perception–action cycle defined by Equation (1) and, moreover, to communicate in that way with neighboring cells. More specifically, each cell updates its cell state solely based on its own and the states of its adjacent neighbors, which, in turn, update their states based on their respective local environment. We explicitly avoid direct environmental feedback to the cells’ perception (such as an individual or collective reward signal) but fully restrict the NCA to intercellular communication (via cell state updates) and intracellular information processing. These uni-cellular agents thus exhibit a certain level of problem-solving competencies that can be utilized for the challenge at hand, in our case, for a collective system-level objective of forming a specific two-dimensional target pattern [95,101,102].

With the explicit partitioning of the genome into a structural part, i.e., ${\mathbf{x}}_j^{\left(\mathrm{S}\right)}$, and a functional part, i.e., ${\mathbf{x}}_j^{\left(\mathrm{F}\right)}$, we can study the effect of direct vs. multi-scale, or the competency-driven encoding of phenotypic traits in the process of evolution, and, moreover, quantitatively tackle the question of whether competent parts affect the process of evolution and evolvability. In any case, the initial cell state pattern is given by the structural part of the genome. Thus, in the absence of noise and without any active functional part in the genome, the set of initial cell states directly represents the final pattern, while otherwise, cell states can either be modified passively by noise in the system or actively through actions by the cells during the developmental stage. Thus, we employ CMA-ES [103] to either evolve the (i) structural, or both (ii) the structural and functional parts of the genome of an NCA simultaneously with the shared objective of self-assembling an $8\times 8$ Czech-flag pattern in $t_{\mathrm{D}}=25$ developmental time steps in the presence of noise ${\xi }_{\mathbf{c}}=0.25$ (cf. Section 3.2 and Section 4.1 for details). More explicitly, in case (i), we restrict the cell state update of the NCA by disabling all cell actions ${\mathbf{a}}_i\left(t_k\right)\to {\mathbf{a}}_i^{*}\left(t_k\right)=0$ but formally keep the functional part of the genome in the evolutionary search. In turn, we allow the NCA in case (ii) to afford both the structural and functional parts of the genome, thus giving the evolutionary process the opportunity to prioritize one over the other. We thus bias the evolutionary process in case (i) to effectively search the space of direct phenotypic encodings, while keeping the search space dimensions balanced in both cases. The results of this experiment are presented in Figure 3.

We can see in Figure 3 that both the evolution of the (i) direct and (ii) multi-scale encoding schemes of the target pattern can be achieved with the presented framework, and a fitness threshold of $F_j=64$ is reached after ≈300–600 generations, thus solving the problem. However, depending on the encoding scheme (i) or (ii), we can identify clear qualitative differences in the strategy and the “efficiency” of the evolutionary process, i.e., how many generations it takes to reach a certain fitness threshold and eventually converge (cf. top and bottom panel of Figure 3, respectively). The respective fitness score of the direct case (i) grows steadily and almost monotonically over successive generations until the threshold of $F_j=64$ is reached after 668 generations for that particular run, and the EA converges at a maximum fitness of $max{F}_j^{\left(i\right)}=70.25$ after 942 generations (see Section 4.1 for details on the threshold fitness values). In contrast, the evolutionary process of the multi-scale case (ii) undergoes significant leaps as reflected by the corresponding fitness score, which can increase rapidly if a suitable innovation, i.e., a favorable crossover or mutation event in the functional parameters ${\theta }_j$, occurs; the initial standard deviation of the fitness of the entire population is significantly larger compared to the direct case (i), yet the threshold fitness of $F_j=64$ is reached in 428 generations, and the EA converges after 679 generations (although at a lower maximum fitness of $max{F}_j^{\left(ii\right)}=69$).

The results presented in Section 4.2 are based on selected evolutionary optimization runs that are representative of related experiments with similar parameterizations. However, one should keep in mind that such results are always susceptible to chance in initial conditions or mutations in the EA but also to developmental noise; moreover, the hyperparameters of the evolutionary search or even the specific ANN architectures can influence the evolvability of such NCA systems. Thus, we present in Section 4.3 below a more statistically significant analysis of the evolutionary implications of direct and multi-scale encodings under various conditions of the cellular agents’ competency levels and the developmental noise.

Our separation of the genotype ${\mathbf{x}}_j$ into a structural ${\mathbf{x}}_j^{\left(\mathrm{S}\right)}={\left\{{\mathbf{c}}_i\left(0\right)\right\}}_j$ and into a functional part ${\mathbf{x}}_j^{\left(\mathrm{F}\right)}={\theta }_j$ moreover allows us to extract the structural (or genotypic) fitness along an entire evolutionary history: we define the structural fitness as the fitness score $r\left({\mathbf{p}}_j^{*}\right)$ of a phenotype ${\mathbf{p}}_j^{*}$ with evolved structural genes ${\left\{{\mathbf{c}}_i\left(0\right)\right\}}_j$ but with disabled agency ${\mathbf{a}}_i\left(t_k\right)\to {\mathbf{a}}_i^{*}\left(t_k\right)=0$. Notably, in the direct case (i), we have ${\mathbf{p}}_j={\mathbf{p}}_j^{*}$, which is illustrated in Figure 1A and reflected in the top panel of Figure 3; the structural fitness of the multi-scale case (ii) is explicitly visualized in the bottom panel of Figure 3. In the latter case, the structural fitness remains essentially detached from the phenotypic fitness ${\mathbf{p}}_j^{*}\ll {\mathbf{p}}_j$ during the entire evolutionary history (which also explains the convergence to a suboptimal maximal fitness level of $max\left(F_j\right)=69$ in this particular NCA solution, as the final Czech flag pattern first needs to be assembled from the corresponding imperfect initial cell configurations ${\mathbf{x}}_j^{\left(\mathrm{S}\right)}$). This all suggests that in contrast to (i), the EA in (ii) can make the most use of exploring the functional part of the genome, i.e., the space of behavior-shaping signaling and information processing [1], and, in turn, that the mere presence of competent parts drastically changes the search space accessible to evolution [3]; to show this explicitly, we present in Appendix E an illustration of the evolution of the morphogenesis process.

Interestingly, we still observe a slow but steady increase in the structural fitness in the long term in case (ii), owed to the small additional reward signal $r_T$ reinforcing the cellular agents to maintain the target pattern over time. This can most efficiently be achieved if the agent starts from a perfect set of initial cell types, representing a particular sub-space in the parameter space that might not necessarily be easily accessible to the EA at all stages during the evolutionary search. However, we would like to stress that such a slow transfer of problem-specific competencies from an agential, highly adaptive functional part ${\mathbf{x}}_j^{\left(\mathrm{F}\right)}$ to a rather rigid structural part ${\mathbf{x}}_j^{\left(\mathrm{S}\right)}$ of the genome could be a manifestation of the Baldwin effect [14].

While remaining neutral with respect to the system level fitness score, this competency transfer seems to affect the entire population presented in Figure 3 as reflected by the successively increasing population-averaged structural fitness score. Notably, and as detailed in Appendix F, we identified an associated decrease in the robustness against increasingly noisy cell state updates of the corresponding solutions with larger structural fitness. This suggests a reduction in uni-cellular competencies and might relate to the “paradox of robustness” discussed in Refs. [51,52,53,121,122,123,124,125,126,127]. Through a computational lens, such a competency transfer would also allow, as soon as the structural part of the genome is reliable enough, to repurpose the system’s competency to adapt to other independent tasks, and thus may facilitate the, in biology, ubiquitous effects of adaptability and polycomputing in related systems [128].

This all illustrates that an agential material [1,94], or more precisely, a substrate composed of competent parts, can have significant effects on the process of evolution and evolvability, especially for morphogenesis tasks. We thus conclude that, if competent parts are available, evolution prefers exploiting competency over direct encoding—if the environment requires competency at all (see discussion in Section 4.3). This leads to the conclusion that “competency at the lowest level greatly affects evolution and evolvability at the system level”.

4.3. Evolution Exploits Competency over Direct Encoding, if Necessary

Here, we investigate the effects of varying different levels of competency at the cellular level of a multi-scale competency architecture on the evolutionary process of morphogenesis. More specifically, we introduce the decision-making probability (I) $P_{\mathrm{D}}$, which constrains the ability of each cell individually to perform cell state updates in the environment: $P_{\mathrm{D}}$ defines the probability at which a proposed cell state update of each individual cell in the NCA is executed (or otherwise omitted). Thus, varying the decision-making probability from $P_{\mathrm{D}}=0$ to $P_{\mathrm{D}}=1$ smoothly transitions the system’s behavior from a direct encoding scheme without competency to an increasingly reliable multi-scale competency architecture (cf. Figure 3).

Another, somewhat hidden, level of competency we already discussed in Section 3.1 is each cell’s ANN architecture: An RGRN agent with internal memory can acquire and execute tasks differently than a simpler FF agent without any feedback connections except for its cell state ${\mathbf{c}}_i\left(t_k\right)$. Comparing the evolutionary implications of such functionally different ANN architectures is, however, not trivial, and is thus kept to a minimum here.

However, we parameterize both FF and RGRN agents such that their controller part of the ANNs (cf. Figure 1C, Section 3.1, and Appendix A) are (II) stacks of R redundant copies of the same controller ANN, each copy with its own set of parameters, which take the same pre-processed aggregated sensor embedding as input, and whose individual outputs are averaged into a single action-output of a cell. Inspired by redundancy in error-correcting codes [105,106], we thus allow cells with higher values of this redundancy number R, i.e., with many alternative routes through the controller part of the ANN, to—in principle—integrate environmental signals more generally compared to $R=1$, thus affecting the cells competency.

While scaling from $P_{\mathrm{D}}=0$ to $P_{\mathrm{D}}=1$ smoothly increases a cell’s competency to reliably regulate its cell state, increasing R enhances the computational capacities of the uni-cellular agents. Henceforward, we interpret (I) $P_{\mathrm{D}}$ and (II) R as two competency levels in our system, which we can vary (I) continuously and (II) discretely.

Analogous to Section 4.1 and Section 4.2, we thus utilize CMA-ES to evolve the genotypic parameters of an NCA to self-assemble the $8\times 8$ Czech flag pattern under different conditions (I–II), and expose the cells to different noise-levels (III) ${\xi }_{\mathbf{c}}$ during cell state updates defined in Equation (1).

In Figure 4A,B, we present the corresponding fitness scores of a maximum of 2000 generations of CMA-ES for different noise levels ${\xi }_{\mathbf{c}}\in [0,0.5]$, averaged over different values of the decision-making probability $P_{\mathrm{D}}\in \{0,12.5\%,25\%,50\%,100\%\}$ for both FFagents and RGRN agents. Moreover, for each realization of ${\xi }_{\mathbf{c}}$ and $P_{\mathrm{D}}$, we utilize experiments with different redundancy numbers $R\in \{1,2,4,8,16\}$ and employ 15 statistically independent EA runs for each parameter combination ${\xi }_{\mathbf{c}}$, $P_{\mathrm{D}}$, and R, and thus arrive at 75 statistically (and functionally, with respect to an agent’s ANN architecture) independent fitness trajectories per ($P_{\mathrm{D}}$, ${\xi }_{\mathbf{c}}$) combination; see Section 4.1 and Appendix D for more details on the EA parameters. In Figure 4C, we present the average number of generations it takes to solve the problem (to reach a fitness threshold of $F_j=64$) for each combination of $P_{\mathrm{D}}$ and ${\xi }_{\mathbf{c}}$, aggregated over the agents’ ANN architectures, FF or RGRN, and the respective redundancy numbers R for 15 statistically independent EA runs each; in Figure 4D, we present the data from Figure 4C but separately for both ANN architectures.

Figure 4. (A,B) The average fitness per generation of the best-performing individual in a population of 65 independent evolutionary processes of the 8 × 8 Czech flag task, evaluated from left to right at different noise levels (decision-making probabilities) and color-coded by the decision-making probabilities (noise-levels), for panels (A,B) respectively; solid lines mark average fitness values, the shaded area marks the standard deviation (to lower values only), and dashed lines indicate when an average fitness threshold of 64 is crossed, solving the problem. (C) Heatmap of the average generation number when the fitness threshold of 64 is crossed at particular combinations of the decision-making probability and noise level as detailed in (A,B); green and red arrows respectively indicate directions along $P_{\mathrm{D}}$ of increasing and decreasing values of the average fitness at fixed noise values. (D) Same as (C) but partitioned by the respective FF-agent or RGRN-agent architectures used in the respective CMA-ES runs.

Figure 4. (A,B) The average fitness per generation of the best-performing individual in a population of 65 independent evolutionary processes of the 8 × 8 Czech flag task, evaluated from left to right at different noise levels (decision-making probabilities) and color-coded by the decision-making probabilities (noise-levels), for panels (A,B) respectively; solid lines mark average fitness values, the shaded area marks the standard deviation (to lower values only), and dashed lines indicate when an average fitness threshold of 64 is crossed, solving the problem. (C) Heatmap of the average generation number when the fitness threshold of 64 is crossed at particular combinations of the decision-making probability and noise level as detailed in (A,B); green and red arrows respectively indicate directions along $P_{\mathrm{D}}$ of increasing and decreasing values of the average fitness at fixed noise values. (D) Same as (C) but partitioned by the respective FF-agent or RGRN-agent architectures used in the respective CMA-ES runs.

We observe in Figure 4 that, depending on these two parameters $P_{\mathrm{D}}$ and ${\xi }_{\mathbf{c}}$, for no- or very low noise levels ${\xi }_{\mathbf{c}}\approx 0$, the evolutionary search is most efficient, i.e., finds the solution in the fewest number of generations, on average, for low values of the competency level $P_{\mathrm{D}}\approx 0$. Thus, in these situations, direct encoding (achieved via $P_{\mathrm{D}}=0$) seems to be preferable to competency-driven encodings with $P_{\mathrm{D}}>0$ (as indicated by the bottom red arrow in Figure 4C); this is partly owed to the specific definition of the cell types $g_i\left(t_k\right)$ given by Equation (2), making a noiseless search very simple for the EA. However, for more realistic, noise conditions ${\xi }_{\mathbf{c}}>0$, the situation changes drastically. With increasing the noise level, the evolutionary efficiency of NCAs with higher competency levels is significantly greater compared to those with low competency levels, especially for the direct encoding scheme (as indicated by the green arrows in Figure 4C); for noise levels of ${\xi }_{\mathbf{c}}=0.375$ and $0.5$, the EA does not even find solutions for the direct encoding case with $P_{\mathrm{D}}=0$ in 2000 generations, as cell state updates become increasingly necessary to counteract the noise in the system. There is a clear trend of increasing the evolutionary efficiency in our in silico morphogenesis experiments by increasing the competency level for increasingly difficult environments with high noise levels.

Thus, we conclude that scaling competency has a strong effect on the process of evolution, and in realistic situations (with moderate to high noise), competency may greatly improve the evolutionary efficiency and evolvability of collective self-regulative systems.

It might be noteworthy that for evolving the $8\times 8$ Czech flag pattern, essentially no qualitative difference in the evolutionary efficiency between FF agents and RGRN agents with the given number of parameters was observed. Also, the evolutionary implications of utilizing a number of $R>1$ redundant copies within the controller ANNs of the cells of an NCA is much less pronounced, compared to the results depicted in Figure 4 as can be seen in Figure A8 of Appendix H. However, for more advanced problems such as assembling a $9\times 9$ smiley-face pattern (see Appendix G), RGRN agents seem to outperform simpler FF agents significantly in terms of evolutionary efficiency. Moreover, a larger redundancy number of $R\ge 4$ is required by the evolutionary process to more efficiently evolve the functional parameters of an NCA compared to a direct encoding scheme, hinting at a capacity bottleneck of the deployed ANNs.

4.4. There Is a Trade-Off between Competency and Direct Encoding Depending on Developmental Noise

A careful analysis of the results shown in Figure 4 reveals that the largest competency level of $P_{\mathrm{D}}=1$ does not result in the highest evolutionary efficiency for any presented noise level. On the contrary, populations with slightly lower competency levels of $P_{\mathrm{D}}=0.5$ or even $P_{\mathrm{D}}=0.25$ perform best at noise levels ${\xi }_{\mathbf{c}}\in \{0.25,0.375,0.5\}$ and $0.125$, respectively (as indicated by the green and red arrow ends in Figure 4C). In fact, cells with an initially random genome (comprising the ANN and initial cell state parameters) that are forced to make “uninformed”, i.e., initially random, decisions at every time step can interfere with the performance of the EA, as even initially perfect cell state configurations will be destroyed during such a randomized developmental stage. We suspect that this leads to corresponding delays in the evolutionary search compared to situations where populations can better rely on the structural part of the genome. Indeed, populations with “overconfident” actions can be trapped in local optima for many generations at all stages of the EA, which, in our system, may only be resolved by very specific but random mutations of the functional part of the genome (as we show later through Figure 5 in Section 4.4). This is reflected in Figure 4A,B by the large deviations in the average fitness trajectories for large $P_{\mathrm{D}}$ values.

The insights from the above lead to the questions of whether there is a “natural” or optimal competency level, with respect to the decision-making probability $P_{\mathrm{D}}$, or whether a mutable competency level can be utilized by the evolutionary process to improve the efficiency of guiding a population towards high-fitness regions in the parameter space. Thus, we include the decision-making probability as an additional competency gene ${\mathbf{x}}_j^{\left(\mathrm{C}\right)}$ into the NCA genome ${\mathbf{x}}_j\to {\mathbf{x}}_j={\mathbf{x}}_j^{\left(\mathrm{S}\right)}\cup {\mathbf{x}}_j^{\left(\mathrm{F}\right)}\cup {\mathbf{x}}_j^{\left(\mathrm{C}\right)}$, cf. Equation (3), and we perform in silico morphogenesis evolution experiments of the $8\times 8$ Czech flag pattern for different noise levels ${\xi }_{\mathbf{c}}$, analogous to Section 4.3. We analogously limit the numerical range of the competency gene ${\mathbf{x}}_j^{\left(\mathrm{C}\right)}$ to the interval $[-3,3]$, and extract the corresponding decision-making probability via $P_{\mathrm{D},j}=\frac{1}{2}(\tanh \left({\mathbf{x}}_j^{\left(\mathrm{C}\right)}\right)+1)$. Notably, for the experiments shown in this sub-section, we use $L_2$ regularization on the genotypic parameters ${\mathbf{x}}_j=(x_{j,1},\cdots ,x_{j,N_x})$ through subtracting $r_{{\mathrm{L}}_2}\times {\sum }_{i=1}^{N_x}{x}_{j,i}^2$ from the fitness score defined in Equation (5), with $r_{{\mathrm{L}}_2}=0.01$ (the $L_2$ regularization applied to ${\mathbf{x}}_j$ does not introduce a bias between the minimal $P_{\mathrm{D},j}=0$ and maximal $P_{\mathrm{D},j}=1$ competency levels, as the $L_2$ regularization is applied to ${\mathbf{x}}_j^{\left(\mathrm{C}\right)}$, not $P_{\mathrm{D},j}$; both $P_{\mathrm{D},j}$ and the $L_2$ regularization are symmetric with respect to the sign of ${\mathbf{x}}_j^{\left(\mathrm{C}\right)}$).

In Figure 5A,B, we present the evolved competency level for different noise levels after fitness thresholds of 64 and 70 are crossed, respectively, for 10 independent lineages per noise level for an RGRN architecture. The problem is considered solved at a fitness of 64, but since we reward the NCAs to maintain the target pattern over time via $r_T$ in Equation (5), a higher maximal fitness score of $70.25$ can be reached after $t_{\mathrm{D}}$ developmental steps for a sufficiently long evolution. Thus, we here relate Figure 5A to the evolutionary stage of having achieved the process of morphogenesis, and Figure 5B of having achieved morphostasis. For both cases, we essentially see two strategies emerging (see also Figure 5C–E): (i) one, where competency is maximized very early during the evolutionary process that then remains near the maximally possible value of $P_{\mathrm{D}}=1$, and (ii) a hybrid strategy where a significantly lower competency level is assumed that still allows to solve the problem.

Notably, strategy (i) is predominantly pursued at high noise levels, where large cell state fluctuations in the environment favor informed actions by the cellular agents. In contrast, the second strategy (ii) emerges more frequently in lineages evolved at low noise levels where, especially at very low noise levels ${\xi }_{\mathbf{c}}\approx 0$, most of the evolutionary processes result in solutions that avoid competency altogether, and a direct encoding scheme ($P_{\mathrm{D}}$ = 0) is evolved. Intermediate competency levels evolve in the corresponding intermediate noise regime. Following the trend of evolving morphogenesis (by crossing a fitness score of 64) to morphostatsis (by converging to the maximal fitness value of ≈70) in Figure 5A through B, we see that the two strategies, (i) and (ii), “sharpen” during the course of the evolutionary process such that $P_{\mathrm{D}}$ predominantly converges to the minimally or maximally possible values of 0 and 1, depending on the environmental conditions.

We also illustrate the evolved competency level of the particular lineage at all noise levels in Figure 5A,B, at which the respective fitness threshold is crossed in the minimal and maximal number of generations (and on average) amongst all 10 independent lineages per noise level. This clearly reveals that evolutionary processes that follow a more direct encoding strategy (ii) can evolve the problem at hand efficiently—if this is permitted by the developmental noise. However, when increasing the noise level, the evolutionary process can afford to evolve—or put differently, increasingly relies on evolving—the multi-cellular intelligence of the NCA to perform morphogenesis and morphostasis, thus following a third strategy (iii) that integrates both strategies (i) and (ii) in a non-trivial way. We observe in Figure 5A that the most efficient strategy for evolving morphogenesis seems indeed to be such a hybrid approach (iii), where a minimally necessary competency level is utilized at a specific noise level such that the corresponding evolutionary process can, again, be very efficient in solving the task.

Moreover, this also holds for the stage where morphostasis is reached, cf. Figure 5B: lineages that efficiently evolved to solve morphogenesis in our experiments also (typically) evolve to solve morphostasis efficiently. To emphasize this, we present in Figure 5C–E the “temporal dynamics” of the population-wise highest fitness and the corresponding competency level per generation for all lineages at selected noise levels ${\xi }_{\mathbf{c}}=\{0,0.125,0.25\}$; we also present for all corresponding lineages that have been evolved at these selected noise levels the genotypic competency level $P_{\mathrm{D},\mathrm{j}}$ against the corresponding phenotypic fitness scores $r_j$, and we find an apparent yet non-trivial relation between these two quantities: typically, an initial rise in fitness $r_j$ in early generations is associated with a decline in $P_{\mathrm{D},\mathrm{j}}$ which is more pronounced at lower noise levels. For intermediate noise levels $0<{\xi }_{\mathbf{c}}\ll 1$, we find that $P_{\mathrm{D},\mathrm{j}}$ often assumes a minimum (i.e., a minimally required yet finite competency level) when the evolutionary process reaches a fitness level of ≈64. We suspect that this allows the evolving morphogenetic process to establish good starting configurations based on changes in the structural genome, which can most efficiently be performed at a minimal(ly necessary) competency level given a certain developmental noise level in the environment. However, the competency is then quickly pulled towards a maximum level of $P_{\mathrm{D},\mathrm{j}}=1$ when the EA converges at a maximum fitness score of ≈70, at the morphostasis stage. For large noise levels, e.g., ${\xi }_{\mathbf{c}}=0.25$ as depicted in Figure 5E, the competency level rises with the corresponding fitness score in a much more monotonic way, emphasizing the necessity of the corresponding NCAs to utilize the cellular competency to solve the problem already at an early stage of the evolutionary process.

Curiously, we also see lineages that settle at the highest possible competency levels throughout their evolutionary history, even in conditions without noise as can be seen in Figure 5C: here, an initial “frozen accident” may cause an entire lineage to maintain high competency levels due to a lack of diversity in the corresponding gene, although this is not even necessary to solve the task. However, these high competency levels early on during the evolutionary process can cause the population to stagnate at sub-optimal regions in the parameter space for many generations if the corresponding policy of the cells is sub-optimal but rigid to strategy changes via small mutations in the genome. The population seems “trapped” until a favorable mutation or crossover event occurs in the functional part of the genome of an individual that guides the entire population towards higher fitness scores, eventually solving the problem. We suspect that this is also the reason for the lower evolutionary efficiency of the “most competent” configurations (with $P_{\mathrm{D}}=1$) compared to the slightly less competent cases (with $P_{\mathrm{D}}=0.5$) of the experiments depicted in Figure 4 [129].

Thus, we conclude, that if the evolutionary process can afford to evolve its own competency level, there seems to be a trade-off—during the entire course of the evolutionary process—between “going direct” or “going competent”, depending on the developmental noise. Moreover, the randomly initialized starting conditions may favor either direct or multi-scale encoding strategies, which may not only affect the “final” competency level that the evolutionary process converges to but can also greatly influence the efficiency of the evolutionary process itself. In general, the most efficient strategy for evolving morphogenesis tasks seems to be a non-trivial trade-off between finding a suitable initial cell state configuration that then allows the competency-based self-assembly of the target pattern to “kick in” and solve the task efficiently.

4.5. Competency Can Lead to Generalization

We are ultimately interested in the question of whether a substrate of competent parts shows the ability to generalize to environmental conditions that have never been experienced by its evolutionary predecessors, and hence would allow the evolutionary process to adapt an organism to changing environmental conditions more efficiently compared to a direct encoding scheme. Thus, we systematically vary in Figure 6 the system parameters, i.e., the noise level and the decision-making probability competency level, for selected NCA solutions of the Czech flag problem that have been trained with certain sets of the system parameters above.

For instance, we utilize NCA solutions that have been evolved to solve the $8\times 8$ Czech flag problem in $t_{\mathrm{D}}=25$ developmental steps (see above) under zero-noise conditions without and with evolvable competency. Here, we utilize such solutions for larger noise levels of ${\xi }_{\mathbf{c}}\in [0,0.5]$ and for lifetimes of 100 time steps and present the average fitness values of 100 statistically independent simulations at each particular noise level in Figure 6A,B, respectively, without any further evolutionary optimization. Analogously, we expose NCA solutions that have evolved with a competency level of $P_{\mathrm{D}}=0.5$ and noise levels of ${\xi }_{\mathbf{c}}=0.25$ and $0.5$, respectively, to vastly different noise levels of ${\xi }_{\mathbf{c}}\in [0,1]$ compared to the conditions during their respective evolutionary processes, and present the results in Figure 6C,D. Eventually, we again deploy the latter NCA solutions but vary the competency level $P_{\mathrm{C}}\in [0,1]$ instead, at respectively fixed noise levels of ${\xi }_{\mathbf{c}}=0.25$ and $0.5$, with the results depicted in Figure 6E,F. Notably, we only consider the “correctness” part of the fitness score, i.e., the first term in Equation (5) by setting $r_T=0$ and $r_S=0$.

The results in Figure 6 demonstrate that the performance of the here evolved NCAs, optimized with evolutionary methods to assemble and maintain a target morphology over time at particular system parameters, differs greatly between NCA solutions that follow the direct- or multi-scale encoding paradigms when subjected to novel environmental conditions: The typical fitness over the lifetime of an NCA without competency that encodes the target phenotype pattern directly (cf. Figure 6A) is constantly affected by random fluctuations and thus decreases in fewer time steps with increasing noise levels in a diffusive process; the duration of how long the corresponding maximum fitness score of 64 can be maintained and the speed at which the fitness eventually decays during the lifetime of the here discussed $8\times 8$ Czech flag NCA depend on the particular noise level and on the values of the initial cell states, which are limited numerically to the interval $[-3,3]$ for each cell. In contrast, NCA solutions with larger competency levels that have been evolved at finite noise-level conditions still perform well—and can maintain the target pattern for exceptionally long times—also when changing the system parameters dramatically (cf. Figure 6B–F); note the noise-level axis of ${\xi }_{\mathbf{c}}=0$ to 1, compared to the maximum noise levels of ${\xi }_{\mathbf{c}}=0.5$ during training.

The results in panel Figure 6B are especially curious, as the corresponding NCA has been trained to evolve its decision-making probability alongside the structural and functional parts of the genome at zero noise conditions. While no competency at all would have been required to solve this task, the presented NCA solution evolved to afford a maximum competency of $P_{\mathrm{D}}=1$ (cf. Figure 5C). Strikingly, this particular NCA is capable of resisting much larger noise levels of ${\xi }_{\mathbf{c}}\approx 0.25$ while maintaining the pattern perfectly for at least $t_{\mathrm{D}}=25$ steps, and the average fitness score of 100 independent solutions does still not drop below a certain threshold of ≈40–50 for even higher noise levels and for 100 time steps. Notably, there appears to be a bifurcation of the long-term behavior of these NCA solutions (not shown here) where the NCA—in some realizations—maintains the target pattern perfectly for long times, while in other independent runs, the fitness drops quickly.

In this sub-section, we thus show that NCAs that have evolved to assembly and maintain a target pattern within a relatively short developmental stage are capable of maintaining the corresponding target pattern over much longer time scales—without any further optimization—and thus show great signs of functional, morphostatic generalizability. Moreover, the here-discussed in silico morphogenesis and morphostasis model systems are capable of handling, essentially on the fly, system–parameter combinations that neither they nor their evolutionary ancestors ever experienced before. Thus, we conclude that such multi-scale competency architectures [1], whose substrate is composed of competent rather than passive parts, can be more than capable of generalizing to changes in their environment—within reasonable boundaries, of course—by allocating robust problem-solving competencies at many scales [93,94].

4.6. Competency Can Augment Transferability to New Problems

Deducing from the discussion in Section 4.5 about the generalizability of multi-scale competency architectures [1] towards changing environmental conditions, such systems should also exhibit increased evolvability and transferability properties to new problems: if such multi-scale competency architectures are capable of adapting their behavior towards changing environmental conditions on the fly during a single lifetime (cf. Figure 6), this has great consequences for the evolutionary process when environmental conditions change.

Thus, we utilized the NCA solution discussed in Figure 5C and Figure 6D and performed subsequent CMA-ES on the $8\times 8$ Czech flag problem at changed environmental conditions, i.e., at higher noise levels: only a single or, at most, a handful of generations are necessary for solving the task even at intermediate and high noise levels of ${\xi }_{\mathbf{c}}=0.25$ and $0.5$.

To emphasize the potential of the transferability of multi-scale competency architectures, we here investigate the adaptation capability of pre-evolved NCAs when their objective function is suddenly changed, i.e., when the environment starts selecting for different target patterns than the one they have originally been evolved for. More specifically, we utilize NCA solutions from Section 4.3, and discussed in Figure 4, which successfully solve the $8\times 8$ Czech-flag task, and additionally perform 1000 evolutionary cycles of CMA-ES on a related $8\times 8$ blue-, white-, red-, Viennese-, blue/white-, and blue/red-flag morphogenesis task for various noise and competency levels. We allow changes to both the structural and functional parts of the genomes of the pre-evolved NCA.

In Figure 7, we present the corresponding number of generations it takes for 10–60 CMA-ES runs on average to adapt a pre-evolved, i.e., “informed”, NCA solution that can solve the $8\times 8$ Czech-flag morphogenesis task to then solve the respective new morphogenesis task under different environmental conditions. We see a clear advantage in terms of the evolvability and adaptability of pre-evolved individuals at high-competency levels (in contrast to individuals with lower competency levels) so that adaptation can happen in as few as ≈10 generations. While the Czech→blue-, white-, and red-flag tasks are rather trivial (see top panels in Figure 7), computationally, the Czech→Viennese-, blue/white-, and blue/red-flag adaptation tasks (bottom panels in Figure 7) are more complicated. Still, the latter can be solved in as few as ≈20 generations compared to ≫100 generations of evolving a corresponding randomly initialized NCA to solve the Czech-flag problem from scratch as shown in Section 4.2 and Section 4.3.

Thus, we conclude that pre-evolved (or “informed”) competency at subordinate scales of a multi-scale competency architecture greatly enhances a collective system’s capability of adaptation. Thus, a competent and informed substrate has great effects on a multi-scale competency architecture’s evolvability towards changing environmental conditions and on the transferability of already acquired (evolved) solutions to new problems.

5. Conclusions

We have investigated the evolutionary implications of multi-scale intelligence on an example of the in silico morphogenesis of two-dimensional tissue of locally interacting cells that are equipped with tunable decision-making machinery. More specifically, we have utilized evolutionary algorithms (EAs) [103] to evolve the parameters of neural cellular automata (NCAs) [100] on morphogenesis tasks under various conditions of the competency level of the uni-cellular agents and the developmental noise in the system.

In this model of a multi-scale competency architecture [1], a two-dimensional grid of locally interacting cells is tasked to self-assemble and maintain a global spatial target pattern of predefined cell types, here, primarily of a two-dimensional, $8\times 8$ Czech flag pattern (we model and investigate the evolution of the process of morphogenesis and morphostasis in silico and deploy our framework to a self-orchestrated pattern formation task, primarily of a two-dimensional $8\times 8$ Czech flag pattern but also for other much more involved target shapes, such as a $9\times 9$ smiley face (see Appendix G)). Each uni-cellular agent’s internal decision-making machinery is modeled by an artificial neural network (ANN), allowing these cells to independently perceive the cell states of their adjacent neighbors on the grid and propose actions to regulate their own cell state over time via local communication rules. Both the ANN parameters and the initial cell states of all permanent cells represent the parameters of the NCA and are optimized by EAs for a specific in silico morphogenesis task, thus forming the functional and structural part of the system’s genome, respectively.

To investigate the effects of competency in a multi-scale competency architecture on the underlying evolutionary process, we introduce (I) a “competency level” parameter that controls the reliability of the uni-cellular agents of an NCA to regulate their cell types during a noisy developmental stage. This allows us to continuously scale the NCA competency level from a direct encoding scheme of the target pattern (no competency) to a multi-scale competency architecture that self-assembles the pattern with perfect reliability in cell decision executions. Furthermore, we introduce (II) a variable number of redundant sub-modules in the NCA ANN, which we utilize as another “axis” of redundancy- and computational capacity-based competency of the cells’ decision-making machinery.

In large-scale simulations, we systematically vary these two competency levels (I, II), expose the corresponding NCA to different noise conditions (III), and perform several statistically independent evolutionary searches at each parameter combination (I–III). In that way, we demonstrate that an evolutionary process proceeds significantly more rapidly (on average) on noisy pattern formation tasks when evolving the parameters of a multi-scale competency architecture compared to evolving the target pattern directly (with no competency involved).

Our multi-scale competency architecture model and the corresponding evolutionary optimization process comprise several scales: At the smallest scale (1), each structural and functional gene is represented by a floating point number. The functional genes parameterize the behavior of artificial neurons (2), our atomic decision-making centers, which are then hierarchically arranged into layers of artificial neurons (3), sub-modules of interconnected layers (4), to an ANN with a predefined architecture (5). Thus, even the uni-cellular phenotypes (6) in our system—ANN-based agents that maintain a particular internal cell state—are composites of smaller (proto-competent) decision-making centers down the hierarchical ladder. The composite uni-cellular agents perceive the cell states of their grid neighbors (7) on the NCA, perform potentially several cycles of internal calculations, and eventually update their own cell state in a single developmental step. In that way, clusters of different tissue types (8) may be formed in successive developmental steps. A fixed number of developmental steps comprise the lifetime of a single NCA, giving rise to self-assembled phenotypic tissue of cell types on the entire grid of the NCA (9), e.g., as in our case, to the Czech flag pattern. The quality of each individual in an evolutionary population of NCAs (10) is evaluated via a phenotypic fitness score, quantifying the deviation of the assumed cell types from a target pattern. Based on the fitness scores of a particular generation of NCAs, the genotypes of potentially better-adapted successor generations are successively sampled by the EA, closing the loop (1) and forming the largest scale in our system, an evolutionary lineage (11). Eventually, on a meta-scale (12), we compare the efficiency of the evolutionary process at different system parameters (I–III), i.e., at different competency- and noise levels, by analyzing the fitness trajectories of statistically independent lineages evaluated at the same system parameters.

We demonstrate that especially in the presence of developmental noise, affecting cell state updates during morphogenesis, the evolutionary process favors a multi-scale competency-based realization over a direct encoding scheme of the target pattern. Moreover, when the competency level itself is left as an evolvable parameter to the EA, there appears to be a non-trivial dynamical trade-off in the evolutionary process’ efficiency between exploiting the competency level of its components or the direct, pre-patterning-like encoding of the target pattern. We thus report that under realistic conditions (i.e., at moderate noise levels), an evolutionary process can be significantly more efficient when working with an agential rather than a passive material [1,94].

Notably, we explicitly omit a reward or fitness feedback from the environment to the NCAs’ uni-cellular agents’ perception, restricting the cells’ decision-making solely to the local communication of cell state updates between grid neighbors. Thus, the cells need to figure out their own communication protocol such that their single-agent decisions align with the global (multi-agent) system-level objectives of assembling the correct target pattern. These uni-cellular competencies are acquired over evolutionary time scales and can be understood as emergent behavior-shaping signaling [1].

On a more technical note, we specifically employ permutation-invariant ANNs as trainable update functions of the NCAs and successfully evolve the corresponding models to perform the here studied pattern formation tasks. We thus show that, contrary to previous assumptions [101,130], a perfect spatial resolution of neighboring cell states in an NCA is not necessary but that a mean-aggregated neighboring cell state can be sufficient for single cells to reliably contribute to the objective of a larger scale collective. Strikingly, we show that such uni-cellular agents do not even need to distinguish between their own states and the states of their neighbors to achieve this task, thus fully integrating into the tissue locally and essentially losing their individuality [93,95].

Also, in contrast to Ref. [101] and similar work, we do not start our morphogenesis experiments from a single “alive” cell but instead evolve the initial cell states of all permanent cells on the grid of an NCA, while the uni-cellular agents are constantly challenged to correct their state from developmental noise (notably, a process reminiscent of the denoising steps of Diffusion Models [131,132,133,134,135]). This allows us to explicitly distinguish between the evolutionary implications of (i) direct and (ii) multi-scale competency-based encodings of the target pattern, where we either constrain the evolutionary process to (i) only evolve the structural part of the genome, or to (ii) evolve both the structural and functional parts simultaneously. Admittedly, the choice of the structural part of the genome limits the scalability of the approach, as the size of the structural genome will grow correspondingly with the number of cells in the system. However, as occurs with biomechanical [136], biochemical [137,138] and bioelectric pre-patterning [8,94,139], the initial states of an NCA of moderate size could be seen as a coarse-grained scaffold, based on which an NCA of a potentially much higher resolution can run its multi-scale competency-based developmental program to self-assemble a high-resolution target pattern [140]. Alternatively, we suggest utilizing a Compositional Pattern Producing Network (CPPN) [120,141] to indirectly encode the initial states of all cells on the grid of an NCA, allowing such a hybrid approach to perform in silico morphogenesis at scale. Unfortunately, it has been proven difficult, if not unfeasible, to exactly reproduce predefined target patterns reliably with the neuroevolution of CPPNs alone [142], which is why we here refrained from this approach; we emphasize, however, that gradient-based methods such as Neural Radiance Fields (NeRFs) [143] to train CPPN-like architectures might be an interesting workaround.

We find that fully evolved NCA solutions, capable of performing the morphogenesis tasks discussed above, show great signs of generalizability toward changing the system parameters, and can—without any further evolutionary optimization or training—handle noise and competency levels that are vastly different from the training conditions. Consequently, this leads to the increased evolvability of such competency-based models to changing environmental conditions: a subsequent evolutionary process can adapt a pre-evolved solution to altered environmental conditions within a handful or sometimes even a single generation. Moreover, we demonstrate that such pre-evolved NCA solutions can even quickly adapt to new, yet related problems. Specifically, we modify the objective function of our evolutionary process from the $8\times 8$ Czech flag task to self-assemble a blue-, red-, white-, Viennese-, diagonal blue/white and blue/red flag instead, respectively. In most of these situations, an adaptation of an existing NCA solution to the new problem can be performed in significantly fewer generations than evolving the initial $8\times 8$ Czech flag task from a randomly initialized configuration. Typically, these adaptations happen faster the larger the competency level of the NCA is, while for the direct encoding scheme (or in situations with low competency), the structural part of the genome is too dominant to allow quick adaptations by the EA. This suggests that multi-scale competency architectures allow the underlying evolutionary process to not over-train on priors, thus augmenting adaptability through a competent substrate.

We conclude that not only can evolutionary processes efficiently utilize and bring forth the intriguing multi-scale problem-solving machines of biological life but that the efficiency of such evolutionary processes, as well as the generalization abilities, evolvability, and transferability of the corresponding phenotypic outcomes, are strongly affected by the level of competency of the underlying agential material. An intriguing open question is whether this implies a positive feedback loop that enhances that quality over time. Judging from the considerable effects of scaling the competency in the here-studied still shallow multi-scale system on a rather simple in silico evolutionary process (i.e., CMA-ES [103]), it becomes increasingly evident that the vastly more complex multi-scale competency architecture of biological life cycles back and thus affects the process of evolution itself.

One of the key opportunities for future work is to apply the ideas explored here in silico to the understanding of biological mechanisms in natural systems, and to the design of new synthetic constructs via bioengineering [144,145]. It is now known that living tissues implement a kind of multi-scale competency architecture [94]. Problem-solving capacities at one scale, for example, the ability to navigate anatomical morphospace despite perturbations (embryogenesis and regeneration), rely on the communication and cooperation of subunits. One of the emerging modalities for this underlying communication is bioelectricity [89], and future work will explore the mapping from the bioelectrical dynamics that implement neural-like [32] computations within cell networks to the robust plasticity observed with respect to dynamic form and function. A closely related set of questions concerns the implications of this computational property of all cells, not just neurons, for the material on which biological evolution acts [1].

For future directions, our multi-scale competency framework is easily extendable to simulate tissue growth via cell migration or division actions proposed by the underlying ANNs of the NCA. More specifically, our framework allows for a minimal set of biologically relevant uni-cellular actions, such as cell state update, cell division, migration, cell death, and an identity operation, only constrained by the NCA spatial grid. Furthermore, the framework is capable of handling flexible ANN architectures, potentially allowing us to investigate intriguing competencies, such as active inference [146], through utilizing world model architectures [147] in a (neuro)evolutionary context. Our system, so far, has a fixed hierarchical architecture that deviates from the scale-free competency architecture of biological life with open-ended functional adaptation (where any abstraction layer becomes the basis for the next one). Thus, in future work, we aim to model precisely this behavior by introducing multiple layers of horizontal communication pathways in an NCA that the ANN-based agents can dynamically traverse in the vertical direction. Moreover, by choosing a proper fitness function related to measuring scale-invariant pattern formation [107,148], critical dynamics [149,150,151,152,153,154], or applying the free-energy principle [146,155], we are confident that we will achieve a biologically more accurate model of the scale-free dynamics and open-ended evolution of life. Such computational models could thus further quantitative studies of the communication strategies and boundaries of individual and groups of cells in an agential, potentially adversarial umwelt, with possible applications in individual and collective aging (as morphostasis defects) [156,157], or cancer research [93,94,139].