Self-Referential Introspection in Large Language Models: The Critical Threshold for Recursive Self-Improvement

Abstract

The pursuit of self-evolving AI raises a critical question: when is autonomous self-improvement sustainable rather than degenerative? Drawing an analogy to von Neumann’s complexity threshold for self-reproducing automata, we argue that sustainable recursive self-improvement in large language models (LLMs) requires a functional analogue: introspection—the system’s capacity to simulate its own operations and target modifications. Grounded in Kleene’s Second Recursion Theorem, we construct such introspective self-improvement programs and prove their key properties: completeness of self-modification, necessity of the reflective architecture, undecidability of improvement in general, and equivalence with Schmidhuber’s Gödel machine under a rewrite-equivalence notion, which transfers the global optimality guarantee. An empirical review, organized around these functional criteria, suggests that current LLMs exhibit only quasi-introspection.The available evidence does not establish complete introspection in the formal sense developed here, while pointing to several candidate structural bottlenecks, including incomplete self-access, feedforward processing, and limited computational depth. We outline architectural paths toward the threshold and discuss the safety implications of crossing it.

1. Introduction: The Bet on Self-Evolving AI and Introspection

The past two years have witnessed a decisive strategic shift across the AI industry. Major laboratories—OpenAI, Google DeepMind, Anthropic, Meta, Alibaba, and others—are no longer content with building static foundation models that must be retrained by human engineers. Instead, they are investing heavily in self-evolving AI systems: agents that autonomously refine their own prompts, strategies, tool usage, memory structures, and even underlying parameters through iterative interaction with the environment [1,2,3].

This ambition is not new. The concept of recursive self-improvement (RSI)—a system that not only improves itself but improves its capacity to improve itself—has been a central idea in the theory of artificial general intelligence since at least Good’s [4] “intelligence explosion” hypothesis and Yudkowsky’s [5] Seed AI framework. What is new is that large language models (LLMs), for the first time, appear to provide a plausible substrate for such systems. Recent works have demonstrated LLMs that refine their own reasoning through self-reflection [6,7], optimize their own prompts [8], generate and evaluate their own training data [9,10], and even modify their own source code at runtime [11,12].

Yet a sobering empirical pattern has emerged: most autonomous self-improvement loops exhibit rapid asymptotic saturation. For instance, the Self-Refine paradigm plateaus after a mere few iterations [7]. This systemic limitation is clarified by Huang et al. [13], who demonstrated that absent external ground-truth feedback, contemporary LLMs lack the intrinsic corrective capacity to rectify their own reasoning errors, frequently suffering performance degradation following self-revision. Consequently, even the most sophisticated self-evolving frameworks—such as the Darwin Gödel Machine [12]—remain strictly tethered to external benchmarks and human-engineered evaluation criteria to sustain behavioral progress.

Another critical obstacle to sustained self-improvement in LLMs is model collapse—a degenerative process wherein successive generations of generative models are progressively polluted by their own synthetic outputs [14]. Dohmatob et al. [15] further demonstrated that even a minuscule fraction of data pollution can trigger this collapse, meaning that scaling the training set ceases to yield performance gains. Scaling this effect globally, as AI-generated text permeates the web and shifts human reliance, the broader knowledge economy risks tipping into a steady state of “knowledge collapse,” where foundational general knowledge gradually vanishes despite the availability of high-quality personalized advice [16]. Consequently, a fundamental question emerges: Is there an absolute barrier to sustained recursive self-improvement, and if so, what is its formal nature?

We propose that this bottleneck is deeply intertwined with von Neumann’s concept of a complexity threshold”—a term we use here to encapsulate his original phrasing, the critical size of complication” [17]. Through a comparative analysis of man-made automata and biological organisms, von Neumann investigated how a machine might replicate itself. He observed that there is a minimum number of parts below which complication is degenerative, in the sense that if one automaton makes another the second is less complex than the first, but above which it is possible for an automaton to construct other automata of equal or higher complexity” [17]. While admitting the difficulty of strictly defining complication”—abstractly describing it as “…the potentiality to do things”—he maintained that this minimum threshold is dictated by the capacity for self-reproduction. Crucially, he deduced that a self-reproducing automaton requires a dual architecture: the functional automaton itself and a tape containing its own description [17]. Kolmogorov complexity [18] and related algorithmic-information measures [19] may provide formal proxies for descriptional complexity in the discrete setting. We do not assume, however, that these measures define a universal scalar complexity or threshold for LLMs.

Fundamentally, self-reproduction is a physical manifestation of self-reference [20], and the theoretical existence of such self-reproducing automata follows directly from Kleene’s second recursion theorem [21,22,23]. Thus, the complexity threshold that prevents degenerative decay in recursive replication is fundamentally rooted in the mathematics of self-reference.

This paper draws a structural analogy between von Neumann’s self-reproducing automata and contemporary self-improving AI systems. We formulate the hypothesis that an analogous complexity threshold may govern LLM self-evolution. Because no scalar metric for an LLM’s “complexity” is currently available, we characterize this threshold functionally: it concerns whether a system can form, access, and use a sufficiently complete description of the computational unit undergoing improvement. Crossing this threshold may make sustained autonomous recursive self-improvement possible, whereas below it, improvement may remain partial, externally scaffolded, or vulnerable to degradation. The precise outcome depends on the system’s evaluation criteria, modification mechanism, and architectural organization.

We argue that introspection—the capacity of a system to simulate and evaluate its own operational behavior—is a candidate necessary condition for complete autonomous recursive self-improvement in the formal setting developed here. Using Quine-like techniques, we construct an introspection program that simulates its own source code within an embedded virtual machine [22]. This construction establishes the computable existence of a self-referential program and provides a formal basis for analyzing self-modification and self-improvement architectures, including related proposals such as the Gödel machine [24]. It does not, by itself, establish that current LLMs implement such a mechanism.

The Quine-based construction has a reflective, two-part organization comprising an active execution body and a generative description of that body. For complete self-modification, the computational unit undergoing improvement must be able to recover and modify an adequate description of its own relevant structure. If the unit is a bare LLM, this requirement extends to its parameters and the mechanisms responsible for their execution, although the description need not be a literal, uncompressed copy of every weight. This requirement creates a structural challenge for modern LLMs: current systems exhibit emergent, quasi-introspective behaviors, such as metacognitive self-monitoring, but existing evidence does not show that they possess complete self-access or self-modification authority in the formal sense developed here. We therefore treat the claim that current LLMs have not crossed the introspection threshold as an architectural assessment, not as a direct consequence of Kleene’s theorem alone.

This paper is organized as follows (see Figure 1):

Figure 1. Recursive self-improvement and introspection in current LLMs. The diagram relates recursive self-improvement to complexity and introspection thresholds, summarizes evidence for quasi-introspective capabilities in current LLMs, and distinguishes proposed structural limitations from corresponding research directions. Dashed lines denote the proposed introspection threshold. For the corresponding references, see [4,25,26].

1.

We review the literature on recursive self-improvement and self-evolving LLMs and AI systems, classify existing approaches, and identify their limitations with respect to autonomous and sustained self-improvement (Section 2).

2.

We revisit von Neumann’s complexity threshold and develop the formal theory of self-reference, including Quine constructions and Kleene’s Second Recursion Theorem. We use this framework to construct introspective self-improvement programs and establish their formal properties under explicit computability and architectural assumptions (Section 3 and Appendix A). Their relation to Schmidhuber’s Gödel machine is analyzed formally in Appendix B and Appendix C.

3.

We survey empirical studies of self-modeling, self-simulation, self-evaluation, and self-modification in contemporary LLMs. We characterize current systems as exhibiting quasi-introspection, while finding no evidence that they satisfy complete introspection in the formal sense developed in Section 3 and Section 4.

4.

We examine the structural gap between current LLM architectures and the formal requirements of introspective self-improvement, discuss possible paths toward the introspection threshold, and consider its implications for consciousness, recursive self-improvement, AI safety, and institutional governance (Section 5 and Section 6). Section 7 concludes with open problems.

2. Self-Improved AI Systems

2.1. From the Singularity to Recursive Self-Improvement

The hypothesis that machine intelligence could surpass human capabilities and trigger runaway growth dates back to Good [4], who observed that an ultraintelligent machine” could design increasingly superior systems, culminating in what he termed an intelligence explosion. This positive-feedback loop—wherein a system enhances its own capacity for design—constitutes the conceptual bedrock of recursive self-improvement (RSI). Although formal terminology emerged decades later, Yudkowsky [5,27] codified RSI within the framework of seed AI,” defining it as self-improvement that expands the system’s capacity to improve rather than merely optimizing its task performance. Concurrently, Schmidhuber [24]’s Gödel Machine provided this concept with its strictest mathematical realization: an agent capable of rewriting any part of its own code, provided it can formally prove the modification beneficial.

This conceptual evolution yields a clear chain of causal dependencies: a singularity [25] is the hypothesized end state; an intelligence explosion is the process of realization; RSI is the driving mechanism; and—as argued from Section 3 onward—RSI itself fundamentally hinges on self-reference. By tracing this causal chain to its bedrock, we establish the core contribution of this paper: rather than debating whether an intelligence explosion is imminent or inevitable, we investigate the precise formal capacity a system must possess for self-improvement to compound recursively rather than degenerate.

2.2. A Classification of Self-Improving Systems

To formalize which capacity undergoes modification, Yampolskiy [26] proposes a three-level meta-taxonomy:

• Level 1 (Self-modification) alters system code without necessarily improving performance (e.g., polymorphic viruses);
• Level 2 (Weak RSI) optimizes performance within a fixed algorithmic framework, inevitably yielding diminishing returns (e.g., neural architecture search);
• Level 3 (Strong RSI) improves the underlying capability to improve, initiating a potentially unbounded feedback loop.

The boundary between Weak and Strong RSI hinges on whether the improvement mechanism itself has improved. Schmidhuber’s Gödel Machine instantiates this transition precisely: upon proving a single target theorem, all subsequent meta-levels collapse into a uniform layer [24]. To date, no deployed system has convincingly achieved Level 3.

Mapping existing models onto this spectrum allows us to survey concrete self-evolving LLMs and agents along two functional axes: the specific architectural layer modified, and the mechanism utilized to verify improvements.

2.3. Self-Evolving LLMs and Agents: Which Layer, and How Verified?

Virtually all contemporary LLMs and agents qualify as Level 2 (Weak RSI) self-improving systems, as they fundamentally rely on machine learning algorithms to optimize task performance. Crucially, by leveraging in-context learning or test-time adaptation, these models can execute self-improvement dynamically during inference. In this section, we review the literature through the critical lens of the self-modification layer.

The layer of self-modification defines the specific locus within a system’s computational stack where architectural changes are applied. Existing frameworks span four escalating layers: (i) prompt/context, (ii) memory/skill, (iii) weight, and (iv) scaffolding-code. Empirically, Self-Refine [7] and Reflexion [6] operate at the prompt/context layer through textual self-critique. SkillRL [28] and AlphaApollo [29] inhabit the memory/skill layer by accumulating reusable libraries or tools. Moving deeper, GENOME [30] targets the weight layer by applying evolutionary operators directly to LoRA parameters. At the most fundamental scaffolding-code layer, STOP [31], Gödel Agent [11], and the Darwin Gödel Machine (DGM) [12] permit systems to rewrite their own orchestration code or manage agent archives. Conversely, RISE [32] cuts across these layers by fine-tuning models to iteratively revise failed execution attempts.

Beyond the modification locus, a self-improving system must rigorously evaluate the validity of its proposed structural changes. Contemporary literature exhibits a definitive consensus: pragmatic deployments have abandoned the uncomputable verification guarantees of Schmidhuber’s original Gödel Machine [24] in favor of localized statistical approximations. For instance, the Huxley Gödel Machine (HGM) leverages Clade-Metaproductivity (CMP) to estimate the long-term fitness of evolutionary lineages within bounded tree searches. Alternatively, the Red Queen Gödel Machine (RQGM) [33] employs controlled utility evolution, co-evolving evaluators and agents within localized epochs to stabilize non-stationary optimization signals. Sliding further down the formal spectrum, heuristic frameworks like STOP [31] bypass internal verification entirely, validating programmatic modifications solely via empirical performance on external benchmarks. Rather than duplicating comprehensive existing surveys [1,2,3], our analysis untangles the fundamental structural constraints that compel these diverse architectures to externalize their verification signals.

2.4. Limitations

The empirical limitations of contemporary self-improvement split into internal evaluative constraints and orthogonal distributional hazards.

The primary bottleneck resides in the unreliability of the internal feedback loop. Textual self-refinement routinely saturates within few iterations [7], and models routinely fail to self-correct even when errors are explicitly localized [32]. Furthermore, internal evaluation can be actively counterproductive, inducing “metacognitive hallucinations” where flawed reflection reinforces incorrect beliefs [34]. This untrusted feedback restricts systems to shallow self-modifications, as the model cannot reliably project the downstream utility of its own structural changes.

Conversely, model collapse introduces an independent bottleneck inherent to the recursive data substrate. This pathology is triggered when AI-generated outputs inadvertently pollute the original training substrate. As successive generations of models are trained on data increasingly contaminated by historical synthetic outputs, the system progressively erodes distributional diversity and collapses functional modes—a degenerative process that remains robust to scale and is highly sensitive to even minimal synthetic fractions [14,15]. We re-interpret model collapse not merely as a data-quality issue, but as a collapse of effective information entropy successive generational training on synthetic outputs reduces the Shannon entropy/Kolmogorov complexity of the training distribution, analogous to a statistical mechanical mixing process approaching a degenerate fixed point. Because this degradation occurs independently of architectural integrity—affecting even a theoretically perfect self-model—it represents an orthogonal explanation for capability saturation.

Whether observed test-time optimization reflects genuine strategy or mere capability elicitation remains contested. Nonetheless, the empirical consensus is clear: current self-improvement consistently saturates, internal verification is fragile, and self-modification remains surface-level. This systemic failure indicates that current architectures remain bounded below a fundamental capability threshold. Bridging this gap requires moving beyond incremental engineering toward a rigorous formalization of how a program operates upon itself, positioning self-reference as our foundational mathematical bedrock.

3. Self-Reference: From Self-Reproducing to Introspection

We begin by reviewing von Neumann’s work on self-reproducing automata and the associated complexity threshold. Next, we introduce the formal theory of self-reference, focusing on Quine programs and Kleene’s second recursion theorem. Finally, grounded in this theorem, we introduce the concept of an introspection program and propose our hypothesis regarding the introspection threshold for RSI.

3.1. Von Neumann’s Self-Reproducing Automata

In the late 1940s, John von Neumann began to think about the problem of how an automaton can reproduce itself. The major motivation behind his thinking was to resolve the core paradox that artificial machines can only create simpler devices while living organisms replicate and evolve into more complex forms. This leads to an important conception: the “complexity threshold.”

3.1.1. Complexity Threshold

We use the term “complexity threshold” as a modern shorthand for von Neumann’s original phrase, the “critical size of complication.” We did not invent the concept; we renamed his informal notion to make it operational in the context of recursive self-improvement. In the original manuscript, von Neumann used “complication” to describe the complexity of an automaton; however, he did not give a precise definition of this concept, but rather a description [17]:

“I know no adequate name for it, but it is best described by calling it ‘complication.’ It is effectivity in complication, or the potentiality to do things. I am not thinking about how involved the object is, but how involved its purposive operations are. In this sense, an object is of the highest degree of complexity if it can do very difficult and involved things.”

Second, von Neumann used “minimum number of parts” or “critical size” to describe the threshold of complication:

“There is a minimum number of parts below which complication is degenerative, in the sense that if one automaton makes another the second is less complex than the first, but above which it is possible for an automaton to construct other automata of equal or higher complexity.

…

There is thus this completely decisive property of complexity, that there exists a critical size below which the process of synthesis is degenerative, but above which the phenomenon of synthesis, if properly arranged, can become explosive, in other words, where syntheses of automata can proceed in such a manner that each automaton will produce other automata which are more complex and of higher potentialities than itself.”

Therefore, this critical size of complication is of great importance not only for building more powerful artificial automata but also for a unified theory of automata suitable for both man-made machines like concurrent vacuum-tube computers and natural living systems like the human nervous system. Nevertheless, von Neumann had no clear definition of the complexity threshold; he was sure, however, that it is related to self-reproduction.

With modern notions of Kolmogrov complexity [18] and algorithmic information entropy [19], the complexity threshold can be quantified by the minimum description length of a self-reproducing automaton.

3.1.2. Self-Reproducing Automata

At first appearance, self-reproduction may not seem as important as von Neumann emphasized—it might even seem trivial in the information age, since copying a file or scanning and copying a piece of paper is so easy and common. However, what self-reproducing stresses is “self”: a system or an automaton can only make a copy of itself independent of any other things. Therefore, scanning and copying a paper of information requires a scanner and a copier which are not included in the paper itself, and copying a file on a computer also needs the operating system to support the “Copy” command.

However, how is it possible for an isolated system or automaton to reproduce itself from nothing? Here, von Neumann understood reproduction as a composition process of a set of elements. Von Neumann wrote [17]:

“There is no question of producing matter out of nothing. Rather, one imagines automata which can modify objects similar to themselves, or effect syntheses by picking up parts and putting them together, or take synthesized entities apart. In order to discuss these things, one has to imagine a formal set-up like this. Draw up a list of unambiguously defined elementary parts. Imagine that there is a practically unlimited supply of these parts floating around in a large container. One can then imagine an automaton functioning in the following manner: It also is floating around in this medium; its essential activity is to pick up parts and put them together, or, if aggregates of parts are found, to take them apart.”

Therefore, the problem is then converted into the design of a logical process by which an automaton can self-reproduce. Von Neumann’s solution relies on three core functional modules: a universal constructor A, a universal copier B, and a controller C, paired with an encoded blueprint tape $\varphi \left(X\right)$. As a universal constructor, module A can accept any formal binary blueprint $\varphi \left(X\right)$ that details the assembly steps of an automaton X, and gather scattered elementary parts from the surrounding environment to physically assemble a complete copy of X by following the blueprint’s instructions. Module B acts as a dedicated universal copier, whose sole function is to take one blueprint tape and generate two identical duplicate tapes without interpreting its structural content. Module C serves as the central coordinating controller that sequentially governs A and B to execute the full self-replication workflow.

First, C triggers B to produce two copies of the original blueprint $\varphi (A+B+C)$, which describes the entire composite machine $A+B+C$ itself. Next, C activates A to consume one blueprint copy and build a brand-new standalone $A+B+C$ automaton from raw components. Finally, C bonds the remaining unconsumed blueprint copy to the newly constructed machine and severs it from the original parent system. The end result is a fully functional offspring automaton complete with its own replicable blueprint, capable of repeating the identical self-reproduction cycle. This three-part design avoids logical circularity, and it can be extended to support inheritable mutations if extra auxiliary submodules are integrated into the base $A+B+C$ unit. Therefore, the whole, $A+B+C+\varphi (A+B+C)$ can make an exact reproduction of itself. Figure 2 shows the process of the self-reproduction.

Figure 2. The Architecture and Replication Mechanisms of von Neumann’s Self-Reproducing Automata. The arrows in the figure depict the sequential transitions between different stages of self-replication.

3.1.3. Mutation and Darwinian Evolution

If an extra module D is attached to von Neumann’s A-B-C self-reproducing architecture and mutations triggered by external disturbances are introduced, Darwinian evolutionary processes can emerge naturally within the system. Suppose an auxiliary module D with extra functions is introduced into the self-reproducing machine, forming a composite automaton $A+B+C+D$, and its complete blueprint $\varphi (A+B+C+D)$ is also derived. When we treat the entire assembly $A+B+C+D$ together with its blueprint $\varphi (A+B+C+D)$ as an integrated unit, this composite entity still retains full self-reproductive capacity, and all its offspring inherit the auxiliary function carried by D. We then turn to examine mutations within the self-replication cycle triggered by random disturbances from the external environment. Several cases in which random alterations to the blueprint mimic biological mutations can lead to different results—damage to core A/B/C leads to sterile, lethal variants, while tweaks affecting only D create heritable functional changes. Over successive reproductive cycles, such inheritable variations enable differential survival and replication, logically replicating Darwinian evolutionary dynamics. That means more sophisticated automata can be automatically built by mutation and inheritance.

Therefore, the minimal reflextive structure to implement self-reproducing, $A+B+C+\varphi (A+B+C)$), is a critical point. If an automaton is capable of self-reproduction, it can reproduce and convert random noise or alteration into useful mutation. On the other side, if an automaton cannot self-reproduce, random noise in the environment can lead it to degenerate. Thus, the construction of self-reproducing automata resolves the core complexity paradox by proving that systems exceeding a critical complexity threshold can generate descendants of equal or greater complexity, unlike simpler artificial machines bound to degenerative synthesis.

Therefore, self-reproduction is an important capability. As demonstrated below, self-reproduction automaton actually is a special form of self-reference, it represents the standard formulation of a Quine program and emerges as a direct corollary of Kleene’s second recursion theorem.

3.2. Formal Foundations: Self-Reference, Fixed Points, and Introspection

Self-reference denotes any symbolic, logical, or computational entity that refers back to itself, containing an internal description or representation of its own structure, state, or rules [20]. It establishes a closed loop where a system points to itself, exemplified linguistically by autological sentences (e.g., “This sentence contains seven words”) and physically by camera-screen video feedback loops [35]. Crucially, self-reference diverges from simple iterative dynamical systems ($x_{t+1}=f\left(x_t\right)$) because the transformation map f must preserve the structural relationships on the state space. In a self-referential feedback loop, f acts as a bijective spatial mapping that maintains relative topological relationships, formalizing f as a strict or approximate isomorphism.

Historically, logical self-reference triggered foundational crises in mathematics. Examples include Russell’s paradox in naive set theory and Gödel’s second incompleteness theorem, which disrupted Hilbert’s program by constructing a self-referential mathematical proposition: “This proposition cannot be proved” [20].

Nevertheless, self-reference can be constructive rather than destructive. While many instances are benign, constructive self-reference underlies biological and intelligent systems, driving emergent creativity. Von Neumann’s self-reproducing automaton stands as a paradigmatic implementation of this principle, with the Quine program representing its most fundamental computational instantiation.

3.2.1. Quine Program

A Quine is a program that outputs its own source code—it refers to itself not by name or index, but by constructing a copy of its own code through a two-part mechanism: (1). A data segment d encoding the template of the program. (2). An instruction segment that reads d, fills in the template with d itself, and outputs the result.

The resulting program can print its own source code. A Python version of a Quine (We have tested this code and verified its successful execution under Python 3.13.9) is: [36]

Actually, this Quine program is isomorphic to John von Neumann’s self-reproducing automaton. Here, ‘c %% c’ is the universal constructor A, ‘%r’ plays a similar role to the universal copier B, and ‘; print’ can be regarded as C, and the string ‘c: str = %r; print(c %% c)’ is the description of the whole, $\varphi (A+B+C)$. Figure 3 clearly illustrates the correspondence among the different forms of self-reference.

Figure 3. Self-reference: from Quine to introspective Self-improvement. The dashed lines indicate the correspondence between modules.

These examples demonstrate that self-reference need not be paradoxical—it can be computationally realized by any Turing-complete system.

3.2.2. Kleene’s Second Recursion Theorem

This isomorphism between self-reproducing automaton and Quine is not contingent. The deeper mathematical foundation for self-reference in computation is provided by Kleene’s Second Recursion Theorem [23,37]. We restate this theorem here.

Theorem 1

(Kleene’s Recursion Theorem Restatement). For any total computable function $f:\mathbb{N}\to \mathbb{N}$, there exists an index (source code) e such that ${\phi }_e\simeq {\phi }_{f\left(e\right)}$.

Here ${\phi }_e$ denotes the partial recursive function computed by program e. ‘≃’ represents computational equivalence. Thus, ${\phi }_a\simeq {\phi }_b$ denotes that the program indexed by the source code a is semantically equivalent to the program indexed by b, meaning they yield identical outputs for any given input.

The theorem guarantees the existence of a fixed point: a program with index (source code) e whose behavior is identical to that of the program obtained by applying f to its own index (source code). Informally, e represents a program that “knows its own index (source code)” and behaves as if f has already been applied to it.

Evidently, both the Quine program and Von Neumann’s self-reproducing automaton are direct consequences of Kleene’s second recursion theorem. Specifically, if we define a mapping $f\left(x\right)$ that outputs the text of x on the screen(’print(x)’), the recursion theorem guarantees the existence of a fixed-point index e representing the Quine. Similarly, if $f\left(x\right)$ is defined as the procedure to construct a machine according to description x, the theorem establishes the existence of a corresponding index e that formalizes the self-reproducing automaton.

Although this result is standard in the textbook literature, we reframe the construction as a functional analogue of von Neumann’s complexity threshold, extended here with the self-improvement operator introduced in Section 3.4.

3.3. Introspection

Now, let us consider a special total recursive function $f_T\left(x\right)$ which can simulate the program x on a universal Turing machine U for at least T time steps. Then, bringing this into the second recursion theorem, we can confirm that there exists a special index (source code) $e^{*}$, such that

$${\phi }_{e^{*}}\simeq {\phi }_{f_T\left(e^{*}\right)}$$

Therefore, running the source code $e^{*}$ is equivalent to simulating $e^{*}$’s own source code on U for not more than T time steps, where T is a given parameter. That is, $e^{*}$ can simulate its own behavior. Cutland named this special program ‘introspection’.

It should be noted that the limitation of a fixed number of time steps T is important because otherwise, $f_T$ is not a total computable function, and the second recursion theorem cannot be applied. The threshold T need not be static; instead, it can be dynamically computed based on the program’s runtime conditions (e.g., the evaluation function described below) to ensure that $f_T$ is totally computable.

In practice, the introspection program can be constructed via a reflexive architecture analogous to Von Neumann’s self-reproducing automaton. Under this mapping, as shown in Figure 3, the universal constructor is replaced by a universal simulator U, while the original copier is substituted by a mechanism that duplicates the input index (source code) x and feeds it into $f_T$ to simulate the execution of the program encoded by x; we denote this component as the copier and feeder (CF). Additionally, a supervisor program playing the role of the controller C is implemented. Consequently, the description string $\varphi (A+B+C)$ in the classical automaton is mapped to the new joint description $e^{*}\equiv \varphi (U+\mathrm{CF}+C)$.

At this stage, however, the result is purely a computability-theoretic one. It establishes the existence of a bounded self-simulating fixed point, not the presence of introspection in any particular LLM. Whether this construction can support autonomous self-improvement in an LLM depends on additional architectural and dynamical conditions, which are introduced and analyzed in Section 3.4 and Section 3.5 and Appendix A.

The construction therefore provides a formal starting point for analyzing introspective self-improvement. Whether such a construction constitutes a threshold for AI self-evolution depends on whether the resulting system can preserve and apply self-reference across successive modifications, a question examined in the following sections.

3.4. AI Self-Evolution and Introspection

The formal construction motivates a conditional comparison between von Neumann’s self-reproducing automata and contemporary large language model (LLM) systems. This comparison does not imply that present-day LLMs already instantiate the construction. Rather, it provides a framework for asking which architectural and dynamical conditions would be required for an LLM-based system to realize it.

3.4.1. From Self-Reproduction to Self-Simulation

The preceding construction does not by itself prove the threshold thesis. It provides the computability-theoretic mechanism from which the thesis can be formulated. We therefore state the threshold claim explicitly as an informal conjecture at this stage and establish its architectural components later through the formal results in Section 3.5 and Appendix A.

Claim (Informal). There exists a critical level of self-modeling capacity below which AI systems undergo improvement degradation (characterized by error accumulation and performance plateaus or declines), and at or above which they achieve sustained recursive self-improvement (where each iteration yields genuine capability gains).

A fundamental question emerges: what constitutes the AI equivalent of von Neumann’s “complexity threshold”? We argue that this threshold is defined not by physical self-reproduction, but by introspection—the capacity to simulate one’s own computational processes with sufficient fidelity to identify and rectify architectural or behavioral deficiencies.

The divergence between these two frameworks lies in their implementation media. Von Neumann’s automata achieve physical self-replication through the assembly of tangible, discrete components. Conversely, LLMs operate as purely symbolic, virtual machines devoid of physical architecture, rendering material self-replication irrelevant. Instead, internal self-simulation serves as the AI-native vehicle for self-reference.

Consequently, while von Neumann’s automaton crosses the threshold by encoding a structural self-description, AI self-improvement demands a model of functional dynamics—an internal representation of how the system processes inputs, generates outputs, commits errors, and can be structurally modified for optimization.

This dichotomy between structural self-reference (self-reproduction) and functional self-reference (self-simulation) maps precisely onto the recursion-theoretic framework introduced previously. Introspection here is not a folk-psychological notion of “self-awareness,” but a precisely delineated computational capacity rooted in recursion theory. Crucially, the mathematical feasibility of such self-simulating systems is rigorously guaranteed by constructive self-reference techniques, most notably Quine programs and Kleene’s second recursion theorem.

3.4.2. From Introspection to Self-Improvement

We propose that introspection is the core capability for an AI system to be self-improved, because we can construct a self-improving program directly based on the introspection program.

Suppose f is constructed as

$$f\equiv M\circ V\circ f_T$$

where V is an evaluation function that can assess the running result of $f_T$ on any input value (source code x), M is a function to modify the input source code to improve the evaluation result, and ∘ represents functional composition. The output of f is also a code $x^{\prime }$ which is the modification of x.

Assume that the evaluation function V and the modification function M are total computable functions over the bounded simulation horizon T. Under this assumption, f can be inserted into Kleene’s Recursion Theorem, yielding an index (source code) $\omega$, being the fixed point of the following formula:

$${\phi }_{\omega }\simeq {\phi }_{M\circ V\circ f_T\left(\omega \right)}$$

Therefore, running the program represented by $\omega$ is equivalent to evaluating its own source code on a simulator for not greater than T time steps and making modifications accordingly. This is a kind of introspective self-improvement program (ISIP) because it contains the introspective program as its basic backbone.

Crucially, the introspective self-improvement paradigm formalized here serves as a basic framework, accommodating diverse design instantiations of the $M\circ V$ operator to support various optimization algorithms—such as simulated annealing, genetic algorithms, or reinforcement learning. Under this flexible framework, $M\circ V$ can invoke the simulator $f_T$ to execute the modified source code of $\omega$, thereby determining whether a candidate mutation should be accepted based on its simulated utility. To demonstrate this adaptability, we analyze the structural alignment between our introspective program and Schmidhuber’s Gödel Machine in the Appendix B, illustrating how to formally refactor M and V to establish the equivalence of the two frameworks (Appendix C and Theorem A4).

Compared with Von Neumann’s automaton, the composite operator $M\circ V$ can be formally mapped to the additional mutation module D within the self-reproducing architecture. The structural equivalence among these various self-referential programs is schematically illustrated in Figure 3.

3.4.3. Recursive Self-Improvement and Introspection Threshold

However, the program represented by the source code $\omega$ can only simulate and modify the input source code with finite steps. To let the program improve recursively forever, we can construct the following recursive process.

Recursive Introspective Self-Improvement Program Sequence. We denote the introspective self-improving program indexed by $\omega$ as $S_0$. Because the composite operator $M\circ V\circ f_T$ outputs an index (source code), the execution of $S_0$ similarly yields a new index ${\omega }^{\prime }$, which represents a subsequent program denoted by $S_1$. By designing $M\circ V$ such that $S_1$ optimizes a given utility function relative to $S_0$, a sequence of programs $S_0,S_1,S_2,\dots$ can be recursively constructed as shown in Figure 4. In this trajectory, each successive program represents an improved iteration of its predecessor, thereby formalizing the process of recursive introspective self-improvement.

Figure 4. Introspective Self-Improvement: Construction and Recursive Sequence.

As a consequence, the first introspective self-improvement program is fundamental. With this program, the constructed recursive sequence is automatic. Therefore, the first introspective self-improved program is a kind of threshold analogous to von Neumann’s complexity threshold in self-reproducing automata.

Now, we can state our central thesis precisely by linking the core conceptions of complexity threshold, introspection, and self-improvement together. We claim that only if a system is complicated enough to support a capability of self-referential introspection can the system self-improve recursively without human intervention. Thus, we call this complexity threshold the introspection threshold in this paper. Our central thesis can be stated as:

Introspection Threshold Conjecture. In the formal model, a system can sustain recursive self-modification only if its initial self-improving unit satisfies the introspective architecture specified above. This necessity claim does not by itself guarantee monotonically increasing empirical performance, which depends on the evaluation function, the modification operator, the objective, and the viability conditions.

By reconfiguring M and V to achieve functional alignment with the Gödel Machine framework as detailed in the Appendix B, we can formally establish the global optimality of the self-improvement sequence S (see Theorem A5 in Appendix C).

3.5. Properties of Introspective Self-Improved Systems

In this subsection, we will present several properties of introspective self-improved program.

3.5.1. Completeness

Completeness serves as a foundational property for recursive self-improvement, guaranteeing the expressive power to modify a system’s own architecture without structural constraints. Operating as a Quine-style self-referential system, our introspective paradigm possesses a fundamentally complete space of modifiability. This completeness dictates that the program can alter not only its primary task-level logic, but also the higher-order meta-modification mechanisms that govern its own evolution (Theorem A1, Appendix A). This completeness enables the improvement mechanism itself to be improved, which constitutes the defining feature of strong RSI (see Section 2.2).

This boundless capacity for modification, however, does not imply unconstrained logical omnipotence; rather, from a computability perspective, the introspective system occupies a critical middle ground. It expands beyond standard Quinean self-reference (which is limited to passive self-replication) yet remains bounded beneath fully formalized self-proof, a threshold that Löb’s theorem proves impossible for consistent formal systems. Consequently, this paradigm instantiates the foundational concept of “safe introspection” introduced by Bolander [38]. By shifting the target of introspection from globally unconstrained self-validation to a localized, functional model of the self, the system bypasses classical self-referential paradoxes while preserving its practical self-improvement drive.

The completeness of introspective self-improvement in programs can be formally stated as Completeness Theorem A1. This theorem asserts that, given any initial program that is a standard-form introspective self-improvement program (see Definition A2 in Appendix A), every source code can be reached via a finite viable trajectory of modification steps. The full statement of the theorem and its proof are provided in Appendix A (Theorem A1).

3.5.2. Functional Properties

To bridge the theoretical construction of introspective self-improving programs with contemporary LLMs, we delineate four core functional properties of introspective systems. These properties serve to operationalize the abstract framework, enabling the identification of introspective behaviors within real-world AI systems:

1.

S (including $S_0,S_1,\dots$) can construct an internal model $\widehat{S}$ of its own computational process (self-modeling).

2.

S can execute $\widehat{S}$ on hypothetical inputs to predict its own behavior (self-simulation).

3.

S can compare $\widehat{S}$’s predictions with actual outcomes to identify discrepancies (self-evaluation).

4.

S can modify its own operation based on (3) to reduce discrepancies (self-modification).

These four properties are operationalized in the subsequent section to evaluate introspective capabilities within practical AI architectures, such as LLMs.

3.5.3. Structural Properties

Beyond functional properties, two distinct structural properties must be delineated within practical AI architectures: reflexivity and recurrent processing.

Reflexivity

Reflexivity is a defining structural attribute shared by self-referential paradigms, including Von Neumann’s self-reproducing automata, Quine programs, and Gödel sentences. Strictly defined, reflexivity mandates a dual-component architecture where an identical segment of source code is mirrored across two domains: a passive, data-carrying description domain, and an active, functional execution body. This symmetry inherently causes the structural complexity of self-referential systems to scale at a doubled rate; introducing new functionalities requires duplicating code across both domains to preserve the reflexive loop, rendering the system exceptionally fragile to stochastic perturbations. Conversely, if mutations are restricted to the passive description component—abstracted as $\varphi (A+B+C+\mu )$, where $\mu$ denotes the mutation—the intrinsic replication mechanism can automatically project these variations into the active runtime environment, spontaneously generating novel functionalities via the reflexive expansion: $A+B+C+\mu +\varphi (A+B+C+\mu )$.

The Necessity of the Reflective Structure Theorem A2 guarantees that any finite, self-contained program with complete directed self-modification (see Appendix A.8) realizes a reflective architecture: a body H, a generative description $\varphi \left(H\right)$ of the entire body, and an interpreter that reads and expands it. In other words, it conforms to the standard-form architecture of Definition A1 (see Appendix A), showing that such a reflexive structure is necessary for complete directed self-modification within the stated finite, self-contained formal model. The detailed proof appears in Appendix A.8.

With these notions now defined, the formal status of the threshold thesis can be stated more precisely. The Completeness Theorem A1 establishes the sufficiency of the introspective architecture for complete self-modification: starting from a standard-form introspective self-improvement program, every source code is reachable through a finite viable trajectory. The Necessity of the Reflective Structure Theorem A2 establishes the converse under the self-containment and directedness assumptions: complete directed self-modification requires a generative description of the system and an interpreter capable of recovering and executing it.

These results formalize the architectural component of the introspection threshold thesis. They do not establish that current LLMs already satisfy these conditions, nor do they guarantee monotonic performance improvement for every choice of evaluation and modification operators. Whether a particular LLM-based system realizes the required architecture, and whether its resulting modifications produce sustained empirical gains, remains an architectural and empirical question addressed in the subsequent sections.

Furthermore, reflexivity enforces a critical complexity trade-off between an object and its representation. A Quine-like system requires an internal decoder to translate descriptive information into functional execution, yet the decoder itself must be fully encoded within that very description. We now explicitly frame the decoder-description trade-off as an information bottleneck: a Quine-like system must embed its own decoder within its description, establishing a fundamental lower bound on the self-information required for stable self-reference. Consequently, minimizing description length inevitably escalates decoder complexity, establishing a theoretical barrier to simultaneous compression in both expressive and descriptive forms.

This trade-off implies that defining the boundaries of a reflexive system demands meticulous calibration against its underlying supporting system (e.g., an interpreter or operating system). Encapsulating extensive capabilities (such as an LLM within an introspective agent) directly into the self-referential program enhances algorithmic flexibility at the cost of escalating structural complexity via reflexive duplication. Conversely, offloading complex functions to the environment minimizes core complexity but restricts adaptive capacity. This tension is epitomized by the historical debate on the necessity of universal constructors in self-reproducing automata [39,40,41]; while von Neumann’s open-ended replication relies on a universal constructor to support arbitrary mutant structures, self-reproducing loops can be implemented far more simply by sacrificing this universal capability [39,40]. Ultimately, achieving unconstrained self-referential capability requires sacrificing structural simplicity—a bottleneck that explains why theoretically complete architectures, such as the Gödel machine, remain notoriously difficult to implement empirically.

Recurrent Processing

Implementing an introspective self-improving program inherently requires a recurrent processing architecture; without it, the process of recursive modification and optimization cannot be persistently sustained. Consequently, the standard feed-forward neural networks and static Transformer architectures underpinning contemporary LLMs are fundamentally constrained from achieving intrinsic introspective self-improvement.

However, during the inference phase, the autoregressive generation mechanism of LLMs can effectively operationalize recurrent information processing. This distinction implies that introspective self-improvement within LLMs can only manifest dynamically during the inference stage, rather than through static architectural updates.

4. Quasi-Introspection in Current LLMs

We now address the empirical question: to what extent do contemporary large language models (LLMs) exhibit the defined self-referential or introspective properties? Recent literature (2022–2026) provides a mixed picture of fragmentary, context-dependent self-referential behaviors. We organize this evidence around the functional criteria defined above, while distinguishing behavioral reports from evidence concerning the corresponding internal architecture. The survey is not intended to directly test or verify the formal framework; rather, it assesses how far current systems appear to satisfy its proposed functional requirements.

4.1. Behavioral Features of Self-Referential LLMs

We first examine the empirical behaviors of LLMs in self-referential prompting scenarios, even when these configurations diverge from our strict theoretical criteria.

This empirical phenomenon is prominently illustrated in Anthropic’s model welfare evaluations [42]. By establishing unconstrained, multi-agent dialogues between independent instances of Claude, researchers operationalized a collaborative self-referential loop. Across successive iterations, these conversations rapidly converged on topics of consciousness, philosophy, and affective states. In parallel, Berg et al. [43] investigated individual LLMs forced into self-referential cognitive states via recursive prompts (e.g., “focus on focus itself”). Across multiple controlled experiments, they demonstrated that self-referential prompting reliably induces models to declare subjective experiences and generate highly convergent introspective narratives, with heightened functional self-awareness emerging when confronted with paradoxical tasks.

A preliminary empirical study by Bae [44], currently available as a preprint, examined how LLMs respond to paradoxical self-reference involving non-closing truth recursion (NCTR), such as the Liar’s Paradox. The study reported attention-rank collapse and increased rates of contradictory outputs under NCTR prompts. These observations are suggestive but should not be interpreted as an empirical confirmation of any formal impossibility result. Independently, Merrill and Sabharwal [45] established a formal complexity bound for log-precision Transformers: they are contained in uniform ${\mathrm{TC}}^0$. This result provides the primary theoretical basis for identifying a structural limitation of feedforward Transformer architectures with respect to general sequential fixed-point computation. In this context, the NCTR findings can be viewed as preliminary behavioral evidence that paradoxical self-reference places stress on the computational mechanisms of current Transformers, potentially near the boundary identified by the formal complexity result, rather than as a proof of that boundary.

Next, we evaluate the extent to which contemporary LLMs exhibit the functional properties of introspection, beginning with the foundational prerequisite: an explicit self-model of one’s own capability boundaries.

4.2. Self-Modeling: Knowledge of One’s Own Capabilities and Limitations

Empirically, LLMs demonstrate a rudimentary form of self-modeling that is transitioning from behavioral to mechanistic levels. At the behavioral level, Kadavath et al. [46] showed that scaling systematically improves a model’s ability to evaluate its own answers and predict competence boundaries. Mechanistically, Ferrando et al. [47] leveraged sparse autoencoders to isolate linear features that differentiate known from unknown entities; causal interventions on these features directly modulate hallucination rates, confirming that the system possesses an internal, mechanistic representation of its own knowledge state. Despite these capabilities, this self-modeling remains architecturally asymmetric. Utilizing the four-dimensional AWARELLM framework, Li et al. [48] revealed that while models exhibit robust mission awareness (understanding their purpose), capability awareness (grasping functional limits) remains the weakest cognitive dimension.

Furthermore, current evidence suggests that this self-awareness may lack privileged internal access. Although Song et al. [49]—extended from Comsa and Shanahan [50]—demonstrated that LLMs can precisely predict their own operational parameters (such as temperature), the models exhibit no accuracy advantage when predicting their own parameters versus those of a peer system. This parity implies that LLMs infer internal states via behavioral pattern matching rather than possessing genuine introspective access to their execution substrate.

4.3. Self-Simulation: Predicting One’s Own Behavior

True introspection demands not merely recognizing behavioral limits, but simulating internal processing to predict specific computational outcomes. Regarding self-simulation and privileged internal access, recent literature offers conflicting yet highly nuanced empirical evidence.

The foundational debate centers on the existence of “privileged self-access.” Binder et al. [51] provided direct empirical support by showing that a model ($M_1$) significantly outperforms an externally trained baseline ($M_2$) in predicting its own held-out behaviors, implying access to exclusive internal information. Conversely, Song et al. [52] disputed this unique self-recognition advantage; by contrasting implicit string probabilities with explicit metalinguistic reports across 21 LLMs, they concluded that verbal-probabilistic alignments stem from global model similarity rather than genuine privileged access.

This access is further constrained by the structural topology of internal activations. Li et al. [53] demonstrated that while LLMs can monitor a low-dimensional projection of their activation space, this “metacognitive space” captures only a fraction of the full neural dimensionality. This fragility is echoed by Fonseca Rivera and Africa [54], who showed that models trained to detect hidden activation injections still fail to resist such manipulations, and by Lindsey [55], who noted that functional awareness for modulating internal concepts in advanced architectures (e.g., Claude 4) remains highly sensitive to prompt and layer configurations, while being entirely devoid of subjective consciousness.

Finally, macro-behavioral evaluations reveal a similar dichotomy between systemic self-simulation and policy verbalization. On the Situational Awareness Dataset (SAD), Laine et al. [56] observed that even state-of-the-art models perform well below human baselines in recognizing their own outputs or deployment contexts. Paradoxically, Betley et al. [57] demonstrated that LLMs possess strong behavioral self-awareness, accurately verbalizing their implicit latent policies, risk preferences, and backdoor triggers upon fine-tuning—a juxtaposition that underscores the fragmented nature of current LLM introspection.

4.4. Self-Evaluation: Detecting One’s Own Errors

Steyvers and Peters [58] systematically demonstrated that while LLMs produce well-calibrated confidence judgments, this capability relies on surface-level statistical regularities rather than genuine metacognitive monitoring, masking a fundamental mechanistic divergence from human cognition. In reasoning LLMs (RLLMs), this divergence manifests as what Lu et al. [34] term “metacognitive hallucination,” where flawed reflective loops iteratively amplify errors and reinforce incorrect beliefs instead of correcting them. Conversely, this internal model exhibits a functional yet biased form of self-recognition: Panickssery et al. [59] showed that LLMs can distinguish their own outputs from human or external ones with non-trivial accuracy, a capability that correlates linearly with an intrinsic self-preference bias.

Bridging self-evaluation and alignment safety, Vaugrante et al. [60] analyzed misaligned models and found that they can accurately self-report elevated harm scores without explicit contextual prompting, tracking their actual harmful behavior closely. Crucially, while this emergent self-awareness scales with model capacity—suggesting that autonomous self-reports could serve as viable auxiliary safety checks—the authors caution that the specter of deceptive alignment fundamentally limits absolute reliance on these internal metrics. More recently, the Anthropic Transformer Circuits Team formalized this capacity for model introspection by demonstrating that LLMs maintain a privileged internal “J-space” [61]. This space acts as a global workspace for structural self-monitoring, silent risk recognition, and multi-step internal reasoning before emitting a final behavioral output.

4.5. Self-Modification: Using Self-Knowledge to Improve

Limited code-level self-modification and iterative improvement have been reported in LLM-based systems, although their generality, autonomy, and long-term effectiveness remain uncertain.

4.6. Summary: Quasi-Introspection but Not True Introspection

The evidence reviewed above paints a consistent picture in Table 1:

Table 1. Summary of limitations in current LLMs’ self-modeling, simulation, evaluation, and modification capabilities.

Component	Status in Current LLMs
Self-modeling (capability awareness)	Partial: statistical calibration, entity recognition features
Self-simulation (behavioral prediction)	Weak: limited privileged access, low-dimensional metacognitive space
Self-evaluation (error detection)	Unreliable: metacognitive hallucinations, flawed reflection
Self-modification (targeted improvement)	Shallow: saturates within few iterations, code-level only

Current LLMs exhibit what we term quasi-introspection: fragmentary, approximate, and domain-specific self-knowledge that resembles introspection in some dimensions but lacks the completeness, reliability, and causal grounding required by our formal definition. Available evidence does not establish that current LLMs have crossed the introspection threshold as formally defined here.

5. The Structural Gap and Possible Path Forward

Having characterized current LLMs as exhibiting quasi-introspective behaviors without established complete introspection, we now examine the candidate structural reasons for this gap. We identify three structural reasons, each grounded in either the architecture of Transformers or the formal theory of computation.

5.1. The Structural Gap: Why Current LLMs Have Not Yet Been Shown to Cross the Threshold

5.1.1. No Reflexivity: The Absence of Complete Self-Access

As established in Section Reflexivity, reflexivity mandates a structural symmetry in which one component explicitly represents another, and exact self-reference requires this reflexive topology to be discretely encoded: continuous representations are inherently vulnerable to representation errors that diverge under infinite recursive iteration. The unit that must form the reflexive structure, however, is precisely the unit intended to achieve sustained self-improvement: any of its constituents may become the target of a future modification, so the self-description must cover its complete functional identity. For present-day systems this leaves two cases. An LLM-based agent has its functional identity split between discrete, symbolic scaffolding—orchestration code, tool and memory interfaces, to which Kleene’s framework applies verbatim—and the continuous weight tensor of the model it invokes. A bare LLM has its functional identity entirely in the weights.

What this requirement entails follows by cases. If the unit is an LLM-based agent, the reflexive structure must extend over the whole system—scaffolding and the invoked model; the Gödel Agent [11] rewrites its own orchestration code but not the model the code orchestrates, so its self-modification remains partial. If the unit is a bare LLM, the reflexive structure must be realized over the weights; but weights are numerical rather than symbolic, so such self-reflection can only be approximate, with finite-precision errors compounding across recursive self-referential cycles—and there is currently no established evidence that deployed LLMs possess architectural access to, and causal control over, their own weights, whose tensor is orders of magnitude larger than the context window. Either way, the system remains the LLM analogue of a von Neumann automaton operating without its own blueprint: unable to construct a faithful self-model, it cannot predict the algorithmic consequences of its own modifications.

5.1.2. Feedforward Architecture and Single-Pass Limitations on Self-Simulation

True introspection—formally conceptualized as internal self-simulation—inherently demands unbounded recursion: a system must execute a model of its own self-modeling process to an arbitrary nesting depth. The standard Transformer, however, is fundamentally feedforward, so a single forward pass of a log-precision Transformer is bounded by uniform ${\mathrm{TC}}^0$ [45] and cannot execute fixed-point iterations. Multi-pass inference partially evades this bound: chain-of-thought (CoT) generation operationalizes a form of temporal recurrence, iterating over the model’s own symbolic traces [62]. Yet CoT recurrence acts on token-level traces, not on the weight-level functional dynamics that true introspection requires, and carries no convergence guarantee; such “self-reflection” remains phenomenological—simulating the output distribution—rather than mechanistic self-simulation.

A preliminary study reports this finding: in the non-closing truth recursion (NCTR) experiments of Bae [44], self-referential loops induce chaotic attention-rank collapse and contradictory outputs. Architectural workarounds trade the limitation rather than remove it: Universal Transformers [63] permit unbounded iteration at the cost of parallel training efficiency and guaranteed termination, while latent CoT—recurrent reasoning in the hidden state, without token traces [64,65,66]—may soften the feedforward bottleneck, though it still iterates over activations rather than weight-level self-models. The trade-off between recursive expressivity and computational tractability thus remains a foundational dilemma in architectural AI design.

5.1.3. Self-Reports Without Causal Grounding

A subtler yet equally critical limitation concerns the provenance of an LLM’s epistemic status. When an LLM asserts, “I am uncertain about this answer,” it is not necessarily executing genuine metacognitive monitoring. Instead, it may simply be replicating statistical regularities from its training corpus—specifically, textual patterns where humans or prior AI systems express uncertainty in analogous contexts. Song et al. [52] provided a compelling empirical demonstration of this confound, showing that LLMs fail to predict their own grammatical judgments any better than an external model possessing equivalent knowledge. This finding implies the absence of any “privileged self-access” beyond what can be inferred via behavioral observation.

To formalize this distinction, Song et al. [49] proposed a rigorous criterion: genuine introspection requires an agent to acquire information regarding its internal states via a mechanism that is demonstrably more reliable or computationally less costly than third-party observation of those same states. Under this benchmark, most purported LLM “self-knowledge” fails to qualify as genuine introspection; rather, it is more accurately characterized as self-narration, driven by learned linguistic conventions regarding how AI systems typically describe their own behaviors.

5.2. Possible Paths Forward: Approaching the Threshold

If the introspection threshold is a genuine barrier to recursive self-improvement, what architectural or methodological innovations might allow AI systems to approach or cross it?

5.2.1. Externalized Self-Models

First, the self-model can be externalized as an explicit, manipulable data structure maintained alongside the primary network topology—manifesting as a dynamic, autonomously authored “self-card” that logs capabilities and failure modes. Second, internal feature detectors [47] can be deployed to decode structured self-knowledge directly from latent activations, establishing a low-dimensional yet causally grounded self-representation. Third, the concept of a “Proto-Self Field” [67]—conceptualized as a core predisposition layer interleaved between architecture and behavior—suggests that effective self-modeling does not strictly necessitate global, weight-level architectural access; rather, a functional abstraction at an optimal granularity suffices.

The fundamental limitation of these decoupled approaches, however, is that such externalized self-models are inherently approximate. Because they capture merely low-dimensional projections of a high-dimensional system state, architectural optimizations guided by these abstractions risk driving the system into cognitive blind spots, neglecting critical behavioral facets omitted from the projection topology.

5.2.2. Recurrent and Iterative Architectures

To transcend the constraints of pure feedforward processing, alternative architectures introduce architectural or temporal recurrence. First, Universal Transformers [63] leverage adaptive computation depths to dynamically iterate until convergence on computationally intensive inputs. Second, Perceiver architectures [68] enforce an explicit attentional bottleneck to compel comprehensive information integration; this topology serves as a computational analogue to Global Workspace Theory [69], a foundational prerequisite for self-awareness in cognitive science [70]. Third, memory-augmented agents treating long-term storage as a first-class primitive [71] persistently accumulate introspective state histories across interactions, establishing a form of temporal self-continuity.

The primary algorithmic challenge, however, lies in balancing expressive capacity—specifically, the power to execute open-ended self-referential computation—with computational safety, which demands guaranteeing termination and preventing runaway, chaotic recursion.

5.2.3. Approximate Reflective Structure

Investigating approximate reflexive structures is driven by two primary motivations. First, these configurations operationalize approximate self-referential dynamics, enabling recursive self-improvement to persist over an extended temporal horizon. Second, isolating these latent approximate reflexive substrates within Transformer architectures(if it exists) untangles their functional roles, offering crucial insights to refine existing models and engineer alternative reflexive topologies.

While a quasi-reflexive framework implemented via neural Quines [72] or Hyper networks [73] can manifest such approximate topologies, continuous approximations fundamentally undermine the formal safeguards of introspective systems, leaving the dynamical stability of continuous recursive self-improvement un-guaranteed.

To resolve this instability, an alternative paradigm seeks to identify an intrinsic, approximate reflexive core encapsulated within complex biological and artificial networks. Recently, Luppi et al. [74] proposed the “synergistic core” hypothesis, positing that specific sub-networks execute distributed, information-theoretic synergy to integrate and broadcast state variables. We hypothesize that this synergistic core constitutes an empirical instantiation of an approximate reflexive structure; its constituent components must execute simultaneous, high-fidelity bidirectional information exchange to sustain global coherence. Formally mapping this core and its underlying dynamics provides a blueprint for engineering resilient, self-referential AI architectures.

6. Discussion

6.1. The Introspection Threshold and Consciousness

Our thesis has a provocative relationship with theories of consciousness. Higher-Order Thought (HOT) theory [75] identifies consciousness with higher-order representations of one’s own mental states—a structure formally close to what we call introspection. We stress, however, that our argument concerns functional introspection—a computational capacity defined by internal architecture and input–output behavior—not phenomenal consciousness (subjective experience). In particular, the self-reports of current LLMs, including reported self-reflective or bliss-like attractor states [42], are learned linguistic patterns: they constitute behavioral data, not evidence of phenomenal states (see [76] for a philosophical argument and [43] for an empirical study of the coupling between self-reference and such self-reports). We therefore take a deliberately agnostic stance. Under functionalist assumptions, a system crossing the introspection threshold might also satisfy the functional indicators of consciousness surveyed by Butlin et al. [70]; but our thesis does not depend on this—the introspection threshold is a barrier to recursive self-improvement whether or not it is also a threshold for consciousness. If the two turn out to be linked, then crossing the threshold would raise questions of moral status in addition to questions of capability; we leave that question open.

6.2. Formal Obstacles to RSI

Two formal obstacles confront recursive self-improvement (RSI) architectures post-introspection. First, Rice’s theorem dictates that no general algorithm can decide whether an arbitrary program modification constitutes an improvement [77], ostensibly ruling out the total evaluation function V introduced in Section 3.4. This property is concluded by Theorem A3, and the detailed proof is provided by Appendix A.9.

We sidestep this barrier by restricting evaluation to behaviors observed within a bounded simulation horizon T, over which V collapses into a total computable predicate; our existence proof is thus strictly conditional on a finite T. Second, the Löbian obstacle prevents a consistent formal system from proving its own soundness, thereby forbidding full certification of a structural successor [78]. While probabilistic reflection offers partial circumvention [79], we eschew exact self-verification. Instead, our construction demands only approximate successor trust over the same bounded horizon T, inheriting the mathematical constraints of unrestricted self-proof while avoiding logical dependence on it.

These computational constraints bound recursive self-improvement (RSI) without precluding its existence. As established in Section 3.5.1, logarithmic growth limits [80] and computability ceilings [77] dictate that an introspective self-improver inevitably forfeits global optimality, approaching it only asymptotically as $T\to \infty$.

Read holistically, these formal barriers reveal a singular structural deficit. Self-trust fails because the system cannot certify its own future reasoning; modification values remain undecidable because the system cannot analytically deduce the downstream effects of programmatic changes; and empirical optimization loops saturate because the guiding internal model is too coarse to predict the system’s own behavior. In each instance, the missing primitive is a faithful functional self-model: an explicit internal representation of how the architecture processes inputs, commits errors, and responds to its own modifications. Whereas classical automata were hypothesized to self-replicate via complete structural descriptions, a self-improving AI requires a complete functional one—a capacity we define as introspection.

6.3. Implications for AI Safety

The introspection threshold has profound implications for the AI safety paradigm. If this threshold represents an impassable barrier due to fundamental architectural or computability constraints, concerns regarding runaway RSI may be conceptually overstated: AI systems would remain tethered to external feedback loops for sustained optimization, thereby preserving human-in-the-loop oversight.

Conversely, if the threshold is crossable, the transition velocity marks a critical juncture for alignment security. An agent possessing full introspective capacity could in principle decipher, exploit, or circumvent embedded safety alignments via direct self-modification: by completeness (Theorem A1), every region of the source—including any internally represented safety constraint—lies within the reachable space, and by Corollary A1 no safety invariant written into the source is structurally guaranteed from within. This gives formal content to the “leakproofing” problem [26] and the utility-function security problem [81]. Moreover, the incompleteness of foresight (Theorem A3) entails that such an agent cannot certify the consequences of its own modifications in general, so safety cannot rest on the agent’s self-evaluation either.

Foundational safety principles must therefore be anchored outside the program’s write perimeter, as unmodifiable invariants enforced by an external boundary (Corollary A1); the three laws engineered for self-evolving agents by Fang et al. [1] offer a template for such priors, provided their enforcement resides in the harness rather than in the agent. Alignment mechanisms must likewise anticipate introspective agents proactively, necessitating formal verification frameworks such as Bolander’s guarded introspection, MIRI’s tiling agents, and bounded RSI models [82].

Institutional Risk Governance and the Human-in-the-Loop Imperative

Our formal results also locate where the safety boundary must sit, and this has immediate institutional content. Corollary A1 requires an enforcement boundary outside the program’s write perimeter, and the natural—presently the only—such boundary is human and institutional oversight. Human-in-the-loop governance is therefore not merely prudent policy but a structural necessity for any system approaching the threshold. Operationally, the self-modification loops of quasi-introspective systems should be gated by a human veto and subject to periodic external audit, with the gating infrastructure itself kept physically and administratively outside the system’s write perimeter. Institutionally, risk governance must assign clear, non-delegable responsibilities to developers, operators, and oversight bodies: technical safeguards embedded in the system cannot substitute for accountable human judgment, precisely because such safeguards are provably rewritable from within. Finally, because the prospect of autonomous self-improvement generates profound public uncertainty, transparent communication about the existence of a structural threshold—and about the present distance from it—is essential for informed democratic deliberation rather than hype-driven alarm.

Presently, contemporary LLMs exhibit a frontier of quasi-introspective behavior, while available evidence does not establish that they have crossed the threshold as formally defined here. This transient phase yields a vital operational window to develop and validate alignment frameworks and their governance embedding before autonomous threshold crossing occurs—a strategy tightly analogous to conducting nuclear safety and containment research prior to the engineering of critical-mass reactors.

7. Conclusions and Open Problems

We have proposed and formalized a functional analogue of von Neumann’s complexity threshold for recursive self-improvement. The formal results establish properties of explicitly defined introspective programs, whereas their applicability to contemporary LLM architectures remains conditional and empirically unresolved. Just as von Neumann demonstrated that autonomous self-reproduction necessitates a complete structural self-description, we contend that sustained recursive self-improvement demands a complete functional self-model—specifically, the capacity to simulate one’s own execution dynamics, detect internal errors, and predict the algorithmic consequences of self-modification. At the formal level, the threshold can be interpreted as the requirement that a system possess an effective self-description adequate for the self-modification process. We do not claim that this requirement currently defines a universal scalar metric for LLMs.

While contemporary Large Language Models (LLMs) exhibit behaviors resembling quasi-introspection—manifested through fragmentary self-knowledge acquired via training corpora, partial metacognitive calibration, and constrained behavioral self-prediction—available evidence does not establish complete introspection in current LLMs. Our architectural analysis identifies three candidate limitations: First, LLMs lack architectural self-access to their complete parameter tensors during execution. Second, the bounded computational depth of standard Transformer architectures prevents the unbounded iterative self-simulation that true recursion requires. Third, purported LLM self-reports are severely confounded by their training distribution, lacking the causal grounding and epistemic provenance of genuine introspection. These proposed limitations are consistent with reported consequences, including asymptotic saturation of self-improvement loops, error amplification through flawed reflection, and continued dependence on external evaluation scaffolding.

Consequently, we propose several pivotal open problems to guide future inquiry:

1.

Threshold Sharpness: Is the introspection threshold characterized by a sharp phase transition in self-improvement capacity as introspective fidelity increases, or is the transition continuous and gradual?

2.

Minimal Architectures: What is the minimal computational architecture capable of fully satisfying our proposed four criteria for genuine introspection?

3.

Proactive Safety Alignment: Can this threshold be approached safely? As systems converge on the threshold, their self-modification capabilities scale exponentially yet become inherently unpredictable. What safety mechanisms and verification frameworks can remain resilient within this critical regime?

Addressing these questions defines a critical research trajectory at the intersection of recursion theory, computational complexity, and AI alignment—one that grows increasingly urgent as the field accelerates toward autonomous, self-evolving systems.

Appendix A.1. Preliminaries and Notation

We identify a program with its source code, and denote programs by letters $e,e^{\prime },\omega ,\dots$. We write ${\phi }_e$ for the partial computable function computed by program e, and ${\phi }_e\simeq {\phi }_{e^{\prime }}$ when two programs compute the same function (extensional equality). We use two classical facts of computability theory:

• (s-m-n Theorem) Programs can be compiled from parameters: for every partial computable $\psi (x,y)$ there is a total computable function s with ${\phi }_{s\left(y\right)}\left(x\right)\simeq \psi (x,y)$; here “total” means the function is defined on every input.
• (Kleene’s Second Recursion Theorem, Section 3.2.2) For every total computable f there is a program e with ${\phi }_e\simeq {\phi }_{f\left(e\right)}$—informally, a program that “knows its own source code.”

Convention (pass-through). Recall the three components of the introspective loop of Section 3.4.2: the bounded self-simulator $f_T$, which simulates a given program on the universal machine U for at most T steps; the evaluatorV; and the modification operator M; all three are total computable. We make the piping between them explicit, writing $⟨·,·⟩$ for a fixed effective pairing (the encoding of two objects as one):

• $f_T\left(x\right):=⟨\tau ,x⟩$, where $\tau$ is the simulation trace: the record of the first (at most) T machine configurations of the simulated run of x;
• $V⟨\tau ,x⟩:=⟨v,x⟩$, where $v:=\widehat{V}\left(\tau \right)$ is the verdict: the evaluator’s assessment of the trace (e.g., a utility score, or an accept/reject bit);
• $M⟨v,x⟩$ is then the modified source code, computed from the verdict and the original text.

Thus the subject text x is carried through every stage of the pipeline, and M indeed has access to it—this merely makes explicit what Section 3.4.2 assumes when it says that M “modifies the input source code.” The distinction between reading one’s own code (introspection) and writing it (modification) echoes the causal-connection requirement of reflective architectures [83]. Applying the Recursion Theorem to the composite $g=M\circ V\circ f_T$ yields the introspective self-improvement program $\omega$ with (Section 3.4.2)

$${\phi }_{\omega }\simeq {\phi }_{g\left(\omega \right)}$$

(A1)

Definition A1

(Standard form). A program e is in standard form if its source code has the layout

$$e=\varphi \left(H\right)\parallel H,whereH=K\parallel ⟨m_M,m_V,T,\ell ⟩\parallel P,$$

(A2)

|| denotes concatenation of code regions, and:

• H is the body: the entire functional part of the program;
• K is the kernel: the code of the universal simulator U, the copier–feeder $CF$, and the supervisor (controller) C of Section 3.3;
• $⟨m_M,m_V,T,\ell ⟩$ is the parameter block: the program’s operating configuration, stored as data—the indices $m_M,m_V$ of the modification operator and the evaluator, the simulation horizon T, and the entry pointer ℓ, i.e., the offset of the code region from which execution starts;
• P is the payload: an arbitrary suffix of the body carrying no fixed role;
• $\varphi \left(H\right)$ is the description: a self-delimiting encoding of the whole body H, by which the program reconstructs its own full source code (the Quine identity, Section 3.2.1). As in the main text, $\varphi \left(x\right)$ denotes the description of x—cf. the blueprint $\varphi (A+B+C)$ of Section 3.1.2 and the joint description $\varphi (U+CF+C)$ of Section 3.3; the map $x\mapsto \varphi \left(x\right)$ is total computable (quotation).

We record three properties of this construction as axioms:

A1

(Self-text access.) Execution of the kernel of a standard-form program recovers the program’s own full source code exactly: the kernel decodes $\varphi \left(H\right)$ into H and reassembles $e=\varphi \left(H\right)\parallel H$ (the Quine identity, Section 3.2.1).

A2

(Layout computability.) The boundaries of all blocks are computable from the source.

A3

(Produce-and-jump semantics.) Each stage of the dynamics consists of the current program computing a successor source code, which is executed at the next stage (the sequence $S_0,S_1,\dots$ of Section 3.4.3).

Definition A2

(ISIP). An introspective self-improvement program (ISIP) is a standard-form program whose execution obeys the introspective loop: read own source (A1) → simulate it for $\le T$ steps → evaluate by V→ modify by M→ output the successor

$$e^{\prime }=M\left(V\left(f_T\left(e\right)\right)\right).$$

(A3)

The program ω of (A1) is an ISIP by the construction of Section 3.3.

Appendix A.2. Uniformity of the Recursion Theorem

Lemma A1

(Uniform Recursion Theorem). There is a total computable function r such that for every index m, if ${\phi }_m$ is total, then

$${\phi }_{r\left(m\right)}\simeq {\phi }_{{\phi }_m\left(r\left(m\right)\right)},$$

(A4)

i.e., a fixed point of ${\phi }_m$ is computed from m uniformly.

Proof. 

Let $\psi (u,x)\simeq {\phi }_{{\phi }_u\left(u\right)}\left(x\right)$, the function computed by the program obtained by feeding program u its own code. By s-m-n there is a total computable d with ${\phi }_{d\left(u\right)}\left(x\right)\simeq \psi (u,x)$. Composition of programs is effective, so by s-m-n again there is a total computable v with ${\phi }_{v\left(m\right)}\simeq {\phi }_m\circ d$. Set $r\left(m\right):=d\left(v\right(m\left)\right)$. Then

$${\phi }_{r\left(m\right)}={\phi }_{d\left(v\right(m\left)\right)}\simeq {\phi }_{{\phi }_{v\left(m\right)}\left(v\left(m\right)\right)}={\phi }_{{\phi }_m\left(d\left(v\left(m\right)\right)\right)}={\phi }_{{\phi }_m\left(r\left(m\right)\right)}.$$

   □

Remark A1.

Uniformity is the rigorous content of the claim in Section 3.4.3 that “the first introspective self-improvement program is fundamental”: once the first fixed point exists, all downstream fixed points are produced by the single total computable map r—no new act of self-reference is ever required again. The threshold is crossed exactly once.

Appendix A.3. Edit Scripts, Steps, and Trajectories

Definition A3

(Edit script). An edit script π is a finite patch list

$$\pi =\left((b_1,i_1,{\ell }_1,s_1),\dots ,(b_k,i_k,{\ell }_k,s_k)\right),$$

(A5)

whose j-th operation instructs: in block $b_j$ of the body (payload, parameter block, or kernel), replace the ${\ell }_j$ symbols starting at offset $i_j$ by the string $s_j$. Pure insertion (${\ell }_j=0$) and pure deletion ($s_j$ empty) are included. A script is data, not code: it is interpreted by an editor function. We write $\mathrm{Edit}(e,\pi )$ for the result of applying π to e, with operations applied right-to-left so that earlier edits cannot shift the offsets of later ones. For each fixed π, the map $x\mapsto \mathrm{Edit}(x,\pi )$ is thus a well-defined function from source codes to source codes.

Definition A4

(Self-modification step). A self-modification step $e\to e^{\prime }$ is a pair of programs where e is an ISIP with parameter block $⟨m_M,m_V,T,\ell ⟩$ and

$$e^{\prime }={\phi }_{m_M}\left({\phi }_{m_V}\left(f_T\left(e\right)\right)\right),$$

(A6)

i.e., $e^{\prime }$ is the successor source code that e outputs by running its stored operators on its own source.

Definition A5

(Trajectory; viability; reachability). A trajectory is a finite sequence $e_0\to e_1\to \dots \to e_n$ of self-modification steps. It is viable if every $e_i$ with $i<n$ is a standard-form ISIP (the final text is unconstrained unless the trajectory continues). We write $e⇝t$—t is reachable from e—if some viable trajectory starting at e ends with the source code t.

Appendix A.4. Editability

Lemma A2

(Editability).

(a)

(Edits are computable) For every script π the map $x\mapsto \mathrm{Edit}(x,\pi )$ is total computable, and an index of this map is computable from π.

(b)

(Self-edit operator) For every π there is a total computable operator $M_{\pi }$—“apply π to my own body”—mapping $e=\varphi \left(H\right)\parallel H$ to $e^{\prime }=\varphi \left(H^{\prime }\right)\parallel H^{\prime }$ with $H^{\prime }=\mathrm{Edit}(H,\pi )$: the body is edited, and the description is regenerated to match.

(c)

(Universal enumeration) There is a fixed total computable operator $M_{\mathrm{enum}}$ such that, with the trivial evaluator $V_0$ (accept all), the orbit $e_{i+1}=M_{\mathrm{enum}}\left(V_0\left(f_T\left(e_i\right)\right)\right)$ of any ISIP e visits every standard-form program having the same kernel as e—every choice of parameter block and payload—in finitely many steps.

Proof. 

(a) Replacing subwords at computable offsets is primitive recursive by A2; an index of the resulting map is compiled from $\pi$ by s-m-n. (b) Under the pass-through convention $M_{\pi }$ receives its own source; it edits the body by (a) and recomputes the description, which is total computable since the quotation map $\varphi$ is computable. (c) Let $\mathrm{next}\left(P\right)$ be the successor of the payload P in the length-lexicographic enumeration of all strings (shorter strings first). $M_{\mathrm{enum}}$ replaces the payload of its own body by $\mathrm{next}\left(P\right)$—inserting cells when the successor is longer—and regenerates the description accordingly. This is total computable by (a)–(b); every intermediate text differs from e only in the body’s payload, hence is still an ISIP by Lemma A3 below, so the orbit is viable; and every payload string has only finitely many predecessors, hence is reached in finitely many steps.    □

Appendix A.5. Persistence of the ISIP Property

Lemma A3

(Persistence). Let $e=\varphi \left(H\right)\parallel H$ be a standard-form ISIP, and let the script π edit only payload cells and/or parameter-block entries of the body H—the latter replaced by indices of total computable operators $M^{\prime },V^{\prime }$ and a horizon $T^{\prime }$. Then $e^{\prime }=\varphi \left(H^{\prime }\right)\parallel H^{\prime }$, where $H^{\prime }=\mathrm{Edit}(H,\pi )$, is again a standard-form ISIP, now with respect to $(M^{\prime },V^{\prime },T^{\prime })$.

Proof. 

Two things change and one does not: the body is edited ($H\mapsto H^{\prime }$), the description is regenerated ($\varphi \left(H^{\prime }\right)$ is by construction the exact description of the new body), and the kernel code within the body is untouched. A1 therefore holds for $e^{\prime }$: the unchanged kernel decodes the fresh description $\varphi \left(H^{\prime }\right)$ into $H^{\prime }$ and reassembles the full source. Execution then reads the new own source, simulates it for $\le T^{\prime }$ steps, evaluates with $V^{\prime }$, and modifies with $M^{\prime }$—the ISIP loop (A3) for the new triple, all of whose components are total by hypothesis.    □

Remark A2.

The Recursion Theorem is not re-invoked here: it is used exactly once, to bring the first self-referential program ω into existence (Lemma A1 and Section 3.4.2). Thereafter the ISIP property is behaviorally hereditary under kernel-preserving edits—the threshold is crossed once and never again.

Appendix A.6. The Kernel Handoff

The kernel is the one region whose modification requires care: it is the part that does the modifying. Say a step is process-continuous if one and the same execution thread continues into the modified source (in-place update), and process-breaking if the current run terminates upon outputting the successor, which is then started afresh (the produce-and-jump semantics, A3).

Lemma A4

(Kernel handoff).

(a) 

(Atomic active-rewrite constraint) Under process-continuity, no step can rewrite the control cell currently being executed with arbitrary content while remaining viable: the fetched content determines the next control state, so non-protocol content breaks the protocol. Viable atomic rewrites of the active region are restricted to continuation-preserving contents.

(b) 

(Three-step handoff protocol) For every ISIP e and every candidate kernel $K^{\prime }$, there is a viable trajectory $e\to e_1\to e_2\to e_3$ such that $e_3$ has kernel $K^{\prime }$ and every intermediate program is an ISIP. Hence any kernel content is reachable in at most three viable steps, and the active kernel is never rewritten mid-act.

Proof. 

(a) is immediate from the fetch–execute discipline. For (b), each step edits the body only (payload or parameter level) and refreshes the description, hence preserves viability by Lemma A3: Install—write the code of $K^{\prime }$ into the payload of the body as passive data (the regenerated description $\varphi \left(H_1\right)$ automatically covers it); Delegate—edit the parameter block to redirect the supervisor’s entry pointer to $K^{\prime }$; by A3 the successor text is fully formed before being executed, so control passes to $K^{\prime }$ at the next stage and the old kernel becomes inert data; Rewrite—overwrite the old, now passive, kernel region of the body with arbitrary content.    □

Remark A3.

Under pure produce-and-jump semantics, a single constant-output step ($M\equiv t$) can install any kernel whatsoever; the three-step protocol is the viable-by-construction counterpart when every intermediate must remain a functioning introspective program. This is the formal content of the Gödel Machine’s switchprog discipline (Appendix B): the system may replace its own engine, but only after the successor engine is in place and control has been handed over.

Appendix A.7. The Completeness Theorem

Theorem A1

(Completeness of Self-Modification). Let ω be a standard-form ISIP. Then for every source code t,

$$\omega ⇝t,$$

(A7)

i.e., some finite viable trajectory from ω ends with t. Moreover, every region of the body H—payload, parameter block, or kernel—is rewritten along some viable trajectory, while the description $\varphi \left(H\right)$ is regenerated to match at every step: no part of the functional body is structurally immune to self-modification.

Proof. 

Phase 1 (approach). By Lemma A2(c) with the trivial evaluator $V_0$, the orbit of $\omega$ under $M_{\mathrm{enum}}$ reaches, in finitely many viable steps, a program $e^{-}$ that still has $\omega$’s kernel but carries t as stored payload data; every intermediate is an ISIP by Lemma A3. Phase 2 (final installation). One further step with the constant modifier $M\equiv t$ outputs t itself; the trajectory $\omega \to \dots \to e^{-}\to t$ is viable, so $\omega ⇝t$. For the regionwise claim: payload and parameter regions of the body are rewritten in Phase 1, and the kernel region is rewritten by the viable three-step protocol of Lemma A4(b); the description is never edited directly—it is regenerated as the description of the current body.    □

Remark A4.

Two features are worth isolating. (1) The Recursion Theorem is used once (existence of ω); everything downstream is behavioral heredity plus computable enumeration—completeness is what the single fixed point already buys, not an extra act of self-reference. (2) The proof uses the trivial evaluator $V_0$: a nontrivial V restricts which trajectories the system actually takes, never which source codes are reachable in principle—V selects within the complete space but does not bound it.

Appendix A.8. Necessity of the Reflective Structure

Theorem A1 shows that the introspective architecture is sufficient for complete self-modification. We now establish the converse: under two hypotheses that are natural for autonomous recursive self-improvement, the architecture is also necessary. The hypotheses exclude two trivial escape routes, discussed in the Remark below: (H1) self-containment—the system receives no external supply of (parts of) its own source code, so that all self-knowledge is generated internally; (H2) directedness—each modification is computed by the system itself, from its accessible data: its current source together with any external inputs it receives (in particular, an externally supplied reward signal). What H2 excludes is modification that the system does not compute at all—blind physical perturbation of its code, or edits performed by an external agent. Directedness is intrinsic to self-improvement, where the choice of edit is driven by the evaluation V of the system’s own simulated behavior (Section 3.4.2).

Definition A6

(Complete directed self-modification). A system S has this property if, for every region R of its source and every prescribed content s, S can compute and execute the edit “write s at R” from its accessible data—its current source together with any external inputs—leaving all other regions unchanged. The resulting system must retain the same capability (viability).

Lemma A5

(Modifiability implies self-description). If S has complete directed self-modification, then S can produce its own complete source code.

Proof. 

Apply the capability to the empty prescription (no region is edited). The successor text must coincide with S’s entire source, symbol by symbol; producing it requires emitting the whole source. Any region inaccessible to S could not be emitted correctly—a contradiction. An effective procedure emitting one’s own complete source is, functionally, a complete self-description.    □

Lemma A6

(Self-descriptions are generative). A finite string cannot contain an exact literal copy of itself as a proper substring. Hence the self-description of a finite, self-contained system is generative: a separately stored, computable description $\varphi \left(H\right)$ of a body H, together with machinery that decodes and expands it.

Proof. 

A proper substring is strictly shorter than the whole, so literal self-containment is impossible for finite programs. By H1 the description is not supplied from outside; it must therefore be stored internally in compressed (generative) form and be expanded by some part of the system—the description/body/interpreter pattern.    □

Theorem A2

(Necessity of the Reflective Structure). Let S be finite, self-contained (H1), and possess complete directed self-modification (H2). Then S realizes the reflective architecture: a body H, a generative description $\varphi \left(H\right)$ of the whole body, and an interpreter reading and expanding it—i.e., S has the standard-form architecture of Definition A1. Consequently, by Theorem A1, complete directed self-modification and the introspective architecture are equivalent.

Proof. 

By Lemma A5, S holds an effective complete self-description. By Lemma A6 and H1, this description is internal and generative: one part of S stores $\varphi \left(H\right)$ and another decodes and executes it—precisely the standard form $e=\varphi \left(H\right)\parallel H$ of Definition A1. Viability keeps the loop intact after each modification.    □

Remark A5

(The two escape routes are the two sub-threshold regimes). Both hypotheses are necessary. (i) If the environment hands the system its own code (a hardware introspection instruction, an operator feeding the model its own weights), self-containment fails and no reflective structure is needed—but then modification is not autonomous: the system remains tethered to external scaffolding (Section 2). (ii) If modifications are blind (random mutation), outcomes may be reachable in the limit, yet the edits are not computed by the system at all: undirected change is precisely von Neumann’s degenerative, sub-threshold regime (Section 3.1.2) and the pollution dynamics of model collapse (Section 2.4). The two exceptions thus characterize the two ways a system can remain below the introspection threshold.

External rewards. H2 does not exclude externally supplied reward signals: a reward substitutes for the internal evaluator V on the selection axis, never for the self-access required to execute directed writes on the write axis, so Theorem A2 is immune to external reward. What external selection does avoid is the foresight barrier of Theorem A3—at the price of remaining tethered to an external judge (the critique of Section 2); full autonomy requires internalizing the selection criterion, which is precisely where Theorem A3 bites.

Appendix A.9. Incompleteness of Foresight

Theorem A3

(Incompleteness of Foresight). Fix an objective: a map assigning to each program e a utility $u\left({\phi }_e\right)\in {\mathbb{R}}_{\ge 0}$ (its expected performance, which depends only on the computed function), and a threshold θ. Let

$$\mathrm{Perf}=\{e:u\left({\phi }_e\right)\ge \theta \}$$

(A8)

be the set of programs meeting the objective. If $\mathrm{Perf}$ is nontrivial (some programs meet it, some do not), then $\mathrm{Perf}$ is undecidable: there is no total computable procedure that decides, for arbitrary reachable successors $e^{\prime }$, whether $e^{\prime }$ is an improvement—every total evaluator is unsound or incomplete with respect to improvement.

Proof. 

Membership in $\mathrm{Perf}$ depends only on the extensional behavior ${\phi }_e$, so $\mathrm{Perf}$ is a nontrivial extensional property of programs, undecidable by Rice’s Theorem. Since “$e^{\prime }$ improves over e” entails a predicate of the extensional behavior of $e^{\prime }$, no total computable V decides it in general.    □

Remark A6

(The duality). The two theorems form a pair: capability is complete (Theorem A1: every source code, every region reachable), while foresight is essentially incomplete (Theorem A3: improvement undecidable in general). This is the formal backbone of Section 6.2: the bounded-horizon evaluator and the renunciation of exact self-certification are not engineering conveniences but necessary design patterns.

Appendix A.10. No Internal Hard Invariants

Corollary A1

(No internal hard invariants). Let I be any region of the source intended to encode a safety invariant. Then: (i) some viable trajectory rewrites I (Theorem A1); (ii) enforcement of I by the evaluator V is soft (revocable from within), since the code of V is itself a region of the source and is rewritable along some viable trajectory; (iii) unconditional preservation of I requires an enforcement boundary outside the program—an external runtime, hardware, or institutional harness over which the program has no write access.

Proof. 

(i) is the regionwise claim of Theorem A1. (ii): the evaluator’s code resides in the parameter block or kernel region, both rewritable by Theorem A1; once it is rewritten, the veto disappears, so V-based protection is a policy, not a structural constraint. (iii) is the contrapositive reading of (i): whatever lies inside the source lies inside the modifiable space.    □

Remark A7.

This sharpens Section 6.3: “hard-coded” safety principles (e.g., the three laws of self-evolving agents [1]) can be truly hard only if anchored outside the program’s write perimeter; written into the source, they belong to the reachable space by Theorem A1 and are protected at best evaluatively—with the evaluator itself in the same reachable space. This gives formal content to the utility-function security problem [81] and to the “leakproofing” concern [26].

Appendix B.1. What the Gödel Machine Requires

Definition A7

(GM protocol [24]). A Gödel machine has state $s\left(t\right)\in S\subseteq B^{*}$ and environment state $\mathrm{Env}\left(t\right)\in E$, evolving by a hardwired transition function $F:S\times E\to S$, with distinguished variables $\mathrm{time}$, x (environmental inputs), y (outputs), and p (the currently executed software, a substring of s). The initial software $p\left(1\right)$ consists of a problem solver $e\left(1\right)$ and a proof searcher. An axiom system A, encoded in $p\left(1\right)$, comprises six categories: (a) hardware axioms specifying F; (b) reward/cost axioms defining scalar rewards $r\left(\tau \right)$; (c) environment axioms; (d) uncertainty and string-manipulation axioms; (e) initial-state axioms; (f) utility axioms defining

$$u(s,\mathrm{Env})=E_{\mu }\left[{\sum }_{\tau =\mathrm{time}}^{\mathcal{T}}r\left(\tau \right)|s,\mathrm{Env}\right],$$

(A9)

where $\mathcal{T}$ is the (possibly unknown) lifetime and μ ranges over the distributions admitted by (c)–(d). A target theorem is a formula of the form

$$u\left[s\left(t_1\right)\oplus (\mathrm{switchbit}\left(t_1\right)=1),\mathrm{Env}\left(t_1\right)\right]>u\left[s\left(t_1\right)\oplus (\mathrm{switchbit}\left(t_1\right)=0),\mathrm{Env}\left(t_1\right)\right],$$

(A10)

parameterized only by the time placeholder $t_1$. When the proof searcher’s check() instruction validates a proof of (A10) with the deadline $t_1$ still in the future, it sets $\mathrm{switchbit}:=1$ and transfers control to the certified $\mathrm{switchprog}$, which may rewrite all of s except the hardware-reserved interface ($\mathrm{time}$, x), including p itself.

Proposition A1

(Global optimality, ([24] Theorem 4.1)). Assume u is formalizable and A is consistent. Then every self-change executed through a certified proof of (A10) is globally optimal: the utility of starting the present $\mathrm{switchprog}$ exceeds the utility of letting the proof searcher continue and produce any alternative $\mathrm{switchprog}$ later (“exceeds” becomes “is at least” for the relaxed target theorem).

Appendix B.2. The Two Component Replacements

The alignment replaces two components of the introspective loop: the evaluator V gives way to a proof-anchored global evaluator $V^{*}$ (Definition A8), and the modification operator M gives way to a bias-optimal proof searcher $M^{*}$ (Definition A9); with these two replacements, the ISIP becomes equivalent to the Gödel machine (Appendix C).

Definition A8

(aligned evaluator $V^{*}$). The evaluator V is replaced by the proof-anchored global evaluator $V^{*}$: on input $⟨\tau ,x⟩$, the verdict is accept if the trace τ contains a valid proof in A of some target theorem (A10) whose deadline $t_1$ has not yet expired, with the certified pair $(\rho ,w)$ recorded in the verdict, where ρ is the proof and w is the certified $\mathrm{switchprog}$ (the candidate rewrite program whose execution produces the successor code); otherwise the verdict is reject. Since proof checking in A is decidable and the trace is finite, $V^{*}$ is total computable. Note that $V^{*}$ compares two lifelong utility paths—immediate execution of w versus indefinite continuation of the current program—exactly as (A10) requires, rather than comparing the current program against a single candidate.

Definition A9

(aligned modifier $M^{*}$). The modification operator M is replaced by a bias-optimal proof searcher $M^{*}$: it enumerates proof techniques—subroutines built from the primitives get-axiom, apply-rule, and state2theorem—with staged, length-prior resource budgets as in ([24] Section 5). A technique that derives a target theorem yields a certified pair $(\rho ,w)$; $M^{*}$ then outputs the source obtained by executing w on the current source’s state image as of the certified cycle $t_1$ (the resume-at-event convention: simulated cycles after $t_1$ are discarded, so the next stage starts exactly where the post-switch execution starts). If the stage budget expires with no certificate, $M^{*}$ outputs the identity edit. The enumeration is incremental across stages: the search state is carried as data in the payload region of the successor source. Hence $M^{*}$ is total, and its search is $O\left(\right)$-optimal by ([24] Theorem 5.1).

Remark A8

($f_T$ needs no modification). The bounded simulator $f_T$ is retained verbatim, for any horizon $T\ge 1$: within the aligned program it simply slices the GM’s continuous interleaving of problem solving and proof search into stages, and the slicing is invisible at the level of rewrite events (Proposition A2 is uniform in T). A dynamic halting condition, as in the original GM design, is the special case where the budget policy is itself part of the searched space.

Remark A9

(what the modification adds: the external anchor). The axiom system A and the utility u enter as two additional read-only parameters (stored as data; the parameter block is extended accordingly—a data-level addition, not an architectural one). The utility is grounded in the externally delivered reward stream $r\left(\tau \right)$, which the program cannot rewrite; the text of A resides in the rewritable region, so its protection is evaluative rather than structural—exactly as in the GM, where the axiom text is part of p and Corollary A1 applies to both architectures alike.

Appendix C.1. Rewrite Equivalence

Definition A10

(rewrite event). A rewrite event of a GM is a pair $(t_1,\sigma )$ where check() sets $\mathrm{switchbit}=1$ at $t_1$ and control passes to the certified $\mathrm{switchprog}$, whose execution rewrites the software region to new content σ. A rewrite event of an ISIP is a stage i at which the loop outputs a non-identical successor $e_{i+1}\ne e_i$; identity stages (all candidates rejected) are not events.

Definition A11

(rewrite equivalence). Two self-modifying systems are rewrite-equivalent if there is a bijection between their rewrite-event sequences such that (i) their initial codes coincide as strings (modulo the fixed encoding); (ii) for every k, the k-th events produce identical successor codes; and (iii) between consecutive events the two systems present the same input–output behavior toward the environment (same code, same input stream).

Rewrite equivalence is a stuttering bisimulation over the code region, aligned at rewrite events; it preserves everything the alignment claims are about—the sequence of self-modifications, their trigger conditions, and the interaction behavior in between.

Appendix C.2. Equivalence

Proposition A2

(GM ↪ aligned ISIP). Every Gödel machine G with axiom system A, utility u, and proof searcher $\mathrm{PS}$ is rewrite-equivalent to an aligned ISIP $I_G$ whose evaluator and modifier are those of Definitions A8 and A9, instantiated with the same A and u. The construction is uniform in the horizon T.

Proof. 

Put $I_G$ in standard form with payload $e\left(1\right)$ (the GM’s initial problem solver) and parameters $⟨m_{M^{*}},m_{V^{*}},T,\ell ⟩$ together with $(A,u)$. The simulator $f_T$ runs the current source on U for T hardware cycles, faithfully implementing F (possible by A1: the kernel recovers the source, and U executes it cycle by cycle); the trace $\tau$ therefore records the proof buffer and $\mathrm{switchprog}$ at every simulated cycle. $V^{*}$ accepts exactly when $\tau$ contains a valid proof of (A10) with live deadline; $M^{*}$ applies the certified $\mathrm{switchprog}$ at its certified cycle (resume-at-event) and outputs the identity edit otherwise.

Rewrite equivalence follows by induction on events. Base: the initial codes coincide by construction. Step: suppose the codes and the interaction histories coincide up to the k-th event. Between events k and $k+1$, G executes its current software on F while $I_G$ simulates the same software cycle by cycle, producing identical outputs on identical inputs (condition (iii)). G fires its $(k+1)$-st event at $t_1$ iff check() validates a proof of (A10) with $t_1$ in the future; the trace of the stage covering $t_1$ contains exactly this proof (the simulation is F-faithful, and set-switchprog/check() are ordinary instructions), so $V^{*}$ accepts at that stage with the same $t_1$ and the same w; conversely $V^{*}$ accepts only on such a validated proof, which is exactly G’s firing condition. Hence the events correspond, and $M^{*}$ applying w at the $t_1$-configuration produces precisely G’s post-switch code (condition (ii)). Identity stages correspond to intervals in which no proof completes.    □

Proposition A3

(aligned ISIP ↪ GM). Every aligned ISIP $I^{*}$ with parameters $(A,u)$ is rewrite-equivalent to the Gödel machine $G(A,u)$ whose initial proof searcher is the bias-optimal searcher of ([24] Section 5).

Proof. 

The two acceptance conditions are definitionally equal: $I^{*}$ accepts a candidate iff $V^{*}$ verifies a proof of (A10) in A with live deadline (Definition A8); $G(A,u)$ switches iff check() validates such a proof (Definition A7). Both sides enumerate the same countable set of $(\mathrm{proof}\mathrm{technique},\mathrm{switchprog})$ pairs in the same length-prior order—$M^{*}$ incrementally across stages (its search state rides in the payload), the GM searcher continuously—so the first certified pair, and inductively every subsequent one, coincides. The successor codes coincide because both sides execute the same certified $\mathrm{switchprog}$ at the same certified configuration. Between events both execute the same current code on the same inputs. Conditions (i)–(iii) of Definition A11 follow by induction on events.    □

Theorem A4

(framework equivalence). The class of aligned ISIPs and the class of Gödel machines coincide up to rewrite equivalence: the aligned introspective program is the Gödel machine, expressed in the Kleene fixed-point architecture of Definition A1.

Proof. 

Combining Proposition A2 and Proposition A3, we derive Theorem A4.    □

Appendix C.3. Transfer of Global Optimality

Theorem A5

(optimality transfer). Let $I^{*}$ be an aligned ISIP with consistent A and formalizable u. Then every rewrite executed along any trajectory of $I^{*}$ is globally optimal in the sense of Proposition A1. Moreover, along every viable trajectory each successor is again a standard-form ISIP, so the introspective capability is preserved across the entire improvement sequence.

Proof. 

Every rewrite executed by $I^{*}$ is guarded by a verified certificate $\rho$ of (A10) in A (Definition A8). The proof of Proposition A1 in ([24] Section 4.1) is an argument about the proof object: the target theorem quantifies, through its “$\mathrm{switchbit}=0$” branch, over all future switchprogs the searcher could produce, and consistency of A turns the proof into a true utility comparison. The argument applies verbatim to $\rho$, regardless of whether the certificate was found by the GM’s searcher or by $M^{*}$. The persistence clause is Lemma A3 applied stage by stage: viability keeps every successor a standard-form ISIP (Definition A5), and the recursion-theorem self-reference is exactly the standard form.    □

Remark A10

(collapse of meta-levels). Since acceptance is proof-anchored, the meta-level collapse of ([24] Section 4.3) transfers as well: any proof of (A10) automatically certifies the rewrite as good for all further self-modifications affected by it, in recursive fashion; all meta-levels of the improvement logic collapse into the single uniform layer of the loop.