Open Problems in Universal Induction & Intelligence

Abstract

Specialized intelligent systems can be found everywhere: finger print, handwriting, speech, and face recognition, spam filtering, chess and other game programs, robots, et al. This decade the first presumably complete mathematical theory of artificial intelligence based on universal induction-prediction-decision-action has been proposed. This information-theoretic approach solidifies the foundations of inductive inference and artificial intelligence. Getting the foundations right usually marks a significant progress and maturing of a field. The theory provides a gold standard and guidance for researchers working on intelligent algorithms. The roots of universal induction have been laid exactly half-a-century ago and the roots of universal intelligence exactly one decade ago. So it is timely to take stock of what has been achieved and what remains to be done. Since there are already good recent surveys, I describe the state-of-the-art only in passing and refer the reader to the literature. This article concentrates on the open problems in universal induction and its extension to universal intelligence.

“The mathematician is by now accustomed to intractable equations, and even to unsolved problems, in many parts of his discipline. However, it is still a matter of some fascination to realize that there are parts of mathematics where the very construction of a precise mathematical statement of a verbal problem is itself a problem of major difficulty.”

—Richard Bellman, Adaptive Control Processes (1961) p.194

1. Introduction

What is a good model of the weather changes? Are there useful models of the world economy? What is the true regularity behind the number sequence 1,4,9,16,...? What is the correct relationship between mass, force, and acceleration of a physical object? Is there a causal relation between interest rates and inflation? Are models of the stock market purely descriptive or do they have any predictive power?

Induction. The questions above look like a set of unrelated inquires. What they have in common is that they seem to be amenable to scientific investigation. They all ask about a model for or relation between observations. The purpose seems to be to explain or understand the data. Generalizing from data to general rules is called inductive inference, a core problem in philosophy [1,2,3] and a key task of science [4,5,6].

But why do or should we care about modeling the world? Because this is what science is about [7]? As indicated above, models should be good, useful, true, correct, causal, predictive, or descriptive [8]. Digging deeper, we see that models are mostly used for prediction in related but new situations, especially for predicting future events [9].

Predictions. Consider the apparently only slight variation of the questions above: What is the correct answer in an IQ test asking to continue the sequence 1,4,9,16,...? Given historic stock-charts, can one predict the quotes of tomorrow? Or questions like: Assuming the sun rose every day for 5000 years, how likely is doomsday (that the sun will not rise) tomorrow? What is my risk of dying from cancer next year?

These questions are instances of the important problem of time-series forecasting, also called sequence prediction [10,11]. While inductive inference is about finding models or hypotheses that explain the data (whatever explain actually shall mean), prediction is concerned about forecasting the future. Finding models is interesting and useful, since they usually help us to (partially) answer such predictive questions [12,13]. While the usefulness of predictions is clearer to the layman than the purpose of the scientific inquiry for models, one may again ask, why we do or should we care about making predictions?

Decisions. Consider the following questions: Shall I take my umbrella or wear sunglasses today? Shall I invest my assets in stocks or bonds? Shall I skip work today because it might be my last day on earth? Shall I irradiate or remove the tumor of my patient? These questions ask for decisions that have some (minor to drastic) consequences. We usually want to make “good” decisions, where the quality is measured in terms of some reward (money, life expectancy) or loss [14,15,16]. In order to compute this reward as a function of our decision, we need to predict the environment: whether there will be rain or sunshine today, whether the market will go up or down, whether doomsday is tomorrow, or which type of cancer the patient has. Often forecasts are uncertain [17], but this is still better than no prediction. Once we arrived at a (hopefully good) decision, what do we do next?

Actions. The obvious thing is to execute the decision, i.e. to perform some action consistent with the decision arrived at. The action may not influence the environment, like taking umbrella versus sunglasses does not influence the future weather (ignoring the butterfly effect) or small stock trades. These settings are called passive [18], and the action part is of marginal importance and usually not discussed. On the other hand, a patient might die from a wrong treatment, or a chess player loses a figure and possibly the whole game by making one mistake. These settings are called (re)active [19], and their analysis is immensely more involved than the passive case [20].

And now? There are many theories and algorithms and whole research fields and communities dealing with some aspects of induction, prediction, decision, or action. Some of them will be detailed below. Finding solutions for every particular (new) problem is possible and useful for many specific applications. Trouble is that this approach is cumbersome and prone to disagreement or contradiction [21]. Some researchers feel that this is the nature of their discipline and one can do little about it [22]. But in science (in particular math, physics, and computer science) previously separate approaches are constantly being unified towards more and more powerful theories and algorithms [23,24]. There is at least one field, where we must put everything (induction+prediction+decision+action) together in a completely formal (preferably elegant) way, namely Artificial Intelligence [25]. Such a general and formal theory of AI has been invented about a decade ago [26].

Contents. In Section 2. I will give a brief introduction into this universal theory of AI. It is based on an unexpected unification of algorithmic information theory and sequential decision theory. The corresponding AIXI agent is the first sound, complete, general, rational agent in any relevant but unknown environment with reinforcement feedback [27,28]. It is likely the best possible such agent in a sense to be explained below.

Section 3. describes the historic origin of the AIXI model. One root is Solomonoff’s theory [29] of universal induction, which is closely connected to algorithmic complexity. The other root is Bellman’s adaptive control theory [30] for optimal sequential decision making. Both theories are now half-a-century old. From an algorithmic information theory perspective, AIXI generalizes optimal passive universal induction to the case of active agents. From a decision-theoretic perspective, AIXI is a universal Bayes-optimal learning algorithm.

Section 4., Section 5., Section 6. and Section 7. constitute the core of this article describing the open problems around universal induction & intelligence. Most of them are taken from the book [27] and paper [31]. I focus on questions whose solution has a realistic chance of advancing the field. I avoid technical open problems whose global significance is questionable.

Solomonoff’s half-a-century-old theory of universal induction is already well developed. Naturally, most remaining open problems are either philosophically or technically deep.

Its generalization to Universal Artificial Intelligence seems to be quite intricate. While the AIXI model itself is very elegant, its analysis is much more cumbersome. Although AIXI has been shown to be optimal in some senses, a convincing notion of optimality is still lacking. Convergence results also exist, but are much weaker than in the passive case.

Its construction makes it plausible that AIXI is the optimal rational general learning agent, but unlike the induction case, victory cannot be claimed yet. It would be natural, hence, to compare AIXI to alternatives, if there were any. Since there are no competitors yet, one could try to create some. Finally, AIXI is only “essentially” unique, which gives rise to some more open questions.

Given that AI is about designing intelligent systems, a serious attempt should be made to formally define intelligence in the first place. Astonishingly there have been not too many attempts. There is one definition that is closely related to AIXI, but its properties have yet to be explored.

The final Section 8. briefly discusses the flavor, feasibility, difficulty, and interestingness of the raised questions, and takes a step back and briefly compares the information-theoretic approach to AI discussed in this article to others.

2. Universal Artificial Intelligence

Artificial Intelligence. The science of artificial intelligence (AI) may be defined as the construction of intelligent systems (artificial agents) and their analysis [25]. A natural definition of a system is anything that has an input and an output stream, or equivalently an agent that acts and observes. Intelligence is more complicated. It can have many faces like creativity, solving problems, pattern recognition, classification, learning, induction, deduction, building analogies, optimization, surviving in an environment, language processing, planning, and knowledge acquisition and processing. Informally, AI is concerned with developing agents that perform well in a large range of environments [32]. A formal definition incorporating every aspect of intelligence, however, seems difficult. In order to solve this problem we need to solve the induction, prediction, decision, and action problem, which seems like a daunting (some even claim impossible) task: Intelligent actions are based on informed decisions. Attaining good decisions requires predictions which are typically based on models of the environments. Models are constructed or learned from past observations via induction. Fortunately, based on the deep philosophical insights and powerful mathematical developments listed in Section 3., these problems have been overcome, at least in theory.

Universal Artificial Intelligence (UAI). Most, if not all, known facets of intelligence can be formulated as goal driven or, more precisely, as maximizing some reward or utility function. It is, therefore, sufficient to study goal-driven AI; e.g. the (biological) goal of animals and humans is to survive and spread. The goal of AI systems should be to be useful to humans. The problem is that, except for special cases, we know neither the utility function nor the environment in which the agent will operate in advance. What do we need (from a mathematical point of view) to construct a universal optimal learning agent interacting with an arbitrary unknown environment? The theory, coined AIXI, developed in this decade and explained in [27] says: All you need is Occam [33], Epicurus [34], Turing [35], Bayes [36], Solomonoff [37], Kolmogorov [38], and Bellman [30]: Sequential decision theory [20] (Bellman’s equation) formally solves the problem of rational agents in uncertain worlds if the true environmental probability distribution is known. If the environment is unknown, Bayesians [39] replace the true distribution by a weighted mixture of distributions from some (hypothesis) class. Using the large class of all (semi)measures that are (semi)computable on a Turing machine bears in mind Epicurus, who teaches not to discard any (consistent) hypothesis. In order not to ignore Occam, who would select the simplest hypothesis, Solomonoff defined a universal prior that assigns high/low prior weight to simple/complex environments, where Kolmogorov quantifies complexity [40,41]. All other concepts and phenomena attributed to intelligence are emergent. All together, this solves all conceptual problems [27], and “only” computational problems remain.

Kolmogorov complexity. Kolmogorov [38] defined the complexity of a string $x\in {\mathcal{X}}^{*}$ over some finite alphabet $\mathcal{X}$ as the length ℓ of a shortest description $p\in {\{0,1\}}^{*}$ on a universal Turing machine U:

$$\text{Kolmogorov complexity:}K\left(x\right):=\underset{p}{min}\{\ell \left(p\right):U\left(p\right)=x\}$$

(1)

A string is simple if it can be described by a short program, like “the string of one million ones”, and is complex if there is no such short description, like for a random string whose shortest description is specifying it bit-by-bit. For non-string objects o one defines $K\left(o\right):=K(\langle o\rangle )$, where $\langle o\rangle \in {\mathcal{X}}^{*}$ is some standard code for o. Kolmogorov complexity [38,42] is a key concept in (algorithmic) information theory [41]. An important property of K is that it is nearly independent of the choice of U, i.e. different choices of U change K “only” by an additive constant (see Section 4.h). Furthermore it leads to shorter codes than any other effective code. K shares many properties with Shannon’s entropy (information measure) S [43,44], but K is superior to S in many respects. Foremost, K measures the information of individual outcomes, while S can only measure expected information of random variables. To be brief, K is an excellent universal complexity measure, suitable for quantifying Occam’s razor. The major drawback of K as complexity measure is its incomputability. So in practical applications it has always to be approximated, e.g. by Lempel-Ziv compression [45,46], or by CTW [47] compression, or by using two-part codes like in MDL and MML, or by others.

Solomonoff induction. Solomonoff [37] defined (earlier) the closely related universal a priori probability $M\left(x\right)$ as the probability that the output of a universal (monotone) Turing machine U starts with x when provided with fair coin flips on the input tape [48]. Formally,

$$\text{Solomonoff prior:}M\left(x\right):=\sum _{p:U\left(p\right)=x*}{2}^{-\ell \left(p\right)},$$

(2)

where the sum is over all (possibly non-halting) so-called minimal programs p which output a string starting with x. Since the sum is dominated by short programs, we have $M\left(x\right)\approx 2^{-K\left(x\right)}$ (formally $-logM\left(x\right)=K\left(x\right)+O(log\ell (x\left)\right)$), i.e. simple/complex strings are assigned a high/low a-priori probability. A different representation is as follows [49]: Let $\mathcal{M}=\left\{\nu \right\}$ be a countable class of probability measures ν (environments) on infinite sequences ${\mathcal{X}}^{\infty }$, $\mu \in \mathcal{M}$ be the true sampling distribution, i.e. $\mu \left(x\right)$ is the true probability that an infinite sequences starts with x, and ${\xi }_{\mathcal{M}}\left(x\right):={\sum }_{\nu \in \mathcal{M}}{w}_{\nu }\nu \left(x\right)$ be the w-weighted average called Bayesian mixture distribution. One can show that $M\left(x\right)={\xi }_{{\mathcal{M}}_U}\left(x\right)$, where ${\mathcal{M}}_U$ includes all computable probability measures and $w_{\nu }=2^{-K\left(\nu \right)}$. More precisely, ${\mathcal{M}}_U:=\{{\nu }_1,{\nu }_2,...\}$ consists of an effective enumeration of all so-called lower semi-computable semi-measures ${\nu }_i$, and $K\left({\nu }_i\right):=K\left(i\right):=K\left(\langle i\rangle \right)$ [41].

M can be used as a universal sequence predictor, which outperforms in a strong sense all other predictors. Consider the classical online sequence prediction task: Given $x_{<t}\equiv x_{1:t-1}:=x_1...x_{t-1}$, predict $x_t$; then observe the true $x_t$; $t⇝t+1$; repeat. For $x_{1:\infty }$ generated by the unknown “true” distribution $\mu \in {\mathcal{M}}_U$, one can show [50] that the universal predictor $M\left(x_t\right|x_{<t}):=M\left(x_{1:t}\right)/M\left(x_{<t}\right)$ rapidly converges to the true probability $\mu \left(x_t\right|x_{<t})=\mu \left(x_{1:t}\right)/\mu \left(x_{<t}\right)$ of the next observation $x_t\in \mathcal{X}$ given history $x_{<t}$. That is, M serves as an excellent predictor of any sequence sampled from any computable probability distribution.

The AIXI model. It is possible to write down the AIXI model explicitly in one line [19], although one should not expect to be able to grasp the full meaning and power from this compact representation.

AIXI is an agent that interacts with an environment in cycles $k=1,2,...,m$. In cycle k, AIXI takes action $a_k$ (e.g. a limb movement) based on past perceptions $o_1{r}_1..o_{k-1}{r}_{k-1}$ as defined below. Thereafter, the environment provides a (regular) observation $o_k$ (e.g. a camera image) to AIXI and a real-valued reward $r_k$. The reward can be very scarce, e.g. just +1 (-1) for winning (losing) a chess game, and 0 at all other times. Then the next cycle $k+1$ starts. Given the above, AIXI is defined by:

$$\text{AIXI:}{a}_k:=arg\underset{a_k}{max}\sum _{o_k{r}_k}\dots \underset{a_m}{max}\sum _{o_m{r}_m}[r_k+\cdots +r_m]\sum _{q:U(q,a_1..a_m)=o_1{r}_1..o_m{r}_m}{2}^{-\ell \left(q\right)}$$

(3)

The expression shows that AIXI tries to maximize its total future reward $r_k+...+r_m$. If the environment is modeled by a deterministic program q, then the future perceptions $...o_k{r}_k..o_m{r}_m=U(q,a_1..a_m)$ can be computed, where U is a universal (monotone Turing) machine executing q given $a_1..a_m$. Since q is unknown, AIXI has to maximize its expected reward, i.e. average $r_k+...+r_m$ over all possible perceptions created by all possible environments q. The simpler an environment, the higher is its a-priori contribution $2^{-\ell \left(q\right)}$, where simplicity is measured by the length ℓ of program q. The inner sum ${\sum }_{q:...}{2}^{-\ell \left(q\right)}$ generalizes Solomonoff’s a-priori distribution M by including actions. Since noisy environments are just mixtures of deterministic environments, they are automatically included. The sums in the formula constitute the averaging process. Averaging and maximization have to be performed in chronological order, hence the interleaving of max and Σ (similarly to minimax for games). The value V of AIXI (or any other agent) is its expected reward sum.

One can fix any finite action and perception space, any reasonable U, and any large finite lifetime m. This completely and uniquely defines AIXI’s actions $a_k$, which are limit-computable via the expression above (all quantities are known).

That’s it! Ok, not really. It takes a whole book and more to explain why AIXI likely is the most intelligent general-purpose agent and incorporates all aspects of rational intelligence. In practice, AIXI needs to be approximated. AIXI can also be regarded as the gold standard which other practical general purpose AI programs should aim at (analogue to minimax approximations/heuristics).

The role of AIXI for AI. The AIXI model can be regarded as the first complete theory of AI. Most if not all AI problems can easily be formulated within this theory, which reduces the conceptual problems to pure computational questions. Solving the conceptual part of a problem often causes a quantum leap forward in a field. Two analogies may help: QED is a complete theory of all chemical processes. ZFC solved the conceptual problems of sets (e.g. Russell’s paradox).

From an algorithmic information theory (AIT) perspective, the AIXI model generalizes optimal passive universal induction to the case of active agents. From a decision-theoretic perspective, AIXI is a suggestion of a new (implicit) “learning” algorithm, which may overcome all (except computational) problems of previous reinforcement learning algorithms. If the optimality theorems of universal induction and decision theory generalize to the unified AIXI model, we would have, for the first time, a universal (parameterless) model of an optimal rational agent in any computable but unknown environment with reinforcement feedback.

Although deeply rooted in algorithm theory, AIT mainly neglects computation time and so does AIXI. It is important to note that this does not make the AI problem trivial. Playing chess optimally or solving NP-complete problems become trivial, but driving a car or surviving in nature do not. This is because it is a challenge itself to well-define the latter problems, not to mention presenting an algorithm. In other words: The AI problem has not yet been well defined (cf. the quote after the abstract). One may view AIXI as a suggestion of such a mathematical definition.

Although Kolmogorov complexity is incomputable in general, Solomonoff’s theory triggered an entire field of research on computable approximations. This led to numerous practical applications [51]. If the AIXI model should lead to a universal “active” decision maker with properties analogous to those of universal “passive” predictors, then we could expect a similar stimulation of research on resource-bounded, practically feasible variants. First attempt have been made to test the power and limitations of AIXI and downscaled versions like AIXI$tl$ and AIξ [52,53], as well as related models derived from basic concepts of algorithmic information theory.

So far, some remarkable and surprising results have already been obtained (see Section 3.). A 2, 12, 60, 300 page introduction to the AIXI model can be found in [19,27,54,55], respectively, and a gentle introduction to UAI in [56].

3. History and State-of-the-Art

The theory of UAI and AIXI build on the theories of universal induction, universal prediction, universal decision making, and universal agents. From a historical and research-field perspective, the AIXI model is based on two otherwise unconnected fundamental theories:

(1)

The major basis is Algorithmic information theory [41], initiated by [37,38,57], which builds the foundation of complexity and randomness of individual objects. It can be used to quantify Occam’s razor principle (use the simplest theory consistent with the data). This in turn allowed Solomonoff to come up with a universal theory of induction [37,50].

(2)

The other basis is the theory of optimal sequential decisions, initiated by Von Neumann [58] and Bellman [30]. This theory builds the basis of modern reinforcement learning [59].

This section outlines the history and state-of-the-art of the theories and research fields involved in the AIXI model.

Algorithmic information theory (AIT). In the 1960’s [37,38,57] introduced a new machine independent complexity measure for arbitrary computable data. The Kolmogorov complexity $K\left(x\right)$ is defined as the length of the shortest program on a universal Turing machine that computes x. It is closely related to Solomonoff’s universal a-priori probability $M\left(x\right)\approx 2^{-K\left(x\right)}$ (see above), Martin-Löf randomness of individual sequences [60], time-bounded complexity [61], universal optimal search [62], the speed prior [63], the halting probability Ω [64], strong mathematical undecidability [65], generalized probability and complexity [66], algorithmic statistics [67,68,69], and others.

Despite its incomputability, AIT found many applications in philosophy, practice, and science: The minimum message/description length (MML/MDL) principles [70,71,72] can be regarded as a practical approximation of Kolmogorov complexity. MML&MDL are widely used in machine learning applications [6,73,74,75,76,77]. The latest, most direct and impressive applications are via the universal similarity metric [46,78]. Schmidhuber produced another range of impressive applications to neural networks [79,80], in search problems [81], and even in the fine arts [82]. By carefully approximating Kolmogorov complexity, AIT sometimes lead to results unmatched by other approaches. Besides these practical applications, AIT is used to simplify proofs via the incompressibility method, improves Shannon information, is used in reversible computing, physical entropy and Maxwell daemon issues, artificial intelligence, and the asymptotically fastest algorithm for all well-defined problems [27,40,41,83,84].

Universal Solomonoff induction. How and in which sense induction is possible at all has been subject to long philosophical controversies [1,27,85]. Highlights are Epicurus’ principle of multiple explanations [34], Occam’s razor (simplicity) principle [33], and Bayes’ rule for conditional probabilities [5,36]. Solomonoff [37] elegantly unified these aspects with the concept of universal Turing machines [35] to one formal theory of inductive inference based on a universal probability distribution M, which is closely related to Kolmogorov complexity K ($M\left(x\right)\approx 2^{-K\left(x\right)}$). The theory allows for optimally predicting sequences without knowing their true generating distribution μ [50], and presumably solves the induction problem. The theory remained for more than 20 years at this stage, till the work on AIXI started, which resulted in a beautiful elaboration and extension of Solomonoff’s theory.

Meanwhile, the (non)existence of universal priors for several generalized computability concepts [66,86,87] has been classified, rapid convergence of M to the unknown true environmental distribution μ [88] and tight error [89] and loss bounds for arbitrary bounded loss functions and finite alphabet [90,91] have been proven, and (Pareto) optimality of M [18,86] has been shown, exemplified on games of chance and compared to predictions with expert advice [18,92]. The bounds have been further improved by introducting a version of Kolmogorov complexity that is monotone in the condition [93,94]. Similar but necessarily weaker non-asymptotic bounds for universal deterministic/one-part MDL [95,96] and discrete two-part MDL [97,98,99,100] have also been proven. Quite unexpectedly [101] M does not converge on all Martin-Löf random sequences [102], but there is a sophisticated remedy [103].

All together this shows that Solomonoff’s induction scheme represents a universal (formal, but incomputable) solution to all passive prediction problems. The most recent studies [104] suggest that this theory could solve the induction problem at whole, or at least constitute a significant progress in this fundamental problem [31].

Sequential decision theory. Sequential decision theory provides a framework for finding optimal reward-maximizing strategies in reactive environments (e.g. chess playing as opposed to weather forecasting), assuming the environmental probability distribution μ is known. The Bellman equations [30] are at the heart of sequential decision theory [25,58,105]. The book [20] summarizes open problems and progress in infinite horizon problems. Sequential decision theory can deal with actions and observations depending on arbitrary past events. This general setup has been called AIμ model in [19,27]. Optimality of AIμ is obvious by construction. This model reduces in special cases to a range of known models.

Reinforcement learning. If the true environmental probability distribution μ or the reward function are unknown, they need to be learned [59]. This dramatically complicates the problem due to the exploration↔exploitation dilemma [27,106,107,108]. In order to attack this intrinsically difficult problem, control theorists typically confine themselves to linear systems with quadratic loss function, relevant in the control of (simple) machines, but irrelevant for AI. There are notable exceptions to this confinement, e.g. the book [109] on stochastic adaptive control and [110,111], and an increasing number of more recent work. Reinforcement learning (RL) (sometimes associated with temporal difference learning or neural nets) is the instantiation of stochastic adaptive control theory [109] in the machine learning community. Current research on RL is vast; the most important conferences are ICML, COLT, ECML, ALT, and NIPS; the most important journals are JMLR and MLJ. Some highlights and surveys are [108,112,113,114,115,116,117,118,119,120,121,122,123,124,125,126,127,128,129,130] and [20,59,131,132,133] respectively. RL has been applied to a variety of real-world problems, occasionally with stunning success: Backgammon and Checkers [59, Chp.11], helicopter control [134], and others. Nevertheless, existing learning algorithms are very limited (typically to Markov domains), and non-optimal — from the very outset they are approximate or asymptotic only. Indeed, AIXI is currently the only general and rigorous mathematical formulation of the addressed problems.

The universal algorithmic agent AIXI. Reinforcement learning algorithms [59,131,135] are usually used in the case of unknown μ. They can succeed if the state space is either small or has effectively been made small by generalization techniques. The algorithms work only in restricted, (e.g. Markov) domains, have problems with optimally trading off exploration versus exploitation, have non-optimal learning rate, are prone to diverge, or are otherwise ad hoc.

The formal solution proposed in [27,55] is to generalize the universal probability M to include actions as conditions and replace μ by M in the AIμ model, resulting in the AIXI model, which is presumably universally optimal. It is quite non-trivial what can be expected from a universally optimal agent and to properly interpret or define universal, optimal, etc [19]. It is known that M converges to μ also in case of multi-step lookahead as occurs in the AIXI model [136], and that a variant of AIXI is asymptotically self-optimizing and Pareto optimal [137,138].

The book [27] gives a comprehensive introduction and discussion of previous achievements on or related to AIXI, including a critical review, more open problems, comparison to other approaches to AI, and philosophical issues.

Important environmental classes. In practice, one is often interested in specific classes of problems rather than the fully universal setting; for example we might be interested in evaluating the performance of an algorithm designed solely for function maximization. A taxonomy of abstract environmental classes from the mathematical perspective of interacting chronological systems [56,139] has been established. The relationships between Bandit problems, MDP problems, ergodic MDPs, higher order MDPs, sequence prediction problems, function optimization problems, strategic games, classification, and many others are formally defined and explored therein. The work also suggests new abstract environmental classes that could be useful from an analytic perspective. In [27], each problem class is formulated in its natural way for known μ, and then a formulation within the AIμ model is constructed and their equivalence is shown. Then, the consequences of replacing μ by M are considered, and in which sense the problems are formally solved by AIXI.

Computational aspects. The major drawback of AIXI is that it is incomputable, or more precisely, only asymptotically computable, which makes a direct implementation impossible. To overcome this problem, the AIXI model can be scaled down to a model coined AIXI$tl$, which is still superior to any other time t and length l bounded agent [27,55]. The computation time of AIXI$tl$ is of the order $t·2^l$. A way of overcoming the large multiplicative constant $2^l$ is possible at the expense of an (unfortunately even larger) additive constant. The constructed algorithm builds upon Levin search [62,140]. The algorithm is capable of solving all well-defined problems p as quickly as the fastest algorithm computing a solution to p, save for a factor of $1+\epsilon$ and lower-order additive terms [84]. The solution requires an implementation of first-order logic, the definition of a universal Turing machine within it and a proof theory system. The algorithm as it is, is only of theoretical interest, but there are more practical variations [81,141]. A different, more limited but more practical scaled-down version (coined AIξ) has been implemented and applied successfully to 2×2 matrix games like the notoriously difficult repeated prisoner problem and generalized variants thereof [52].

4. Open Problems in Universal Induction

The induction problem is a fundamental problem in philosophy [1,5] and science [142]. Solomonoff’s model is a promising universal solution of the induction problem. In [31], an attempt has been made to collect the most important fundamental philosophical and statistical problems, regarded as open, and to present arguments and proofs that Solomonoff’s theory overcomes them. Despite the force of the arguments, they are likely not yet sufficient to convince the (scientific) world that the induction problem is solved. The discussion needs to be rolled out much further, say, at least one generally accessible article per one allegedly open problem. Indeed, this endeavor might even discover some catch in Solomonoff’s theory. Some problems identified and outlined in [31] worth to investigate in more detail are:

a)

The zero prior problem. The problem is how to confirm universal hypotheses like $H:=$“all balls in some urn (or all ravens) are black”. A natural model is to assume that balls (or ravens) are drawn randomly from an infinite population with fraction θ of black balls (or ravens) and to assume some prior density over $\theta \in [0;1]$ (a uniform density gives the Bayes-Laplace model). Now we draw n objects and observe that they are all black. The problem is that the posterior proability $P\left[H\right|{black}_1...{black}_n]\equiv 0$, since the prior probability $P\left[H\right]=P[\theta =1]\equiv 0$. Maher’s [143] approach does not solve the problem [31].

b)

The black raven paradox by Carl Gustav Hempel goes as follows [144, Ch.11.4]: Observing Black Ravens confirms the hypothesis H that all ravens are black. In general, $\left(i\right)$ hypothesis $R\to B$ is confirmed by R-instances with property B. Formally substituting $R⇝\neg B$ and $B⇝\neg R$ leads to $\left(ii\right)$ hypothesis $\neg B\to \neg R$ is confirmed by $\neg B$-instances with property $\neg R$. But $\left(iii\right)$ since $R\to B$ and $\neg B\to \neg R$ are logically equivalent, $R\to B$ must also be confirmed by $\neg B$-instance with property $\neg R$. Hence by $\left(i\right)$, observing Black Ravens confirms Hypothesis H, so by $\left(iii\right)$, observing White Socks also confirms that all Ravens are Black, since White Socks are non-Ravens which are non-Black. But this conclusion is absurd. Again, neither Maher’s nor any other approach solves this problem.

c)

The Grue problem [145]. Consider the following two hypotheses: $H1:=$“All emeralds are green”, and $H2:=$“All emeralds found till year 2020 are green, thereafter all emeralds are blue”. Both hypotheses are equally well supported by empirical evidence. Occam’s razor seems to favor the more plausible hypothesis $H1$, but by using new predicates grue:=“green till y2020 and blue thereafter” and bleen:=“blue till y2020 and green thereafter”, $H2$ gets simpler than $H1$.

d)

Reparametrization invariance [146]. The question is how to extend the symmetry principle from finite hypothesis classes (all hypotheses are equally likely) to infinite hypothesis classes. For “compact” classes, Jeffrey’s prior [147] is a solution, but for non-compact spaces like $IN$ or $IR$, classical statistical principles lead to improper distributions, which are often not acceptable.

e)

Old-evidence/updating problem and ad-hoc hypotheses [148]. How shall a Bayesian treat the case when some evidence $E\widehat{=}x$ (e.g. Mercury’s perihelion advance) is known well-before the correct hypothesis/theory/model $H\widehat{=}\mu$ (Einstein’s general relativity theory) is found? How shall H be added to the Bayesian machinery a posteriori? What is the prior of H? Should it be the belief in H in a hypothetical counterfactual world in which E is not known? Can old evidence E confirm H? After all, H could simply be constructed/biased/fitted towards “explaining” E. Strictly speaking, a Bayesian needs to choose the hypothesis/model class before seeing the data, which seldom reflects scientific practice [5].

f)

Other issues/problems. Comparison to Carnap’s confirmation theory [149] and Laplace rule [150], allowing for continuous model classes, how to incorporate prior knowledge [151,152], and others.

Solomonoff’s theory has already been intensively studied in the predictive setting [18,50,89,91,94] mostly confirming its power, with the occasional unexpected exception [103]. Important open questions are:

g)

Prediction of selected bits. Consider a very simple and special case of problem 5.i, a binary sequence that coincides at even times with the preceding (odd) bit, but is otherwise incomputable. Every child will quickly realize that the even bits coincide with the preceding odd bit, and after a while perfectly predict the even bits, given the past bits. The incomputability of the sequence is no hindrance. It is unknown whether Solomonoff works or fails in this situation. I expect that a solution of this special case will lead to general useful insights and advance this theory (cf. problem 5.i).

h)

Identification of “natural” Turing machines. In order to pin down the additive/multiplicative constants that plague most results in AIT, it would be highly desirable to identify a class of “natural” UTMs/USMs which have a variety of favorable properties. A more moderate approach may be to consider classes ${\mathcal{C}}_i$ of universal Turing machines (UTM) or universal semimeasures (USM) satisfying certain properties ${\mathcal{P}}_i$ and showing that the intersection ${\cap }_i{\mathcal{C}}_i$ is not empty. Indeed, very occasionally results in AIT only hold for particular (subclasses of) UTMs [153]. A grander vision is to find the single “best” UTM or USM [154] (a remarkable approach).

i)

Martin-Löf convergence. Quite unexpectedly, a loophole in the proof of Martin-Löf (M.L.) convergence of M to μ in the literature has been found [101]. In [102] it has been shown that this loophole cannot be fixed, since M.L.-convergence actually can fail. The construction of non-universal (semi)measures D and W that M.L. converge to μ [103] partially rescued the situation. The major problem left open is the convergence rate for $W\stackrel{M.L.}{⟶}\mu$. The current bound for $D\stackrel{M.L.}{⟶}\mu$ is double exponentially worse than for $M\stackrel{w.p.1}{⟶}\mu$. It is also unknown whether convergence in ratio holds. Finally, there could still exist universal semimeasures M (dominating all enumerable semimeasures) for which M.L.-convergence holds. In case they exist, they probably have particularly interesting additional structure and properties.

j)

Generalized mixtures and convergence concepts. Another interesting and potentially fruitful approach to the above convergence problem is to consider other classes of semimeasures $\mathcal{M}$ [63,66,86], define mixtures ξ over $\mathcal{M}$, and (possibly) generalized randomness concepts by using this ξ to define a generalized notion of randomness. Using this approach, in [87] it has been shown that convergence holds for a subclass of Bernoulli distributions if the class is dense, but fails if the class is gappy, showing that a denseness characterization of $\mathcal{M}$ could be promising in general. See also [155,156].

k)

Lower convergence bounds and defect of M. One can show that $M\left({\overline{x}}_t\right|x_{<t})\ge 2^{-K\left(t\right)}$, i.e. the probability of making a wrong prediction ${\overline{x}}_t$ converges to zero slower than any computable summable function. This shows that, although M converges rapidly to μ in a cumulative sense, occasionally, namely for simply describable t, the prediction quality is poor. An easy way to show the lower bound is to exploit the semimeasure defect of M. Do similar lower bounds hold for a proper (Solomonoff) normalized measure $M_{norm}$? I conjecture the answer is yes, i.e. the lower bound is not a semimeasure artifact, but “real”.

l)

Using AIXI for prediction. Since AIXI is a unification of sequential decision theory with the idea of universal probability one may think that the AIXI model for a sequence prediction problem exactly reduces to Solomonoff’s universal sequence prediction scheme. Unfortunately this is not the case. For one reason, M is only a probability distribution on the inputs but not on the outputs. This is also one of the origins of the difficulty of proving general value bounds for AIXI. The questions is whether, nevertheless, AIXI predicts sequences as well as Solomonoff’s scheme. A first weak bound in a very restricted setting has been proven in [27, Sec.6.2], showing that progress in this question is possible.

The most important open, but unfortunately likely also the hardest, problem is the formal identification of natural universal (Turing) machines (h). A proper solution would eliminate one of the two most important critiques of the whole field of AIT. Item (l) is an important question for universal AI.

5. Open Problems regarding Optimality of AIXI

AIXI has been shown to be Pareto-optimal and a variant of AIXI to be self-optimizing [137]. These are important results supporting the claim that AIXI is universally optimal. More results can be found in [27]. Unlike the induction case, the results are not strong enough to alley all doubts. Indeed, the major problem is not to prove optimality but to come up with a sufficiently strong but still satisfiable optimality notion in the reinforcement learning case. The following items list four potential approaches towards a solution:

a)

What is meant by universal optimality? A “learner” (like AIXI) may converge to the optimal informed decision maker (like AIμ) in several senses. Possibly relevant concepts from statistics are, consistency, self-tuningness, self-optimizingness, efficiency, unbiasedness, asymptotically or finite convergence [109], Pareto-optimality, and some more defined in [27]. Some concepts are stronger than necessary, others are weaker than desirable but suitable to start with. It is necessary to investigate in more breadth which properties the AIXI model satisfies.

b)

Limited environmental classes. The problem of defining and proving general value bounds becomes more feasible by considering, in a first step, restricted concept classes. One could analyze AIXI for known classes (like Markov or factorizable environments) and especially for the new classes (forgetful, relevant, asymptotically learnable, farsighted, uniform, and (pseudo-)passive) defined in [27].

c)

Generaliztion of AIXI to general Bayes mixtures. Alternatively one can generalize AIXI to AIξ, where $\xi (·)={\sum }_{\nu \in \mathcal{M}}{w}_{\nu }\nu (·)$ is a general Bayes-mixture of distributions ν in some class $\mathcal{M}$ and prior $w_{\nu }$. If $\mathcal{M}$ is the multi-set of all enumerable semi-measures, then AIξ coincides with AIXI. If $\mathcal{M}$ is the (multi)set of passive semi-computable environments, then AIXI reduces to Solomonoff’s optimal predictor [18]. The key is not to prove absolute results for specific problem classes, but to prove relative results of the form “if there exists a policy with certain desirable properties, then AIξ also possesses these desirable properties”. If there are tasks which cannot be solved by any policy, AIξ should not be blamed for failing.

d)

Intelligence Aspects of AIXI. Intelligence can have many faces. As argued in [27], it is plausible that AIXI possesses all or at least most properties an intelligent rational agent should posses. Some of the following properties could and should be investigated mathematically: creativity, problem solving, pattern recognition, classification, learning, induction, deduction, building analogies, optimization, surviving in an environment, language processing, planning.

Sources of inspiration can be previously proven loss bounds for Solomonoff sequence prediction generalized to unbounded horizon, optimality results from the adaptive control literature, and the asymptotic self-optimizingness results for the related AIξ model. Value bounds for AIXI are expected to be, in a sense, weaker than the loss bounds for Solomonoff induction because the problem class covered by AIXI is much larger than the class of sequence prediction problems.

In the same sense as Gittins’ solution to the bandit problem and Laplace’ rule for Bernoulli sequences, AIXI may simply be regarded as (Bayes-)optimal by construction. Even when accepting this “easy way out”, the above questions remain significant: Theorems relating AIXI to AIμ would no longer be regarded as optimality proofs of AIXI, but just as how much harder it becomes to operate when μ is unknown, i.e. progress on the items above is simply reinterpreted.

A weaker goal than to prove optimality of AIXI is to ask for reasonable convergence properties:

f)

Posterior convergence for unbounded horizon. Convergence of M to μ holds somewhat surprisingly even for unbounded horizon, which is good news for AIXI. Unfortunately convergence can be slow, but I expect that convergence is “reasonably” fast for “slowly” growing horizon, which is important in AIXI. It would be useful to quantify and prove such a result.

g)

Reinforcement learning. Although there is no explicit learning algorithm built into the AIXI model, AIXI is a reinforcement learning system capable of receiving and exploiting rewards. The system learns by eliminating Turing machines q in the definition of M once they become inconsistent with the progressing history. This is similar to Gold-style learning [157]. For Markov environments (but not for partially observable environments) there are efficient general reinforcement learning algorithms, like $TD\left(\lambda \right)$ and Q learning. One could compare the performance (learning speed and quality) of AIξ to e.g. $TD\left(\lambda \right)$ and Q learning, extending [52].

h)

Posterization. Many properties of Kolmogorov complexity, Solomonoff’s prior, and reinforcement learning algorithms remain valid after “posterization”. With posterization I mean replacing the total value $V_{1m}$, the weights $w_{\nu }$, the complexity $K\left(\nu \right)$, the environment $\nu \left(o{r}_{1:m}\right|a_{1:m})$, etc. by their “posteriors” $V_{km}$, $w_{\nu }\left(ao{r}_{<k}\right)$, $K\left(\nu \right|ao{r}_{<k})$, $\nu \left(o{r}_{k:m}\right|o{r}_{<k}{a}_{1:m})$, etc, where k is the current cycle and m the lifespan of AIXI. Strangely enough for $w_{\nu }$ chosen as $2^{-K\left(\nu \right)}$ it is not true that $w_{\nu }\left(ao{r}_{<k}\right)\sim 2^{-K\left(\nu \right|ao{r}_{<k})}$. If this property were true, weak bounds as the one proven in [27, Sec.6.2] (which is too weak to be of practical importance) could be boosted to practical bounds of order 1. Hence, it is highly import to rescue the posterization property in some way. It may be valid when grouping together essentially equal distributions ν.

i)

Relevant and non-computable environments μ. Assume that the observations of AIXI contain irrelevant information, like noise. Irrelevance can formally be defined as being statistically independent of future observations and rewards, i.e. neither affecting rewards, nor containing information about future observations. It is easy to see that Solomonoff prediction does not decline under such noise if it is sampled from a computable distribution. This likely transfers to AIXI. More interesting is the case, where the irrelevant input is complex. If it is easily separable from the useful input it should not affect AIXI. One the other hand, even in prediction this problem is non-trivial, see problem 4.g. How robustly does AIXI deal with complex but irrelevant inputs? A model that explicitly deals with this situation has been developed in [129,130].

j)

Grain of truth problem [158]. Assume AIXI is used in a multi-agent setup [159] interacting with other agents. For simplicity I only discuss the case of a single other agent in a competitive setup, i.e. a two-person zero-sum game situation. We can entangle agents A and B by letting A observe B’s actions and vice versa. The rewards are provided externally by the rules of the game. The situation where A is AIXI and B is a perfect minimax player was analyzed in [27, Sec.6.3]. In multi-agent systems one is mostly interested in a symmetric setup, i.e. B is also an AIXI. Whereas both AIXIs may be able to learn the game and improve their strategies (to optimal minimax or more generally Nash equilibrium), this setup violates one of the basic assumptions. Since AIXI is incomputable, AIXI(B) does not constitute a computable environment for AIXI(A). More generally, starting with any class of environments $\mathcal{M}$, then $\mu \widehat{=}$AI${\xi }_{\mathcal{M}}$ seems not to belong to class $\mathcal{M}$ for most (all?) choices of $\mathcal{M}$. Various results can no longer be applied, since $\mu \notin \mathcal{M}$ when coupling two AIξs. Many questions arise: Are there interesting environmental classes for which AI${\xi }_{\mathcal{M}}\in \mathcal{M}$ or AI$\xi t{l}_{\mathcal{M}}\in \mathcal{M}$? Do AIXI($A/B$) converge to optimal minimax players? Do AIXIs perform well in general multi-agent setups?

From the optimality questions above, the first one (a) is the most important, least defined, and likely hardest one: In which sense can a rational agent in general and AIXI in particular be optimal? The multi-agent setting adds another layer of difficulty: The grain of truth problem (j) is in my opinion the most important fundamental problem in game theory and multi-agent systems. Its satisfactory solution should be worth a Nobel prize or Turing award.

6. Open Problems regarding Uniqueness of AIXI

As a unification of two optimal theories, it is plausible that AIXI is optimal in the “union” of their domains, which has been affirmed but not finally settled by the positive results derived so far. In the absence of a definite answer, one should be open to alternative models, but no convincing competitor exists to date. Most of the following items describe ideas which, if worked out, might result in alternative models:

a)

Action with expert advice. Expected performance bounds for predictions based on Solomonoff’s prior exist. Inspired by Solomonoff induction, a dual, currently very popular approach, is “prediction with expert advice” (PEA) [11,160,161]. Whereas PEA performs well in any environment, but only relative to a given set of experts, Solomonoff’s predictor competes with any other predictor, but only in expectation for environments with computable distribution. It seems philosophically less compromising to make assumptions on prediction strategies than on the environment, however weak. PEA has been generalized to active learning [11,162], but the full reinforcement learning case is still open [52]. If successful, it could result in a model dual to AIXI, but I expect the answer to be negative, which on the positive side would show the distinguishedness of AIXI. Other ad-hoc approaches like [126,163] are also unlikely to be competitive.

b)

Actions as random variables. There may be more than one way for the choice of the generalized M in the AIXI model. For instance, instead of defining M as in [27] one could treat the agent’s actions a also as universally distributed random variables and then conditionalize M on a.

c)

Structure of AIXI. The algebraic properties and the structure of AIXI has barely been investigated. It is known that the value of AIμ is a linear function in μ and the value of AIXI is a convex function in μ, but this is neither very deep nor very specific to AIXI. It should be possible to extract all essentials from AIXI which finally should lead to an axiomatic characterization of AIXI. The benefit is as in any axiomatic approach: It would clearly exhibit the assumptions, separate the essentials from technicalities, simplify understanding and, most importantly, guide in finding proofs.

d)

Parameter dependence. The AIXI model depends on a few parameters: the choice of observation and action spaces $\mathcal{O}$ and $\mathcal{A}$, the horizon m, and the universal machine U. So strictly speaking, AIXI is only (essentially) unique, if it is (essentially) independent of the parameters. I expect this to be true, but it has not been proven yet. The U-dependence has been discussed in problem 4.h. Countably infinite $\mathcal{O}$ and $\mathcal{A}$ would provide a rich enough interface for all problems, but even binary $\mathcal{O}$ and $\mathcal{A}$ are sufficient by sequentializing complex observations and actions. For special classes one could choose $m\to \infty$ [20]; unfortunately, the universal environment M does not belong to any of these special classes. See [27,32,164] for some preliminary considerations.

7. Open Problems in Defining Intelligence

A fundamental and long standing difficultly in the field of artificial intelligence is that (generic) intelligence itself is not well defined. It is an anomaly that nowadays most AI researchers avoid discussing intelligence, which is caused by several factors: It is a difficult old subject, it is politically charged, it is not necessary for narrow AI which focusses on specific applications, AI research is done mainly by computer scientists who mainly care about algorithms rather than philosophical foundations, and the popular belief that general intelligence is principally unamenable to a mathematical definition. These reasons explain but only partially justify the low effort in trying to define intelligence.

Assume we had a definition, ideally a formal, objective, non-anthropocentric, and direct method of measuring intelligence, or at least a very general intelligence-like performance measure that could serve as an adequate substitute. This would bring the higher goals of the field into tight focus and allow us to objectively compare different approaches and judge the overall progress. Indeed, formalizing and rigorously defining a previously vague concept usually constitutes a quantum leap forward in the field: Cf. set theory, logical reasoning, infinitesimal calculus, energy, temperature, etc. Of course there is (some) work on defining [165] and testing [166] intelligence (see [32] for a comprehensive list of references):

The famous Turing test [167,168,169] involves human interaction, so is unfortunately informal and anthropocentric, others are large “messy” collections of existing intelligence tests [170,171] (“shotgun” approaches), which are subjective and lack a clear theoretical grounding, and are potentially too narrow.

There are some more elegant solutions based on classical [172] and algorithmic [173] information theory (“C-Test” [174,175,176]), the latter closely related to Solomonoff’s [37] “perfect” inductive inference model. The simple program in [177] reached good IQ scores on some of the more mathematical tests.

One limitation of the C-Test however is that it only deals with compression and (passive) sequence prediction, while humans or machines face reactive environments where they are able to change the state of the environment through their actions. AIXI generalizes Solomonoff to reactive environments, which suggested an extremely general, objective, fundamental, and formal performance measure [56,178]. This so-called Intelligence Order Relation (IOR) [27] even attracted the popular scientific press [179,180], but the theory surrounding it has not yet been adequately explored. Here I only describe three non-technical open problems in defining intelligence.

a)

General and specific performance measures. Currently it is only partially understood how the IOR theoretically compares to the myriad of other tests of intelligence such as conventional IQ tests or even other performance tests proposed by AI other researchers. Another open question is whether the IOR might in some sense be too general. One may narrow the IOR to specific classes of problems [139] and compare how the resulting IOR measures compare to standard performance measures for each problem class. This could shed light on aspects of the IOR and possibly also establish connections between seemingly unrelated performance metrics for different classes of problems.

b)

Practical performance measures. A more practically orientated line of investigation would be to produce a resource bounded version of the IOR like the one in [27, Sec.7], or perhaps some of its special cases. This would allow one to define a practically implementable performance test, similar to the way in which the C-Test has been derived from incomputable definitions of compression using $Kt$ complexity [175]. As there are many subtle kinds of resource bounded complexity [41], the advantages and disadvantages of each in this context would need to be carefully examined. Another possibility is the recent Speed Prior [63] or variants of this approach.

c)

Experimental evaluation. Once a computable version of the IOR had been defined, one could write a computer program that implements it. One could then experimentally explore its characteristics in a range of different problem spaces. For example, it might be possible to find correlations with IQ test scores when applied to humans, like has been done with the C-Test [174]. Another possibility would be to consider more limited domains like classification problems or sequence prediction problems and to see whether the relative performance of algorithms according to the IOR agrees with standard performance measures and real world performance.

A comprehensive collection, discussion and comparison of verbal and formal intelligence tests, definitions, and measures can be found in [32].

8. Conclusions

The flavor of the open questions. While most of the key questions about universal sequence prediction have been solved, many key questions about universal AI remain open to date. The questions in Section 4., Section 5., Section 6. and Section 7. are centered around the AIT approach to induction and AI, but many require interdisciplinary working. A more detailed account with technical details can be found in the book [27] and paper [31]. Most questions are amenable to a rigorous mathematical treatment, including the more philosophically or vaguely sounding ones. Progress on the latter can achieved in the usual way by cycling through $\left(i\right)$ craft or improve mathematical definitions that resemble the intuitive concepts to be studied (e.g. “natural”, “generalization”, “optimal”), $\left(ii\right)$ formulate or adapt a mathematical conjecture resembling the informal question, $\left(iii\right)$ (dis)prove the conjecture. Some questions are about approximating, implementing, and testing various ideas and concepts. Technically, many questions are on (the interface between) and exploit techniques used in (algorithmic) information theory, machine learning, Bayesian statistics, (adaptive) control theory, and reinforcement learning.

Feasibility, difficulty, and interestingness of the open questions. I concentrated on questions whose answers probably help to develop the foundations of universal induction and UAI. Some problems are very hard, and their satisfactory solution worth a Nobel prize or Turing award, e.g. problem 5.j. I included those questions that looked promising and interesting at the time of writing this article. In the following I try to estimate their relative feasibility, difficulty, and interestingness:

• Problems roughly sorted from most important or interesting to least:
  5.j,4.h,7.b,5.a,7.c,4.b,4.f,4.l,5.d,5.f,5.i,4.a,4.c,4.d,4.e,4.g,5.c,6.a,6.b,5.g,5.h,4.j,5.b,6.c,7.a,4.i,4.k,6.d.
• Problems roughly sorted from most to least time consuming:
  4.b,4.h,5.d,5.j,7.c,4.c,5.b,5.c,6.c,6.a,5.g,5.i,7.b,4.i,4.j,4.l,5.a,6.d,6.b,5.f,5.h,4.d,4.f,4.g,4.k,7.a,4.a,4.e.
• Problems roughly sorted from hard to easy:
  4.h,6.c,5.j,4.b,4.i,4.l,5.b,6.a,7.b,4.c,4.g,5.a,5.c,6.b,5.f,5.i,4.j,5.h,7.a,4.d,4.f,4.k,5.d,6.d,7.c,4.a,4.e,5.g.

These rankings hopefully do not mislead but give the interested reader some guidance where (not) to start. The final paragraphs of this article are devoted to the role UAI plays in the grand goal of AI.

Other approaches to AI. There are many fields that try to understand the phenomenon of intelligence and whose insights help in creating intelligent systems: Cognitive psychology and behaviorism [181], philosophy of mind [182,183], neuroscience [184], linguistics [185,186], anthropology [187], machine learning [59,188], logic [189,190], computer science [25,191], biological evolution [192], and others. In computer science, most AI research is bottom-up; extending and improving existing or developing new algorithms and increasing their range of applicability; an interplay between experimentation on toy problems and theory, with occasional real-world applications. The agent perspective of AI [25] brings some order and unification in the large variety of problems the fields wants to address, but it is only a framework rather than a complete theory. In the absence of a perfect (stochastic) model of the environment, machine learning techniques are needed and employed. Apart from AIXI, there is no general theory for learning agents. This resulted in an ever increasing number of limited models and algorithms in the past.

The information-theoretic approach to AI. Solomonoff induction and AIXI are mathematical top-down approaches. The price for this generality is that the full models are computationally intractable, and investigations have to be mostly theoretical at this stage. From a different perspective, UAI strictly separates the conceptual and algorithmic AI questions. Two analogies may help: Von Neumann’s optimal minimax strategy [58] is a conceptual solution of zero-sum games, but is infeasible for most interesting zero-sum games. Nevertheless most algorithms are based on approximations of this ideal. In physics, the quest for a “theory of everything” (TOE) lead to extremely successful unified theories, despite their computational intractability [23,24]. The role of UAI in AI should be understood as analogous to the role of minimax in zero-sum games or of the TOE in physics.

Epilogue. As we have seen, algorithmic information theory offers answers to the following two key scientific questions: (1) The problem of induction, which is what science itself is mostly about: Induction ≈ finding regularities in data ≈ understanding the world ≈ science. (2) Understanding intelligence, the key property that distinguishes humans from animals and inanimate things.

This modern mathematical approach to both questions (1) and (2) is quite different to the more traditional philosophical, logic-based, engineering, psychological, or neurological approaches. Among the few other mathematical approaches, none captures rational intelligence as completely as the AIXI model does. Still, a lot of questions remain open. Raising and discussing them was the primary focus of this article.

Imagine a complete practical solution of the AI problem (by the next generation or so), i.e. systems that surpass human intelligence. This would transform society more than the industrial revolution two centuries ago, the computer last century, and the internet this century. Although individually, some questions I raised seem quite technical and narrow, they derive their significance from their role in a truly outstanding scientific endeavor. As with most innovations, the social benefit of course depends on its benevolent use.