A Philosophical Analysis of Causality and Correlation—The Debate on Causality between Hume and Bayes ^†

Abstract

In response to Hume’s challenge to causation, Kant and Bayes made two oriented responses, each addressing an aspect of Hume’s problem. Kant answered the question of cause and effect as an empirical possibility, which makes the certainty and necessity of causality ascribe to human reason itself. Bayes proposed the idea of replacing causality with correlation within the empirical possibility and gave a method for calculating the probability of correlation, finding a mathematical representation and reasoning for Hume’s notion of “probability”. The idea of correlation and the logic of probability are better adapted to deal with a world with uncertainty and are important in the deep learning of artificial intelligence.

1. Introduction

Cause and effect is the most important pair of relationships that guarantees human understanding and scientific research, and both philosophy and science begin with an inquiry into cause. In the 18th century, Hume’s questioning of the necessity of causal links undoubtedly removed the foundations of human knowledge from the bottom. Since Kant, philosophers have responded to this question by attempting to construct a system of knowledge with certainty through the necessity and universality of causality. At the same time, mathematicians have responded to the problem of cause and effect in their own way, with Bayes, a contemporary of Hume, being the most important representative, using probabilistic thinking to assign quantifiable and comparable values to the correlation of independent conditions. Since the 19th century, with the new understanding of the fundamental particles that make up the world through physical theories such as quantum mechanics, human cognition has begun to face the dilemma of uncertainty. The development of artificial intelligence has also taken two basic directions: one is based on logic, given a defined procedure to express information and develop deep reasoning from a small number of strongly correlated variables; the other consists of simple reasoning using massively parallel processing of information involving a large number of weakly correlated variables, an approach called deep learning. The underlying logic of the former is cognitive certainty and the necessity of causality, while the latter unfolds on the basis of uncertainty and correlation (which do not involve the issue of causal precedence). This year, the success of ChatGPT has demonstrated the great utility of the latter. The justification of deep learning as an approach and its potential for the future requires a philosophical discernment of causality versus relevance, and certainty versus uncertainty in the first place. This is significant for the way forward for artificial intelligence, and for the study of human cognitive abilities.

2. Hume’s Challenge against Causation

2.1. Do Empirical Associations of Cause and Effect Have Rational Roots?

Generally thought to have questioned the reliability of causal connections, what Hume really questioned was whether causation had a rational origin. Hume believes that cause and effect are an empirical connection, and he does not deny the reliability of experience, but as to what experience is and why it is so, no one can answer. “The question was not, whether the concept of cause is right, useful, and, with respect to all cognition of nature, indispensable, for this Hume had never put in doubt; it was rather whether it is thought through reason a priori, and in this way has an inner truth independent of all experience, and hence also a much more widely extended use that is not limited merely to objects of experience: regarding this Hume awaited enlightenment.” [1].

Hume sees causality not as a connection between things in themselves, but as one of the ways in which ideas are connected, and Hume suggests that there is more than one way of connecting ideas to ideas. “To me, there appear to be only three principles of connexion among ideas, namely, Resemblance,Contiguity in time or place, and Cause or Effect.” [2]. Not just causality, but all three have their source in experience and do not have a rational basis. “The conclusions we draw from that experience are not based on inference, nor on any process of understanding.” [2]. In these three kinds of relations, the connection between ideas cannot be deduced by reason. This is an important point of view that distinguishes Hume’s theory of cognition from its predecessors. For how ideas are related to each other, Hume puts it this way: “ In reality, all arguments from experience are founded on the similarity which we discover among natural objects, and by which we are induced to expect effects similar to those which we have found to follow from such objects.......From causes which appear similar we expect similar effects.This is the sum of all our experimental conclusions.” [2]. Causation is the last of the three connections, and Hume highlights this connection because “All reasonings concerning matter of fact seem to be founded on the relation of Cause and Effect.” [2]. By virtue of this relation, one can go beyond the evidence provided by the senses and memory to obtain an inference or prediction about what actually happened. But the kind of inference is not obtained through the faculty of reason, but simply by virtue of a habit formed by the many repetitions of experience. Hume argues that human reason is not capable of reasoning about facts, and that this is the result of the weakness and narrow scope of human reason. As to what experience is and how it is possible, Hume considered this to be an unanswerable question. His inquiry into the question of cause and effect stops at the concept of ‘experience’.

2.2. How Probability Is Expressed and Reasoned about in Experience?

On the premise that the connection of ideas is empirical and within the realm of experience, Hume suggests that we must give each outcome its own weight and credence according to how often it occurs, which is Hume’s solution to the cognitive dilemma of uncertainty. Here, Hume introduces the concept of ‘probability’, but this concept has not received as much attention as the question of causation.

3. Responses to the Humean Question

3.1. Kant’s Response

Kant’s critical philosophy was derived from a response to Hume’s question. Kant said that his Critique of Pure Reason is a statement of Hume’s problem on its largest possible scale [1]. Kant answered Hume’s legacy of crystallising the concepts of the faculty of reason and experience.

3.1.1. Requires a Critical Reason

The bulleted lists look like this: Kant attempts to conceive of Hume’s objection universally, that is, in terms of the entire capacity of knowing. It is thus found that “the concept of the connection of cause and effect is far from being the only concept through which the understanding thinks connections of things a priori; rather, metaphysics consists wholly of such concepts.” [1]. Kant finds a place for causality in reason: ‘cause and effect’ is one of the categories of ‘relation’. At the same time, Kant suggests that we need a discipline to define the capacities of reason, its possibilities, and its limits, i.e., to critique it. Within this boundary, the causal connection is universal and necessary. Through his critique of reason, Kant finds a basis for the causal connection with certainty.

3.1.2. The Concept of “Experience” and the Question of the Possibility of Experience

Kant’s specific approach to this problem is that the possibility of ‘experience’ is a precondition for the possibility of ‘objects of experience’. It is the innate connection that makes possible the experience of the causal connection, and it is under this condition that the concrete objects corresponding to cause and effect are presented to our perception. The supreme principle of innate synthesis is that every object obeys the necessary conditions for the synthesis and unification of the intuitive multitude of possible experiences. The causal connection as a category is one such condition. Objects can only be presented to us as experience in this form.

Through the innate categories, Kant takes up Hume’s question about the concept of experience and answers the question of what experience is that Hume thought could no longer be answered.

3.2. Bayesian Response

Hume referred to the concepts of uncertainty and probability, but did not further address the question of how uncertain information can be represented and reasoned. Bayes mathematised Hume’s notions to obtain a set of probabilistic reasoning methods. Bayesian probabilistic reasoning can deal with two main kinds of problems.

3.2.1. Inferring the Complete Nature of Things from Limited Information

Bayes invented Bayesian statistics in order to argue that “eyewitness testimony can never prove that a miracle occurred” by asking for the inverse probability of a hypothetical event occurring before the established facts. Prior to this, probability solved the problem of ‘cis-probability’, i.e., the overall situation was set and the random probability of the event was inferred. For example, if eight of the balls in the cup are known to be black and two are white, calculate the probability that the ball drawn will be white. “The problem solved by ‘inverse probability’ is to predict the proportion of balls in the cup based on the results of several draws, without knowing the proportion of balls in the cup. Of course, this prediction can never be 100% accurate. But it gives a way to predict the whole picture and the future with limited information.

3.2.2. Probability Measures Whether Factor A Is Correlated with Factor B under Uncertainty

As a priest, Bayes set out to argue against Hume’s questioning of causality and to demonstrate the necessary reliability of causal links, since the existence of God can only be justified if causal links can be established. Bayes ultimately concluded that we can establish a causal link between two factors, two events, by means of probability. Bayes introduced ‘conditional probability’, which completes a reversal of the cause-and-effect process, i.e., if B is observed, there is an x% probability that A is the cause. Conditional probability is the probability that an event A will occur if another event B has already occurred. Expressed as P(A/B), this is the probability of A conditional on B. For example, for Hume’s example of whether the cause of a stone getting hot is sunbathing, we can go about calculating the conditional probability of the event A, that the stone is hot, given that event B, sunbathing, has already occurred. Unlike deductive reasoning, probability does not have absolute validity, but x can be infinitely close to 100, provided that it is based on many experiences, and indeed countless experiences throughout human history. In this way, empirically, we can use this method to reach reliable conclusions. Bayes used probability to bridge cause and effect with practicality—we cannot determine cause and effect, but we can calculate the probability that two factors are correlated.

4. Reflections on Two Ways of Responding

Hume’s question arising from the causal connection in fact contains two sub-questions: the question of the possibility of experience, and the question of contingency in experience. Kant and Bayes in fact each answered one part of Hume’s question. Kant’s response was aimed at determining the necessity of causal connections and thus laying the foundations for natural science. Kant was confronted with the problem of laying the epistemological foundations for the physics of Newton’s time, the search for certainty, when science considered the laws of things, and the relations between things, to be certain.

Bayesian responses in terms of probabilities do not guarantee completely certain cause and effect, but this gives a precise method for obtaining valid knowledge when information is uncertain. Even if it is not possible to determine inevitable cause and effect deductively, the probability that things and events are correlated when they are undetermined can be calculated, and a belief value about this knowledge can then be obtained based on this correlation probability, on the basis of which a series of applications can be developed. With the development of physics, relativity and quantum mechanics have taught us that the world is uncertain, and with uncertainty following human cognition, how to make judgements and predictions under conditions of uncertainty has become an important epistemological issue. Bayesian probability theory is one logic that addresses this problem; it encompasses the logic about relevance, the logic about confidence, and the logic about relevance preferences on behaviour in the condition of uncertainty.

While causal connections correspond to the certainty of knowledge, correlations correspond to the uncertainty of the traits of things. Deterministic knowledge of either the properties of things or the relationships between things is impossible. This is mainly because it is impossible to obtain complete information. Therefore, the cognition that one can acquire is relative, partial, and uncertain. In this cognitive situation, Bayesian methods of calculating probabilities of relevance are particularly important for cognition.

5. Current Applications: Reasoning under Uncertainty—Deep Learning

Traditional machine learning involves inputting a deterministic program, on the basis of which the machine learns data; the most advanced machine learning model available is deep learning, which is a purely data-driven model. Deep learning is based on relevance theory rather than deterministic causal theory; AI does not need to perform fully deterministic causal reasoning at every step, it only needs to have the computational power to make probabilistic measures of the relevance of multiple factors. “Connection” and “probability” are the two key words in deep learning. The term “deep” refers to the multi-layered structure of neural networks, with the use of connections and probabilities throughout the structure. A neural network is made up of connections of neurons; ChatGPT-3 has over 170 billion connections, and GPT-4 has grown to a trillion, each with its own weight and threshold, and whether or not to send a signal to the next layer of the network depends on the combined threshold of the multiple connections in that layer. The judgements it makes in this process are not black and white, but assign different probabilities to different connections, which means it is able to arrive at valid cognitions despite the presence of uncertainty. Bayesian ideas of relevance and probabilistic reasoning are among the most important theoretical foundations. This suggests to us that relevance will be of great use in the future world of artificial intelligence.

In an age of uncertainty, where probability seems to explain the world better than determinism, correlation seems to have more realistic utility. It is necessary to revisit Bayes’ theory from an epistemological perspective.