Statistical Signatures of Quantum Contextuality

Abstract

Quantum contextuality describes situations where the statistics observed in different measurement contexts cannot be explained by a measurement of the independent reality of the system. The most simple case is observed in a three-dimensional Hilbert space, with five different measurement contexts related to each other by shared measurement outcomes. The quantum formalism defines the relations between these contexts in terms of well-defined relations between operators, and these relations can be used to reconstruct an unknown quantum state from a finite set of measurement results. Here, I introduce a reconstruction method based on the relations between the five measurement contexts that can violate the bounds of non-contextual statistics. A complete description of an arbitrary quantum state requires only five of the eight elements of a Kirkwood–Dirac quasiprobability, but only an overcomplete set of eleven elements provides an unbiased description of all five contexts. A set of five fundamental relations between the eleven elements reveals a deterministic structure that links the five contexts. As illustrated by a number of examples, these relations provide a consistent description of contextual realities for the measurement outcomes of all five contexts.

1. Introduction

The interpretation of quantum mechanics is controversial, because it is impossible to identify individual measurement outcomes with an underlying measurement-independent reality [1,2]. Unfortunately, the familiar terminology of quantum mechanics tends to encourage images of underlying realities by carelessly identifying quantum state components with measurement outcomes, even when no such measurements are ever performed. It cannot be stressed enough that quantum mechanics contradicts such fantasies directly by explaining the relation between different measurement contexts in terms of superpositions of their outcomes. Still, quantum mechanics does not seem to provide any good alternative. How can it be that we can describe the failure of objective realism in great detail, but we cannot find any good alternative that would explain the relation between different measurement contexts in a manner consistent with the highly successful theoretical formalism? Perhaps we need a more appropriate formulation of the problem so that we can identify the correct relations between different measurements without any speculation about unseen realities.

A very good illustration of the problem was recently given by Frauchiger and Renner, who pointed out that the paradoxical correlations between quantum systems observed in a scenario similar to Hardy’s version of a Bell non-locality prevent any consistent description of measurement processes within a quantum system [3,4]. The measurement problem can thus be identified with the problem of contextuality originally explored by Kochen and Specker [5]. To solve the measurement problem, we need to understand how quantum formalism relates different contexts to each other. Fortunately, we can build on a vast literature concerning this problem. Here, we focus on a three-dimensional Hilbert space, which is the most simple system in which Kochen–Specker contextuality can be demonstrated [6,7,8,9]. The demonstration of contextuality requires a set of five different measurement contexts related to each other through shared measurement outcomes. Quantum formalism then predicts the violation of an inequality that would apply to all non-contextual theories of measurement. It is thus easy to prove that quantum theory must be contextual. However, surprisingly little effort has been made to relate the assumptions of non-contextual theories directly to quantum formalism. Is it possible to formulate quantum theory in such a way that the difference between its predictions and non-contextual theories becomes more apparent?

Conventional formulations of quantum mechanics tend to represent states in a specific context, effectively hiding the relations with other contexts in the coherences between orthogonal states that represent the different outcomes of a single measurement context. In fact, the origin of contextuality can always be traced to state-independent relations between different contexts expressed in terms of fundamental relations between non-commuting operators [10,11,12,13]. Nevertheless, the experimentally observable evidence can only be obtained in the form of measurement statistics associated with specific quantum states [14]. In the end, contextuality describes statistical relations between different measurements that apply to arbitrary quantum states. To better understand the origin of contextuality in quantum formalism, it is therefore useful to investigate representations of quantum states that relate different measurement contexts to each other. This can be done by interpreting quantum formalism as a generalized probability theory [15,16,17,18]. Not surprisingly, the results show that quantum formalism introduces non-positive quasiprobabilities into the statistics, and this explains the differences between the predictions of quantum mechanics and the non-contextual hidden variable theories proposed as an alternative. The problem is that there is a wide range of possible quasiprobabilities based on different selections of measurements or different operator expansions [19]. To identify an appropriate quasiprobability, it is necessary to consider the specific relations between the different measurement contexts that characterize any given scenario. Here, I will derive the optimal quasiprobability for the representation of quantum contextuality by considering a possible reconstruction of quantum states from experimentally accessible data, also known as quantum state tomography. Originally, these methods emphasized the similarity between quantum theory and classical statistics, e.g., when a semiclassical analysis is used to reconstruct a phase space representation of the quantum state [20]. It has also been shown that weak measurements or a variety of joint measurements can be used to reconstruct a Kirkwood–Dirac distribution of two non-commuting observables [21,22,23,24,25,26]. These approaches to quantum tomography show that the reconstruction of an unknown state can be achieved by applying methods of classical statistics to characterize the non-classical correlations between the outcomes of incompatible measurements. Non-positive quasiprobabilities are obtained because these correlations cannot be explained by a positive-valued probability distribution over the possible combination of incompatible outcomes. As observed very early in the history of quantum mechanics, any attempt to describe quantum states as joint probabilities of complementary observables necessarily results in such non-positive distributions [27,28].

In the following, I introduce a method of tomography that relates five different contexts to each other, highlighting the statistical features responsible for the demonstration of quantum contextuality in a three-level system [29]. The contexts involved in this tomographical reconstruction can be visualized using a recently introduced three-path interferometer [30]. Non-contextual realities correspond to hypothetical paths of a classical particle through the interferometer. The tomographic method provides the operator algebra needed to identify the different paths with elements of the context-independent quantum statistics. Relations between different measurement contexts can then be identified with specific elements of the operator algebra. The results of the analysis show that the quantum state can be reconstructed from a set of only five Kirkwood–Dirac quasiprobabilities. Each of these Kirkwood–Dirac terms describes the statistical weight of one of the eleven quasirealities describing the path of a classical particle. Operator algebra can be used to derive five relations that define the Kirkwood–Dirac terms of the remaining paths. These five relations between the different contexts express the deterministic structure of Hilbert space formalism, preventing an arbitrary assignment of hidden realities even in cases where the individual measurement results would allow it. The framework established by the tomographic approach to quantum contextuality thus provides details of quantum contextuality that are hidden by the “black box” approach of no-go theorems.

Quantum theory makes precise statements about the relations between different contexts. These statements show that quantum mechanics is not constrained by the assumptions of a measurement-independent reality. Instead, the uncertainties of the initial state obtain a different meaning in each of the different contexts. This is not just an interpretation of the formalism: the universality of tomographic relations indicate that these are fundamental laws of physics. By designing experiments with sufficient care, it is possible to characterize the specific dependence of reality on the type of measurement that will eventually be performed. The relations presented in the following are a first step towards a fully deterministic description of the physical world that resolves the measurement problem by identifying the precise role that the measurement context plays in shaping the observable reality of all objects. It should be noted that this kind of progress is made possible by the realization that there is no need for a measurement-independent reality. The application of formalism strongly suggests that the familiar structure of reality that we experience in our actual lives is sufficiently explained in terms of the deterministic relations between different measurements. This is a point that is often overlooked in discussions of the measurement problem such as [1,2]. We do not have any direct access to any microscopic realities, and our ability to perform measurements is limited by our macroscopic interventions in the world around us. If we cannot reconcile our actual experience with our descriptions of “reality”, we need to question the description and not the experience.

2. Characterization of Measurement Contexts

Many fundamental investigations of quantum statistics start with a two-dimensional Hilbert space of a logical qubit. The physical analogy is a spin-1/2 system, where every direction in space is a potential measurement basis. It is difficult to formulate a logical relation between the different measurement contexts, since no two measurements can be performed jointly, and each measurement outcome belongs to only one possible measurement context. The situation changes when a three-dimensional Hilbert space is considered. It is now possible to construct a much wider range of possible measurements, and two different measurement contexts can share the same measurement outcome. The Kochen–Specker theorem uses such relations to investigate the statistical relations between incompatible measurements, demonstrating that the relation between different measurement contexts cannot be described by uncertainty-limited classical statistics [5]. In the most simple scenario, five different measurement contexts are related to each other by shared measurement outcomes in a three-dimensional Hilbert space [6,7,8,9]. Each measurement context is described by an orthogonal basis, where one of the basis states is unique to this context, and the other two are shared with two other contexts, resulting in a cyclic relation between all five contexts. As shown in Figure 1, the symmetry of this scenario can be illustrated by a pentagram, where each measurement context is represented by a triangle, and the shared outcomes form a pentagon at the center. It might be worth noting that this central pentagon is in principle sufficient to explain contextuality, as clarified in the work of Klyachko [8]. The representation used here is a more complete version of the graph derived in [29]. I am using the same terminology, with the context $\{1,2,3\}$ at the top, the contexts $\{1,S1,D1\}$ and $\{2,S2,D2\}$ sharing the outcomes 1 and 2 at their respective sides, and the contexts $\{S1,f,P1\}$ and $\{S2,f,P2\}$ at the bottom, which are joined by the shared outcome f. A statistical proof of contextuality can now be formulated using only the five shared outcomes. Since each of these five outcomes excludes two of the others, non-contextual logic requires that no more than two of the shared outcomes can be true at the same time. The sum of the probabilities for these five outcomes should therefore have an upper bound of two. However, this restriction does not apply in Hilbert space, resulting in possible violations of the non-contextual bound.

The conventional formulation of quantum theory forces us to chose a basis in which to represent the states and the operators. This can cause serious misunderstandings, since each basis only represents one of the five possible measurements. It is important to remember that the physics of a quantum system should be independent of the basis chosen to describe it. Although Figure 1 correctly identifies the shared measurement outcomes, it remains unclear what the relations between the other outcomes are. In fact, quantum theory defines the relation between two measurement contexts in terms of a unitary transformation, and in the case of the five contexts shown in Figure 1, the transformations between adjacent contexts involves a transformation in the two-dimensional subspace orthogonal to the shared context [29]. As I have shown recently, this relation can be illustrated by a three-path interferomer, where the measurement outcomes of the different contexts are represented by the paths along which a single quantum particle propagates through the interferometer [30]. The transformation between adjacent contexts is then implemented using beam splitters that control the overlap between the quantum states representing the paths. Figure 2 shows the interferometer. Each of the five beam splitters can be identified with the path parallel to it, corresponding to the shared measurement outcome of the contexts before and after the beam splitter. As was shown in [29,30], selecting any two beam splitter reflectivities will determine the other three. An option that is particularly easy to analyze is obtained when $R_1=R_2=1/2$, which results in the remaining reflectivities being given by $R_{S1}=R_{S2}=1/3$ and $R_f=1/4$.

What can be said about the relation between different measurement contexts? The identification of measurement contexts with the different stages of single-particle propagation in a three-path interferometer illustrates this relation by associating measurement outcomes with physical paths. A non-contextual hidden variable model would then assign a specific path through the interferometer to each particle. Since the input context $\{1,2,3\}$ is perfectly recovered at the output ports, paths originating from input 1 must end up in output 1 and so forth, leaving a total of eleven distinct possibilities for the propagation of the particle.

Table 1. State vectors associated with the eleven paths through the three-path interferometer, corresponding to the notion of a non-contextual reality of all five contexts. The states belonging to one path all have non-zero inner products. One of the paths has five segments, five of the paths have four segments, and five of the paths have three segments.

Path	State Vectors
$\left[0\right]$	$\left|3\right.〉$	$\left|D1\right.〉$	$\left|P1\right.〉$	$\left|P2\right.〉$	$\left|D2\right.〉$
$[1,P2]$	$\left|1\right.〉$		$\left|P1\right.〉$	$\left|P2\right.〉$	$\left|D2\right.〉$
$[S1,D2]$	$\left|3\right.〉$	$\left|S1\right.〉$		$\left|P2\right.〉$	$\left|D2\right.〉$
$[f,3]$	$\left|3\right.〉$	$\left|D1\right.〉$	$\left|f\right.〉$		$\left|D2\right.〉$
$[S2,D1]$	$\left|3\right.〉$	$\left|D1\right.〉$	$\left|P1\right.〉$	$\left|S2\right.〉$
$[2,P1]$	$\left|2\right.〉$	$\left|D1\right.〉$	$\left|P1\right.〉$	$\left|P2\right.〉$	$\left|2\right.〉$
$[1,f]$	$\left|1\right.〉$		$\left|f\right.〉$		$\left|D2\right.〉$
$[S1,S2]$	$\left|3\right.〉$	$\left|S1\right.〉$		$\left|S2\right.〉$
$[2,f]$	$\left|2\right.〉$	$\left|D1\right.〉$	$\left|f\right.〉$		$\left|2\right.〉$
$[1,S2]$	$\left|1\right.〉$		$\left|P1\right.〉$	$\left|S2\right.〉$
$[2,S1]$	$\left|2\right.〉$	$\left|S1\right.〉$		$\left|P2\right.〉$	$\left|2\right.〉$

illustrates the eleven paths by assigning a sequence of Hilbert space vectors to each of them. As the table shows, two path segments $[a,b]$ are sufficient to define each of the paths, except for the path composed of the five segments that are unique to each context, which is therefore labeled $\left[0\right]$. The quantum mechanical relations that replace the sequence of paths in the Hilbert space formalism are inner products of the state vectors that represent the measurement outcomes of the different contexts [29]. Although quantum contextuality shows that quantum states cannot be represented by positive-valued probability distributions over the eleven paths in Table 1, it should be possible to find a quasiprobability representation of the density matrix that relates directly to these 11 paths. It has already been noted elsewhere that Kirkwood–Dirac quasiprobabilities are ideally suited for this purpose [30,31,32]. In the following, I will develop an expansion of the density matrix that can serve as a starting point for a detailed investigation of the relations between the five measurement contexts described by the Hilbert space formalism.

3. Relations between Non-Orthogonal States

As explained in previous work, Kirkwood–Dirac distributions are the most consistent representation of quantum statistics, accurately reproducing the relations between non-orthogonal states in the Hilbert space formalism [33,34,35,36,37,38,39,40,41]. Such relations represent the non-classical structure of operator statistics and ensure that quantum theory cannot be reconciled with any classical statistical theory [19,36]. Here, the goal is to identify a specific set of relations between five different measurement contexts that make optimal use of the orthogonality relations used to demonstrate quantum contextuality. To achieve this, we can start with a selection of non-orthogonal states representing one of the five trajectories composed of three states, e.g., $\{\left|1\right.〉,\left|f\right.〉,\left|D2\right.〉\}$. These three state vectors form a complete non-orthogonal basis of the three-dimensional Hilbert space. The corresponding set of contravariant vectors is $\{\left|S2\right.〉,\left|2\right.〉,\left|S1\right.〉\}$, and the expansion of any state vector in this non-orthogonal basis can be expressed by representing the identity operator as a sum of three non-Hermitian operators:

$${\displaystyle \frac{\left|1\right.〉\left.〈S2\right|}{\langle S2|1\rangle }}+{\displaystyle \frac{\left|f\right.〉\left.〈2\right|}{\langle 2|f\rangle }}+{\displaystyle \frac{\left|D2\right.〉\left.〈S1\right|}{\langle S1|D2\rangle }}=\widehat{\mathrm{I}}.$$

(1)

The decomposition of the identity operator $\widehat{\mathrm{I}}$ into non-Hermitian operators provides us with an interesting alternative to the decomposition into projectors of orthogonal states used in conventional expansions of the quantum state. The individual non-Hermitian operators in Equation (1) can be used to represent the pre- and post-selected ensembles of weak measurements [42] and appear in the reconstruction of quantum states using direct measurements of the Kirkwood–Dirac distribution [21,22,34]. When interpreted statistically, these operators thus represent the quantum mechanical analog of a logical AND. To simplify the terminology, we can define the operator:

$$\widehat{\Lambda }(a,b):={\displaystyle \frac{\left|a\right.〉\left.〈b\right|}{\langle b|a\rangle }}.$$

(2)

As shown in [42], these operators describe the weak measurement statistics of an initial state $\left|a\right.〉$ under the post-selection condition $\left|b\right.〉$. The expectation values of these operators for the states $\left|a\right.〉$ and $\left|b\right.〉$ are necessarily one. Likewise, they are necessarily zero for every state that is orthogonal to either $\left|a\right.〉$ or $\left|b\right.〉$. These properties explain why the identity operator can be decomposed in the manner shown in Equation (1). In total, there are five operator relations of this kind:

$$\begin{array}{ccc}\hfill \widehat{\Lambda }(1,S2)+\widehat{\Lambda }(f,2)+\widehat{\Lambda }(D2,S1)& =& \widehat{\mathrm{I}}\hfill \\ \hfill \widehat{\Lambda }(S1,2)+\widehat{\Lambda }(S2,1)+\widehat{\Lambda }(3,f)& =& \widehat{\mathrm{I}}\hfill \\ \hfill \widehat{\Lambda }(f,1)+\widehat{\Lambda }(2,S1)+\widehat{\Lambda }(D1,S2)& =& \widehat{\mathrm{I}}\hfill \\ \hfill \widehat{\Lambda }(S2,S1)+\widehat{\Lambda }(1,f)+\widehat{\Lambda }(P1,2)& =& \widehat{\mathrm{I}}\hfill \\ \hfill \widehat{\Lambda }(2,f)+\widehat{\Lambda }(S1,S2)+\widehat{\Lambda }(P2,1)& =& \widehat{\mathrm{I}}\hfill \end{array}$$

(3)

It should be noted that each combination $(a,b)$ corresponds to one of the paths in Table 1. The five relations in Equation (3) express the identity as a sum of three operators corresponding to paths that do not share the same outcome in any of the five contexts. This mathematical relation can be interpreted intuitively by considering the relations between the operators $\widehat{\Lambda }(a,b)$ and the various basis states representing the different measurement contexts. The diagonal elements of the operator in the basis representing a specific context will be one for the state that is part of the path and zero for the two other states. For example, the state $\left|3\right.〉$ is part of paths $[S1,D2]$, $[f,3]$, $[S2,D1]$, and $[S1,S2]$. Therefore,

$$\left.〈3\right|\widehat{\Lambda }(S1,D2)\left|3\right.〉=\left.〈3\right|\widehat{\Lambda }(f,3)\left|3\right.〉=\left.〈3\right|\widehat{\Lambda }(D2,S1)\left|3\right.〉=\left.〈3\right|\widehat{\Lambda }(S1,S2)\left|3\right.〉=1.$$

(4)

For all other operators $\widehat{\Lambda }(a,b)$,

$$\begin{array}{ccc}& & \left.〈3\right|\widehat{\Lambda }(1,P2)\left|3\right.〉=\left.〈3\right|\widehat{\Lambda }(2,P1)\left|3\right.〉=\left.〈3\right|\widehat{\Lambda }(1,f)\left|3\right.〉\hfill \\ & & =\left.〈3\right|\widehat{\Lambda }(2,f)\left|3\right.〉=\left.〈3\right|\widehat{\Lambda }(1,S2)\left|3\right.〉=\left.〈3\right|\widehat{\Lambda }(2,S1)\left|3\right.〉=0.\hfill \end{array}$$

(5)

The operators $\widehat{\Lambda }(a,b)$ thus provide a quantum mechanical representation of 10 of the 11 paths shown in Table 1. In the following, these operators and the related expansion of arbitrary quantum states will be used to derive a method of quantum tomography that highlights the symmetry between the five different contexts using a multi-context version of a Kirkwood–Dirac quasiprobability.

4. Complete Expansions of Arbitrary States

The completeness relation in Equation (1) can be used to express any pure state $\left|\psi \right.〉$ as a superposition of the three states $\{\left|S2\right.〉,\left|2\right.〉,\left|S1\right.〉\}$, where the coefficients are given by inner products with the states $\{\left|1\right.〉,\left|f\right.〉,\left|D2\right.〉\}$:

$$\left|\psi \right.〉={\displaystyle \frac{\langle 1|\psi \rangle }{\langle 1|S2\rangle }}\left|S2\right.〉+{\displaystyle \frac{\langle f|\psi \rangle }{\langle f|2\rangle }}\left|2\right.〉+{\displaystyle \frac{\langle D2|\psi \rangle }{\langle D2|S1\rangle }}\left|S1\right.〉.$$

(6)

It should be noted that non-orthogonal expansions reveal a fundamental problem with the notion that each of the components represents a possible “reality”. The coefficients of the expansion are not zero when the state is orthogonal to that component, but they instead refer to a different set of states. This means that expansions of states orthogonal to one or two of the components will include non-zero contributions of those components. The problem is best illustrated by the expansion of the state $\left|D2\right.〉$:

$$\left|D2\right.〉={\displaystyle \frac{\langle 1|D2\rangle }{\langle 1|S2\rangle }}\left|S2\right.〉+{\displaystyle \frac{\langle f|D2\rangle }{\langle f|2\rangle }}\left|2\right.〉+{\displaystyle \frac{1}{\langle D2|S1\rangle }}\left|S1\right.〉.$$

(7)

Destructive interference between $\left|2\right.〉$ and $\left|S1\right.〉$ ensures that $\langle S2|D2\rangle =0$, and destructive interference between $\left|S2\right.〉$ and $\left|S1\right.〉$ ensures that $\langle 2|D2\rangle =0$. This requires non-trivial relations between the inner products:

$$\begin{array}{ccc}\hfill \langle 1|D2\rangle \langle D2|S1\rangle & =& -\langle 1|S2\rangle \langle S2|S1\rangle \hfill \\ \hfill \langle f|D2\rangle \langle D2|S1\rangle & =& -\langle f|2\rangle \langle 2|S1\rangle \hfill \end{array}$$

(8)

As this example shows, all of the inner products are related to each other by the orthogonality relations between the five different measurement contexts [29]. To simplify the relations, it is useful to introduce a specific numerical example. As noted in [29], the example can be defined by the expansion of the states $\left|S1\right.〉$ and $\left|S2\right.〉$ in the $\{\left|1\right.〉,\left|2\right.〉,\left|3\right.〉\}$ basis. It is therefore convenient to chose equal superpositions:

$$\begin{array}{ccc}\hfill \left|S1\right.〉& =& {\displaystyle \frac{1}{\sqrt{2}}}\left(\left|2\right.〉+\left|3\right.〉\right)\hfill \\ \hfill \left|S2\right.〉& =& {\displaystyle \frac{1}{\sqrt{2}}}\left(\left|1\right.〉+\left|3\right.〉\right).\hfill \end{array}$$

(9)

All other superpositions can then be derived using the orthogonality conditions of the relations between the five measurement contexts. Most notably, $\left|f\right.〉$ is orthogonal to both $\left|S1\right.〉$ and $\left|S2\right.〉$, so the representation of the state in the $\{\left|1\right.〉,\left|2\right.〉,\left|3\right.〉\}$ basis is necessarily given by

$$\left|f\right.〉={\displaystyle \frac{1}{\sqrt{3}}}\left(\left|1\right.〉+\left|2\right.〉-\left|3\right.〉\right).$$

(10)

With these definitions, the expansion of $\left|D2\right.〉$ in Equation (7) is given by

$$\left|D2\right.〉=\left|S2\right.〉+\sqrt{2}\left|2\right.〉-2\left|S1\right.〉.$$

(11)

Note that the physics represented by the state $\left|D2\right.〉$ cannot depend on its representation. The characterization of an unknown state should not be based on the components of a representation, but on the coefficients of the expansion and any physical meaning that they may have that is independent of the expansion in which they are used.

The coefficients of the non-orthogonal expansion of quantum states given by Equation (6) describe coherences between non-orthogonal states. Such coherences can be represented by Kirkwood–Dirac quasiprobabilities:

$$\varrho (a,b)=\langle b|a\rangle \left.〈a\right|\widehat{\rho }\left|b\right.〉.$$

(12)

As mentioned before, the Kirkwood–Dirac quasiprobabilities describe the relation between the quantum statistics of $\left|a\right.〉$ and $\left|b\right.〉$ in terms of the weak values of $\left|a\right.〉\left.〈a\right|$ post-selected in $\left|b\right.〉$ [21,22,43]. In the following, I will refer to them as KD terms. When the non-orthogonal expansion of Equation (6) is applied to a density operator $\widehat{\rho }$, the coefficients of the expansion can be expressed through a combination of probabilities and KD terms:

$$\begin{array}{ccc}\hfill \widehat{\rho }& =& 2P\left(1\right)\left|S2\right.〉\left.〈S2\right|+3P\left(f\right)\left|2\right.〉\left.〈2\right|+4P\left(D2\right)\left|S1\right.〉\left.〈S1\right|\hfill \\ & & +3\sqrt{2}\varrho (1,f)\left|S2\right.〉\left.〈2\right|+3\sqrt{2}\varrho (f,1)\left|2\right.〉\left.〈S2\right|\hfill \\ & & -2\varrho (1,D2)\widehat{\Lambda }(S2,S1)-2\varrho (D2,1)\widehat{\Lambda }(S1,S2)\hfill \\ & & -3\varrho (f,D2)\widehat{\Lambda }(2,S1)-3\varrho (D2,f)\widehat{\Lambda }(S1,2).\hfill \end{array}$$

(13)

The expansion combines the probabilities of the three outcomes 1, f, and $D2$ with KD terms for all three combinations: $(1,f)$, $(1,D2)$, and $(f,D2)$. These six elements correspond to the diagonal and off-diagonal terms of a density matrix, where the coherences are replaced by complex quasiprobabilities that assign a joint statistical weight to each pair of outcomes.

The expansion given in Equation (13) did not require any normalization of the density operator $\widehat{\rho }$. It is therefore possible to derive an additional relation between the six coefficients using the condition $\mathrm{Tr}\left(\widehat{\rho }\right)=1$. For example, it is possible to express the probability of $D2$ as

$$P\left(D2\right)={\displaystyle \frac{1}{4}}\left(1-2P\left(1\right)-3P\left(f\right)+\mathrm{Re}\left(4\varrho (1,D2)+6\varrho (D2,f)\right)\right).$$

(14)

A normalized quantum state is completely determined by the two probabilites $P\left(1\right)$ and $P\left(f\right)$, as well as the three KD terms $\varrho (1,f)$, $\varrho (1,D2)$, and $\varrho (D2,f)$. Experimentally, KD terms can be determined by weak measurements or by a variety of other methods involving quantum interferences [21,22,23,24,25,26]. Equations (13) and (14) show that the statistics of all possible measurements can be expressed by linear combinations of these five coefficients and the normalization. In the following, we will take a look at how this expansion of the quantum state relates different contexts to each other.

5. Reconstruction of a Kirkwood–Dirac Distribution

In the previous section, we have considered a complete expansion of arbitrary quantum states using non-orthogonal basis states. This expansion results in an unusual combination of KD terms with probabilities that seem to involve no less than three measurement contexts, which are represented by the states $\mid 1\rangle$, $\mid f\rangle$, and $\mid D2\rangle$. This is very different from a conventional Kirkwood–Dirac distribution, which describes the relation between two contexts based on all $d^2$ combinations of outcomes for a d-dimensional Hilbert space. Indeed, Kirkwood–Dirac distributions are usually obtained by expanding a density matrix in the context $\{\mid a\rangle \}$ on the left-hand side and in the context $\{\mid b\rangle \}$ on the right-hand side [28]. This non-symmetric expansion is a complete representation of the quantum state if all inner products $\langle a\mid b\rangle \}$ are non-zero [21,22,33,34]. However, this direct definition of the Kirkwood–Dirac quasiprobability as a mathematically complete expansion of the density operator cannot be applied to any pair of contexts within the five contexts discussed above, since at least two of the outcomes from different contexts will be orthogonal to each other. By definition, the KD terms for two orthogonal states are zero. This means that the Kirkwood–Dirac quasiprobability provides no direct information about the coherence between two orthogonal elements. It is therefore noteworthy that the expansion in Equation (13) can solve this problem by involving outcomes from more than two contexts. The selection of the specific expansion given by Equation (1) defines expansion coefficients associated with the path $[1,f]$ in Table 1. It is possible to relate these coefficients to a complete Kirkwood–Dirac distribution for the contexts defined by the unique outcomes $\left|3\right.〉$ and $\left|P2\right.〉$. This Kirkwood–Dirac distribution has a total of eight non-zero elements, as shown in

Table 2. Kirkwood–Dirac distribution of the contexts $\left|a\right.〉=\{\left|1\right.〉,\left|2\right.〉,\left|3\right.〉\}$ and $\left|b\right.〉=\{\left|f\right.〉,\left|S2\right.〉,\left|P2\right.〉\}$. The five KD terms highlighted in red are sufficient for a complete reconstruction of an unknown state.

	$\left|1\right.〉$	$\left|2\right.〉$	$\left|3\right.〉$
$\left|f\right.〉$	ϱ(1, f)	ϱ(2, f)	ϱ(3, f)
$\left|S2\right.〉$	ϱ(1, S2)	—–	$\varrho (3,S2)$
$\left|P2\right.〉$	ϱ(1, P2)	$\varrho (2,P2)$	$\varrho (3,P2)$

. Five of these elements are needed to determine the five coefficients of the expansion in Equation (13). It is convenient to use the five coefficients marked in red, since the coefficients can then be determined by simple sums. The probabilities $P\left(1\right)$ and $P\left(f\right)$ are marginals of the following distribution:

$$\begin{array}{ccc}\hfill P\left(1\right)& =& \varrho (1,f)+\varrho (1,S2)+\varrho (1,P2),\hfill \\ \hfill P\left(f\right)& =& \varrho (1,f)+\varrho (2,f)+\varrho (3,f).\hfill \end{array}$$

(15)

The remaining two coefficients are given by KD terms that involve $D2$. Since $\left|S2\right.〉$ is orthogonal to $\left|D2\right.〉$, $\varrho (1,D2)$ does not include the contribution of $\varrho (1,S2)$ to $P\left(1\right)$. The same holds true for $\varrho (D2,f)$ and $\varrho (2,f)$. The two remaining KD terms in the expansion are therefore given by

$$\begin{array}{ccc}\hfill \varrho (1,D2)& =& \varrho (1,f)+\varrho (1,P2),\hfill \\ \hfill \varrho (D2,f)& =& \varrho (1,f)+\varrho (3,f).\hfill \end{array}$$

(16)

Only five of the eight elements of the Kirkwood–Dirac distribution for the contexts $\{\left|1\right.〉,\left|2\right.〉,\left|3\right.〉\}$ and $\{\left|f\right.〉,\left|S2\right.〉,\left|P2\right.〉\}$ are needed to completely define an arbitrary quantum state. The relation between the contexts represented by the orthogonality of $\left|2\right.〉$ and $\left|S2\right.〉$ thus simplifies the characterization of an unknown state.

The Kirkwood–Dirac distribution is a helpful representation of the relations between two different contexts. It is therefore interesting to investigate how Equation (13) relates the five coefficients that appear in the expansion to the remaining three elements of the Kirkwood–Dirac distribution in Table 2. The easiest way to find the result is to consider the probabilities of $S2$ and 2:

$$\begin{array}{ccc}\hfill P\left(2\right)& =& {\displaystyle \frac{1}{2}}\left(1-2P\left(1\right)+3P\left(f\right)+\mathrm{Re}\left(4\varrho (1,D2)-6\varrho (D2,f)\right)\right)\hfill \\ \hfill P\left(S2\right)& =& {\displaystyle \frac{1}{4}}\left(1+6P\left(1\right)-3P\left(f\right)-\mathrm{Re}\left(12\varrho (1,D2)-6\varrho (D2,f)\right)\right).\hfill \end{array}$$

(17)

These marginals of the Kirkwood–Dirac distribution determine the KD terms $\varrho (3,S2)$ and $\varrho (2,P2)$ from the elements marked in red in Table 2. The remaining element $\varrho (3,P2)$ can be determined from the normalization of the distribution. Since $\left|2\right.〉$, $\left|S2\right.〉$, and $\left|D2\right.〉$ form the orthogonal basis of a third context, it is also possible to determine $\varrho (3,P2)$ from the probability of $D2$, as given in Equation (14). The three probabilities are related to the elements of the Kirkwood–Dirac distribution by

$$\begin{array}{ccc}\hfill P\left(2\right)& =& \varrho (2,f)+\varrho (2,P2),\hfill \\ \hfill P\left(S2\right)& =& \varrho (1,S2)+\varrho (3,S2),\hfill \\ \hfill P\left(D2\right)& =& \varrho (1,f)+\varrho (3,f)+\varrho (1,P2)+\varrho (3,P2).\hfill \end{array}$$

(18)

The remaining elements of the Kirkwood–Dirac distribution can then be expressed by the five KD terms used in the reconstruction of the quantum state:

$$\begin{array}{ccc}\hfill \varrho (3,P2)& =& {\displaystyle \frac{1}{4}}\left(1+\varrho (1,f)-3\varrho (2,f)-\varrho (3,f)-2\varrho (1,S2)-2\varrho (1,P2)\right),\hfill \\ \hfill \varrho (2,P2)& =& {\displaystyle \frac{1}{2}}\left(1-\varrho (1,f)+\varrho (2,f)-3\varrho (3,f)-2\varrho (1,S2)+2\varrho (1,P2)\right),\hfill \\ \hfill \varrho (3,S2)& =& {\displaystyle \frac{1}{4}}\left(1-3\varrho (1,f)-3\varrho (2,f)+3\varrho (3,f)+2\varrho (1,S2)-6\varrho (1,P2)\right).\hfill \end{array}$$

(19)

These equations represent a set of universal rules that relate the elements of a single Kirkwood–Dirac distribution to each other. It is worth looking at these rules in detail to figure out what kind of statistical relations they express.

The most obvious reason why there should be a relation between the elements is the normalization of the distribution. Probabilities attributed to the five elements used in the reconstruction will be missing from the three additional elements. If all of the elements used in the reconstruction are zero, the state is $\left|S1\right.〉$, and the three additional elements are $\varrho (3,P2)=1/4$, $\varrho (2,P2)=1/2$, and $\varrho (3,S2)=1/4$. It is interesting to observe that there is no degree of freedom left. There is only one state with $P\left(1\right)=P\left(f\right)=0$. Equation (19) then defines the necessary uncertainty of this state in terms of a joint probability of the remaining outcomes $(2,3)$ and $(S2,P2)$. We can also consider cases where there is only one non-zero element among the five elements used in the reconstruction of the state. Most obviously, this is the case for $\left|2\right.〉$, where $\varrho (2,f)=1/3$, as well as for $\left|S2\right.〉$, where $\varrho (1,S2)=1/2$. In both cases, a contribution proportional to the non-zero element is subtracted from two of the three elements in Equation (19), leaving only one additional element with a non-zero value. Naturally, this element obtains exactly the value needed for a total probability of one: $\varrho (2,P2)=2/3$ for $\left|2\right.〉$ and $\varrho (3,S2)=1/2$ for $\left|S2\right.〉$. It is worth noting that a value of $\varrho (2,f)$ greater than $1/3$ would result in a negative value of $P\left(S2\right)$, indicating a strict limit to the concentration of quasiprobabilities in a single KD term. A lower value of $\varrho (2,f)$ is possible, since it corresponds to a mixed state of $\left|S1\right.〉$ and $\left|2\right.〉$. The reason for the upper limit of $\varrho (2,f)$ is that the reconstructed state would be a non-positive state—specifically, a mixture of $\left|2\right.〉$ and $\left|S1\right.〉$ with a negative contribution from $\left|S1\right.〉$.

Finally, it may be instructive to consider states with particularly high probabilities of 1 and f, since the reconstruction method and the Kirkwood–Dirac distribution put special emphasis on these two outcomes. For $\left|f\right.〉$, the KD terms are $\varrho (1,f)=1/3$, $\varrho (2,f)=1/3$, and $\varrho (3,f)=1/3$, while $\varrho (1,S2)=\varrho (1,P2)=0$. As expected, Equation (19) then states that the remaining three elements of the distribution are zero. Any attempt to redistribute probabilities between the three non-zero elements will result in a redistribution of a total probability of zero between $P\left(S2\right)$ and $P\left(P2\right)$, resulting in at least one negative probability. Equation (19) thus allows only one probability distribution of 1, 2, and 3 for $P\left(f\right)=1$. The same holds true for the state $\left|1\right.〉$, where the only possible probability distributions are $\varrho (1,f)=1/3$, $\varrho (1,S2)=1/2$, and $\varrho (1,P2)=1/6$, while $\varrho (2,f)=\varrho (3,f)=0$.

Since we are interested in the relations between the two contexts in the Kirkwood–Dirac distribution, we can now take a look at a state that achieves high probabilities for both 1 and f simultaneously. Since such a state closely approximates the trajectory $[1,f]$, I will refer to it as the state $\left|T_{1f}\right.〉$. To keep the numerics simple, let us consider the following superposition:

$$\begin{array}{ccc}\hfill \left|T_{1f}\right.〉& :=& \sqrt{{\displaystyle \frac{4}{11}}}\left|1\right.〉+\sqrt{{\displaystyle \frac{3}{11}}}\left|f\right.〉\hfill \\ & =& {\displaystyle \frac{1}{\sqrt{11}}}\left(3\left|1\right.〉+\left|2\right.〉-\left|3\right.〉\right).\hfill \end{array}$$

(20)

This state has $P\left(S1\right)=0$ so that it can be represented as a superposition of $\left|1\right.〉$ and $\left|f\right.〉$. The interference between these two components is constructive, adding a probability of $4/11$ to the probabilities associated with the expansion coefficients. The state can also be written as

$$\left|T_{1f}\right.〉\left.〈T_{1f}\right|={\displaystyle \frac{4}{11}}\left|1\right.〉\left.〈1\right|+{\displaystyle \frac{3}{11}}\left|f\right.〉\left.〈f\right|+{\displaystyle \frac{2}{11}}\widehat{\Lambda }(1,f)+{\displaystyle \frac{2}{11}}\widehat{\Lambda }(f,1).$$

(21)

When written in this form, it is easy to derive the probabilities for any of the five measurement contexts. For $\left|1\right.〉$, we obtain contributions of 1 from all operators except for $\left|f\right.〉\left.〈f\right|$, where the contribution is $1/3$. The total probability is therefore $P\left(1\right)=9/11$. Similarly, the $\left|f\right.〉$ results in contributions of 1 except for $\left|1\right.〉\left.〈1\right|$, where the contribution is 1/3. The total probability is $P\left(f\right)=25/33$. In classical statistics, these probabilities would suggest a joint probability of 1 and f of at least $19/33$, obtained under the assumption that the system is always either in f or in 1. However, the corresponing KD term is only $\varrho (1,f)=15/33$. This is possible because negative values appear in the KD terms that are neither related to 1 nor to f.

Table 3. Kirkwood–Dirac distribution of the state $\left|T_{1f}\right.〉$. Quantum coherences between $\left|1\right.〉$ and $\left|f\right.〉$ increase $\varrho (1,f)$ and reduce $\varrho (S2,3)$ and $\varrho (2,P2)$ to negative values.

	$\left|1\right.〉$	$\left|2\right.〉$	$\left|3\right.〉$
$\left|f\right.〉$	15/33	5/33	5/33
$\left|S2\right.〉$	9/33	—–	−3/33
$\left|P2\right.〉$	3/33	−2/33	1/33

shows the complete Kirkwood–Dirac distribution of the state $\left|T_{1f}\right.〉$. According to Equation (19), an increase in $\varrho (1,f)$ will result in a decrease in $\varrho (3,S2)$ and $\varrho (2,P2)$. In the present case, we find that $\varrho (3,S2)=-3/33$, and $\varrho (2,P2)=-2/33$. The third KD term in Equation (19) has a positive value of $\varrho (3,P2)=1/33$. Equation (19) shows that these three values correspond to an addition of $+4/33$ to $\varrho (1,f)$ compared with a situation where all three KD terms are initially zero—the situation obtained for any incoherent mixture of $\left|1\right.〉$ and $\left|f\right.〉$. The superposition state $\left|T_{1f}\right.〉$ thus corresponds to a mixture of $\left|1\right.〉$ and $\left|f\right.〉$ with an addition of $+4/33$ to $\varrho (1,f)$. Specifically, a mixture of $6/11$ times $\left|1\right.〉\left.〈1\right|$ and $5/11$ times $\left|f\right.〉\left.〈f\right|$ has the same values for $\varrho (1,S2)$, $\varrho (1,P2)$, $\varrho (2,f)$, and $\varrho (3,f)$ as $\left|T_{1f}\right.〉$, but all of the remaining probabilities are found in $\varrho (1,f)=11/33$. The coherence between $\left|1\right.〉$ and $\left|f\right.〉$ increases the value of $\varrho (1,f)$ by $+4/33$, and this increase in $\varrho (1,f)$ results in an increase of $1/4$ times $4/33$ or $1/33$ in $\varrho (3,P2)$, a decrease of $-1/2$ times $4/33$ or $-2/33$ in $\varrho (2,P2)$, and a decrease of $-3/4$ times $4/33$ or $-3/33$ in $\varrho (3,S2)$. Equation (19) thus shows that negative KD terms are needed to achieve high values in other KD terms. The analysis in this section has shown that these relations are already obtained when only two contexts are related to each other. This is especially interesting, because the Kochen–Specker theorem assigns a cyclic and symmetric role to all five contexts, as illustrated in Figure 1. In the following, I will take a look at the inequality violation used to demonstrate contextuality in the present scenario and use it as a starting point for a formulation of cyclic relations between KD terms that correspond to non-contextual hidden variable theories for the five contexts.

6. Inequality Violations and Cyclic Relations between All Five Contexts

Non-contextual hidden variable models assume that the outcomes of all measurements are pre-determined for each individual system. For the three-path interferometer in Figure 2, this means that each particle follows a specific path through the five contexts corresponding to the paths shown in Table 1. If this is indeed the case, it is possible to identify extremal values of collective properties that define inequalities for the experimentally observable statistics. In the well-studied case of the five contexts defined here, the maximal number of shared outcomes that a path may include is two. This means that two is the maximal value we should obtain if we sum the probabilities of all shared outcomes [8]:

$$P\left(1\right)+P\left(2\right)+P\left(S1\right)+P\left(S2\right)+P\left(f\right)\le 2.$$

(22)

It is well known that this inequality can be violated, and various formulations of this paradox have been discussed [8,9]. Here, our goal is to relate the paradox to the KD terms that describe the relations between different measurement contexts.

In the tomographic reconstruction considered above, we have direct access to $P\left(1\right)$ and $P\left(f\right)$, as shown in Equation (15). $P\left(2\right)$ and $P\left(S2\right)$ are given in Equation (17). Since only the sum of $P\left(2\right)$ and $P\left(S2\right)$ is needed, it is sufficient to refer to $P\left(D2\right)$ in Equation (14) to obtain

$$P\left(2\right)+P\left(S2\right)={\displaystyle \frac{3}{4}}+{\displaystyle \frac{1}{2}}P\left(1\right)+{\displaystyle \frac{3}{4}}P\left(f\right)-\mathrm{Re}\left(\varrho (1,D2)+{\displaystyle \frac{3}{2}}\varrho (D2,f)\right).$$

(23)

This leaves only the probability $P\left(S1\right)$. Using Equations (13) and (14), this probability can be given as

$$P\left(S1\right)=1-{\displaystyle \frac{3}{2}}P\left(1\right)-{\displaystyle \frac{3}{2}}P\left(f\right)+3\mathrm{Re}\left(\varrho (1,f)\right).$$

(24)

The sum of all five probabilities is then given by

$$\begin{array}{ccc}\hfill \Sigma & :=& P\left(1\right)+P\left(2\right)+P\left(S1\right)+P\left(S2\right)+P\left(f\right)\hfill \\ & =& {\displaystyle \frac{7}{4}}+{\displaystyle \frac{1}{4}}P\left(f\right)+\mathrm{Re}\left(3\varrho (1,f)-\varrho (1,D2)-{\displaystyle \frac{3}{2}}\varrho (D2,f)\right)\hfill \\ & =& {\displaystyle \frac{7}{4}}+{\displaystyle \frac{1}{4}}\mathrm{Re}\left(3\varrho (1,f)+\varrho (2,f)-5\varrho (3,f)-4\varrho (1,P2)\right).\hfill \end{array}$$

(25)

An inequality violation is observed whenever the KD terms used in the reconstruction of the state satisfy

$$3\varrho (1,f)+\varrho (2,f)-5\varrho (3,f)-4\varrho (1,P2)>1.$$

(26)

Despite its negative KD terms, $\left|T_{1f}\right.〉$ does not satisfy this requirement and therefore fails to violate the inequality in Equation (22). This is not surprising, since the probability of $S1$ in $\left|T_{1f}\right.〉$ is zero, and the remaining probabilities in the sum $\Sigma$ can be divided into pairs from the same context. In general, the inequality given by Equation (22) cannot be violated when one of the five probabilities is zero. As pointed out in [8], a state that optimally achieves a violation can be obtained by superpositions of all five states representing the outcomes that contribute to the inequality, simultaneously achieving very similar probabilities for all five outcomes. A state that is close to the maximal inequality violation is given by [44]:

$$\left|N_x\right.〉={\displaystyle \frac{1}{3}}\left(2\left|1\right.〉+2\left|2\right.〉+\left|3\right.〉\right).$$

(27)

The tomographic data for this state is given by

$$\begin{array}{ccc}\hfill \varrho (1,f|N_x)& =& 2/9,\varrho (1,S2|N_x)=1/3,\hfill \\ \hfill \varrho (2,f|N_x)& =& 2/9,\varrho (1,P2|N_x)=-1/9,\hfill \\ \hfill \varrho (3,f|N_x)& =& -1/9.\hfill \end{array}$$

(28)

The value of the sum $\Sigma$ obtained with this data is $\Sigma =20/9$, violating the inequality by $2/9$. The individual probabilities are $P\left(1\right)=P\left(2\right)=4/9$, $P\left(S1\right)=P\left(S2\right)=1/2$, and $P\left(f\right)=1/3$.

Equation (26) indicates that the violation of the non-contextual inequality in Equation (22) can be explained by the negative $KD$ terms of the paths $[3,f]$ and $[1,P2]$. The tomographic data of Equation (28) thus explains the inequality violation in terms of its negative KD terms. However, Equation (22) does not require negative KD terms for its violation, and the specific selection of the tomographic data hides the fact that all five contexts play an equivalent role in the violation of the inequality. It is therefore desirable to derive a more complete form of the inequality, where the role of negativity and its relation to generalized probabilities as discussed in [15,16] is more apparent. In the case of the $\left|N_x\right.〉$ state, we would expect negative KD terms for all paths with four segments in Table 1. In addition to the paths $[3,f]$ and $[1,P2]$, these are the paths $[2.P1]$, $[S1,D2]$, and $[S2,D1]$. If we simply complete the Kirkwood–Dirac distribution, we obtain the terms $\varrho (3,S2)$, $\varrho (2,P2)$, and $\varrho (3,P2)$. According to Table 1, each of these combinations of outcomes occurs in two of the eleven paths. The Hilbert space algebra confirms that the KD terms can be expressed in terms of the corresponding sums:

$$\begin{array}{ccc}\hfill \varrho (3,S2)& =& \varrho (S1,S2)+\varrho (D1,S2),\hfill \\ \hfill \varrho (2,P2)& =& \varrho (2,S1)+\varrho (2,P1),\hfill \\ \hfill \varrho (3,P2)& =& \varrho (S1,D2)+\varrho \left(0\right).\hfill \end{array}$$

(29)

All of the KD terms defined by pairs of outcomes can be determined using Equation (12). The KD term $\varrho \left(0\right)$ of the path through all five unique outcomes needs a different definition. The general definition can be obtained by referring to the normalization of the Kirkwood–Dirac distribution:

$$\begin{array}{cc}\hfill \varrho \left(0\right)+\varrho (S1,D2)+\varrho (D1,S2)+\varrho (2,P1)+\varrho (1,P2)+\varrho (3,f)& \\ \hfill +\varrho (1,f)+\varrho (2,f)+\varrho (1,S2)+\varrho (2,S1)+\varrho (S1,S2)& =1.\hfill \end{array}$$

(30)

This equation can be simplified by summarizing the KD terms associated with 1 and 2 for

$$\varrho \left(0\right)+\varrho (S1,D2)+\varrho (D1,S2)+\varrho (3,f)+\varrho (S1,S2)=P\left(3\right).$$

(31)

Independently, the fact that $\left|S1\right.〉\left.〈S1\right|+\left|D1\right.〉\left.〈D1\right|$ projects states into the Hilbert space spanned by $\left|2\right.〉$ and $\left|3\right.〉$, while $\left|S2\right.〉\left.〈S2\right|+\left|D2\right.〉\left.〈D2\right|$ does the same for $\left|1\right.〉$ and $\left|3\right.〉$, can be used to derive a very similar expression for $P\left(3\right)$:

$$P\left(3\right)=\varrho (S1,S2)+\varrho (D1,S2)+\varrho (S1,D2)+\varrho (D1,D2).$$

(32)

Based on these relations, a conveniently symmetric definition of $\varrho \left(0\right)$ is

$$\varrho \left(0\right)=\varrho (D1,D2)-\varrho (3,f).$$

(33)

In general, $\varrho \left(0\right)$ contributes to all KD terms defined by pairs of outcomes that are unique to two different contexts.

Equation (30) shows that all quantum states can be characterized by a quasiprobability distribution given by eleven KD terms that correspond to the 11 possible paths through the five contexts shown in Table 1. If all KD terms are positive, these eleven terms can be interpreted as a probability distribution of the possible joint assignment of measurement outcomes to all five contexts. The eleven KD terms can thus establish a link between quantum statistics and hidden variable theories. However, quantum theory introduces additional relations between the eleven KD terms. Since only five KD terms are needed to characterize the quantum state completely, there should be a set of six relations that determine the remaining six KD terms as a function of these five. One of these relations is the normalization, and this relation can be used to determine $\varrho \left(0\right)$ once the other five KD terms have been determined. The remaining five relations can be derived from Equation (3), where each KD term is given by the expectation value of the operator $\widehat{\Lambda }(a,b)$ according to

$$\varrho (a,b)={\left|\langle a|b\rangle \right|}^2\mathrm{Tr}\left(\widehat{\rho }\widehat{\Lambda (b,a)}\right).$$

(34)

Equation (3) thus defines a set of deterministic relations between the KD terms that express a distribution of statistical weights over the possible quasirealities obtained by assigning a specific measurement outcome to each context. These fundamental constraints satisfied by the KD terms of all possible input states are given by

$$\begin{array}{ccc}\hfill 2\varrho (1,S2)+3\varrho (f,2)+4\varrho (D2,S1)& =& 1\hfill \\ \hfill 2\varrho (S1,2)+2\varrho (S2,1)+3\varrho (3,f)& =& 1\hfill \\ \hfill 3\varrho (f,1)+2\varrho (2,S1)+4\varrho (D1,S2)& =& 1\hfill \\ \hfill 4\varrho (S2,S1)+3\varrho (1,f)+6\varrho (P1,2)& =& 1\hfill \\ \hfill 3\varrho (2,f)+4\varrho (S1,S2)+6\varrho (P2,1)& =& 1\hfill \end{array}$$

(35)

The KD terms in each line correspond to paths that do not share the same measurement outcome in any of the five contexts. It is therefore possible to assign exactly one KD term to each outcome in each of the five lines. For outcomes that are shared by two contexts, this results in three non-zero KD terms, e.g., $\varrho (1,S2)=1/2$, $\varrho (1,f)=1/3$, and $\varrho (1,P2)=1/6$ for $\left|1\right.〉$. Since the sum is one, the eleventh KD term is $\varrho \left(0\right)=0$. On the other hand, outcomes that are unique to only one context have a total of five non-zero KD terms. For the state $\left|3\right.〉$, these are the four positive KD terms $\varrho (D2,S1)=1/4$, $\varrho (3,f)=1/3$, $\varrho (D1,S2)=1/4$, and $\varrho (S1,S2)=1/4$, as well as the fifth negative KD term $\varrho \left(0\right)=-1/12$.

The appearance of a negative KD term in states associated with a specific measurement outcome may seem surprising, since such a state is related to the other contexts directly by its inner products with the outcomes of those contexts. Indeed, $\varrho \left(0\right)$ is somewhat special, since it cannot be derived from a single weak value and does not appear in any Kirkwood–Dirac distribution of only two measurement contexts. It can be confirmed that, for all outcomes that are unique to only one context, the value of this KD term is $\varrho \left(0\right)=-1/12$. The origin of this value is found in a Bargmann invariant defined by the Hilbert space vectors representing the five outcomes that are unique to each context:

$$\langle 3|D1\rangle \langle D1|P1\rangle \langle P1|P2\rangle \langle P2|D2\rangle \langle D2|3\rangle =-{\displaystyle \frac{1}{12}}.$$

(36)

In principle, it is possible to show that Equation (33) identifies this Bargmann invariant with $\varrho \left(0\right)$ for the states $\left|3\right.〉$, $\left|D1\right.〉$, $\left|P1\right.〉$, $\left|P2\right.〉$, and $\left|D2\right.〉$. However, it might be more useful to show that the value of $\varrho \left(0\right)$ must be the same for each of these states. This can be done by determining $\varrho \left(0\right)$ for a maximally mixed state, $\widehat{\rho }=\widehat{\mathrm{I}}/3$. For that state, $\varrho (a,b)={\left|\langle a|b\rangle \right|}^2/3$, and $\varrho \left(0\right)=-1/36$. Specifically, $\varrho \left(0\right)$ is negative, because ${\left|\langle 3|f\rangle \right|}^2>{\left|\langle D1|D2\rangle \right|}^2$. Since $\varrho \left(0\right)$ is zero for each state shared by two contexts, the negative value of the maximally mixed state must correspond to one third of the value of the state unique to that context for each decomposition of the mixed state into a mixture of states representing the measurement outcomes of any of the three contexts.

In general, all quantum states can be described by the 11 quasiprobabilities of the different paths through the contexts. However, even mixed states tend to have negative KD values. The only states that result in positive non-contextual probabilities are states that can be expressed as mixtures of the five outcomes that are shared by two contexts. Significantly, this does not include the maximally mixed state $\widehat{\mathrm{I}}/3$ with its negative KD term of $\varrho \left(0\right)=-1/36$. The violation of non-contextuality as given by Equation (22) requires a very specific combination of negative KD terms. To find these, we can make use of the correspondence between KD terms and classical paths through the contexts. Path $\left[0\right]$ has no shared outcomes; paths $[1,P2]$, $[S1,D2]$, $[f,3]$, $[S2,D1]$, and $[2,P1]$ have one shared outcome each; and the remaining five paths have two shared outcomes. The average number of shared outcomes can then be found by subtracting the KD terms of the paths with less than two shared outcomes from an initial value of two:

$$\Sigma =2-\varrho (S1,D2)-\varrho (D1,S2)-\varrho (2,P1)-\varrho (1,P2)-\varrho (3,f)-2\varrho \left(0\right).$$

(37)

This expression for the probability sum $\Sigma$ shows that a violation of inequality (22) is impossible unless some of the KD terms given are negative. As suggested by Spekkens in [15], the origin of contextuality paradoxes can be traced to negative KD terms that suppress the contributions of paths with one shared outcome or none at all. Importantly, Kochen–Spekker scenarios require a joint probability characterizing more than two contexts. The eleven KD terms introduced here achieve this for all five contexts, even though each KD term is defined by only two outcomes. This is the significance of Table 1. Quantum contextuality actually provides an extended probability theory that corresponds to the assignment of outcomes to all five contexts at the same time.

7. Contextual Fluctuations

Quasiprobabilities are useful expressions of non-classical statistics, since they establish the closest possible analogy between “classical” statistics based on measurement-independent realities and the correct mathematical formulation of contextual quantum statistics. The closest analogy between the “classical” reality of photon paths through all five contexts and quantum statistics is achieved by using the complete set of eleven KD terms that assign joint statistical weights to all five contexts. Here, each KD term represents a specific combination of outcomes for each of the five contexts, corresponding to one of the paths in Table 1. Like other quasiprobabilities, this multi-context Kirkwood–Dirac distribution seeks to bridge the gap between a naive realism that assumes a co-existence of the measurement outcomes along a path and an equally naive anti-realism that simply assumes that there is no connection between the outcomes of different measurements. To guard ourselves against both misconceptions, we should accept that the 11 element Kirkwood–Dirac distribution describes a deterministic relation between the five different measurement contexts that cannot be reconciled with any joint reality of outcomes. To understand the concept of quantum contextuality better, we need to focus on the fact that only one measurement context describes the reality of an actual event. Measurement probabilities are obtained from sums of KD terms consistent with this event, and all of these sums will necessarily be positive. Negative KD terms can never be observed in isolation, since their experimentally observable effects appear only in sums where the contributions from positive KD terms outweigh those from the negative ones. Quantum contextuality suggests that the reality observed in one measurement context is fundamentally different from the reality observed in a different measurement context. However, the KD terms $\varrho (a,b)$ still describe a well-defined relation between the outcomes a and b. We should consider this in more detail—if the relation between a and b is valid when we measure a, then the measurement of a determines the reality of b as well. The set of values provided by the KD terms thus describes a contextual reality of b conditioned by the directly observed outcome a.

The problem that prevents us from understanding quantum mechanics is that we identify physical properties with their eigenvalue, clinging to the belief that these eigenvalues are a complete and exclusive set of possibilities from which we can construct an objective reality. However, this is a very unlikely interpretation of the formalism. Instead, we should consider a much wider variety of possibilities, where the outcomes of context a are related to the contextual realities of the outcomes of b that are different from the limited options of the binary yes or no associated with the eigenvalues of the projectors [45,46,47,48]. We need to accept that the values of the projectors associated with other measurement outcomes can be different from 0 or 1. Is it possible to make intuitive sense of such a possibility? Perhaps a change of terminology is needed—the “truth value” applies only to actual observations. Here, I will use the term “outcome value”, or more specifically, “contextual outcome value”. It is easiest to visualize this change by considering the case of a single particle propagating along paths. As has already been demonstrated experimentally [46], a particle is genuinely de-localized between different paths when these paths interfere in the output. We can now generalize this result to all quantum systems by describing the conditional reality of outcomes of b as a physical distribution of the total outcome value of 1 over the outcomes of b.

The concept of a context-dependent reality can be illustrated using the relations between measurement contexts described by the KD terms. Each measurement context has three outcomes: two of which are shared with other contexts and one of which is unique to that context. The measurement probabilities are given by the sums of three KD terms for each of the shared outcomes and by a sum of the remaining five KD terms for the outcome unique to this context. For instance, the probabilities of the context $\{\left|1\right.〉,\left|2\right.〉,\left|3\right.〉\}$ are given by

$$\begin{array}{ccc}\hfill P\left(1\right)& =& \varrho (1,f)+\varrho (1,S2)+\varrho (1.P2),\hfill \\ \hfill P\left(2\right)& =& \varrho (2,f)+\varrho (2,S1)+\varrho (2.P1),\hfill \\ \hfill P\left(3\right)& =& \varrho (3,f)+\varrho (S1,D2)+\varrho (D1,S2)+\varrho (S1,S2)+\varrho \left(0\right).\hfill \end{array}$$

(38)

Each measurement outcome assigns a truth value of 1 to one of the three outcomes and a truth value of 0 to the others. These truth values correspond to eigenvalues of the projection operators $\left|a\right.〉\left.〈a\right|$ of the measurement context. Quantum mechanics strongly suggests that we should not assign such eigenvalues to unobserved measurement outcomes. Instead, weak values can be used to relate the outcomes of b to a post-selected outcome of the measurement context a [30,45,46]. Using the terminology above, these weak values are the contextual outcome values of b in the context of a. If the measurement outcome a is obtained with a probability of $P\left(a\right)$, the contextual outcome value $W\left(b\right|a)$ of an outcome b is given by

$$W\left(b\right|a)={\displaystyle \frac{\varrho (b,a)}{P\left(a\right)}}.$$

(39)

Technically, $W\left(b\right|a)$ corresponds to the complex conditional probabilities discussed in some of my previous work [33,34], where the status of the Kirkwood–Dirac distribution as a symmetric quasiprobability of two measurement contexts was emphasized. However, contextuality suggests that we need to abandon the idea of a truth value of zero or one for outcomes that are not observed in their own context—or “eigencontext”, as I shall refer to it in the following.

If the contextual outcome values $W\left(b\right|a)$ are considered as individual values of the observable represented by the projector $\left|b\right.〉\left.〈b\right|$, it is possible to define the fluctuations of this value in the context of a by assigning the probabilities $P\left(a\right)$ to the corresponding outcome values $W\left(b\right|a)$. The dependence of $W\left(b\right)$ on a thus represents the fluctuations of b in the context a. Since the average value of $\left|b\right.〉\left.〈b\right|$ is equal to the probability $P\left(b\right)$, the fluctuation of $W\left(b\right|a)$ is given by

$$\Delta W_b^2=\sum _a{(W\left(b\right|a)-P\left(b\right))}^{2}P\left(a\right).$$

As shown in a number of previous works, the fluctuations of weak values over a complete set of post-selected outcomes are equal to the uncertainty of the corresponding observable for pure state inputs [45,49,50]. For the contextual values $W\left(b\right|a)$,

$$\begin{array}{ccc}\hfill \sum _a{\left(W\left(b\right|a)\right)}^{2}P\left(a\right)& \le & P\left(b\right),\hfill \\ \hfill \Delta W_b^2& \le & P\left(b\right)(1-P(b\left)\right).\hfill \end{array}$$

(40)

In the limit of pure states and precise measurements, the contextual fluctuations of the outcome values are equal to the fluctuations of the truth value observed in a direct measurement of the corresponding context. This is strong evidence that the values $W\left(b\right|a)$ describe a context-dependent reality.

To connect this analysis with quantum tomography, it is necessary to consider the role of the initial state in the definition of contextual fluctuations. If the state is a pure state, it is characterized by a deterministic statement corresponding to a certain prediction of a measurement outcome. Whether this prediction is explicit of not, it has an effect on the outcome values. An example may help to explain this dependence. Let us consider the state $\left|P2\right.〉$. Its eleven KD terms are given by

$$\begin{array}{ccccc}\hfill \varrho (S1,S2|P2)& =0\hfill & \hfill \varrho (3,f|P2)& =0\hfill & \hfill \\ \hfill \varrho (1,S2|P2)& =0\hfill & \hfill \varrho (2,P1|P2)& =1/6\hfill & \hfill \\ \hfill \varrho (2,S1|P2)& =1/2\hfill & \hfill \varrho (1,P2|P2)& =1/6\hfill & \hfill \\ \hfill \varrho (1,f|P2)& =0\hfill & \hfill \varrho (S1,D2|P2)& =1/4\hfill & \hfill \\ \hfill \varrho (2,f|P2)& =0\hfill & \hfill \varrho (D1,S2|P2)& =0\hfill & \hfill \varrho \left(0\right|P2)=-1/12.\end{array}$$

(41)

As explained above, the KD term $\varrho \left(0\right)$ of this state is negative. We can therefore expect to find negative outcome values for some of the outcomes when the measurement context is not the eigencontext of that outcome. For instance, the values $W\left(D1\right|P2)$ of the outcome $\left|D1\right.〉$ and their probabilities in the context $\{\left|1\right.〉,\left|2\right.〉,\left|3\right.〉\}$ are given by

$$\begin{array}{ccccc}\hfill P\left(1\right|P2)& =1/6\hfill & \hfill W\left(D1\right|P2;1)& =0\hfill & \hfill \\ \hfill P\left(2\right|P2)& =2/3\hfill & \hfill W\left(D1\right|P2;2)& =1/4\hfill & \hfill \\ \hfill P\left(3\right|P2)& =1/6\hfill & \hfill W\left(D1\right|P2;3)& =-1/2.\hfill \end{array}$$

(42)

Note that I have included the initial state in the conditions that determine the value $W\left(D1\right)$. This is intended as a reminder of the physical meaning of these values. The combination of state preparation $P2$ and measurement outcome m completely describes the physical reality of the system. The value $W\left(D1\right)$ for this physical reality is determined by these conditions, leaving no uncertainty in the value of the projector $\left|D1\right.〉\left.〈D1\right|$. We can confirm this assumption by finding the fluctuation of $W\left(D1\right)$. In every measurement context, the average value of $W\left(D1\right)$ in the state $\left|P2\right.〉$ is equal to the probability of $D1$, which is $P\left(D1\right|P2)=1/12$. The fluctuations of $W\left(D1\right)$ in $\left|P2\right.〉$ are given by

$$\Delta W_{D1}^2={\left({\displaystyle \frac{1}{12}}\right)}^2{\displaystyle \frac{1}{6}}+{\left({\displaystyle \frac{1}{6}}\right)}^2{\displaystyle \frac{2}{3}}+{\left({\displaystyle \frac{7}{12}}\right)}^2{\displaystyle \frac{1}{6}}={\displaystyle \frac{1}{12}}\left(1-{\displaystyle \frac{1}{12}}\right).$$

(43)

The fluctuations obtained for the context $\{\left|1\right.〉,\left|2\right.〉,\left|3\right.〉\}$ are the same as the fluctuations obtained in the eigencontext of $\left|D1\right.〉$, even though the outcome values themselves are very different. This is a strong indication that the individual realities associated with this fluctuation depend on the measurement context. In its eigencontext, the outcome $D1$ simply has a low probability. In the context $\left|1\right.〉,\left|2\right.〉,\left|3\right.〉\}$, the main contribution to the expectation value of $P\left(D1\right)=1/12$ originates from the most likely outcome of 2, for which the outcome value of $D1$ is $1/4$, which is much smaller than a truth value of 1 but still three times larger than the average. The negative outcome value of $-1/2$ for the less likely outcome of 3 is needed to achieve the same average and the same fluctuations as the eigencontext of $D1$. This is similar to the negative presence of a particle in a two-path interferometer that was reported in the neutron interference experiment in [46]. In general, negative contextual outcome values are observed when the probability of the observed outcome is low because of the destructive interference between outcomes of the unobserved context. Quantum formalism assigns precise conditional values to these unobserved outcomes, providing a context dependent description of the quantum fluctuations of $D1$ in the input state $\left|P2\right.〉$. Weak values and KD terms represent deterministic relations that allow statements about the values of unobserved physical properties. In the case of projectors, negative values indicate a particularly strong deterministic suppression of contributions from this outcome.

The initial state defines averages and fluctuations. It does not define outcome values and probabilities. These can only be determined when the context is fixed. It is therefore interesting to take a look at states that seek to minimize the uncertainties of the outcomes in all five contexts. As discussed above, the state $\left|T_{1f}\right.〉$ achieves this by enhancing the KD term that corresponds to the path $[1,f]$ through the five contexts. Table 3 already shows five of the eleven KD terms for this state. The complete set of eleven KD terms can be obtained from the table by making use of the observation that the $\left|T_{1f}\right.〉$ state has $P\left(S1\right)=0$. Therefore, each of the eight elements in the table appears as one of the eleven KD terms, while the remaing three KD terms have values of zero. The complete set of KD terms is given by

$$\begin{array}{ccccc}\hfill \varrho (S1,S2|T_{1f})& =0\hfill & \hfill \varrho (3,f|T_{1f})& =5/33\hfill & \hfill \\ \hfill \varrho (1,S2|T_{1f})& =3/11\hfill & \hfill \varrho (2,P1|T_{1f})& =-2/33\hfill & \hfill \\ \hfill \varrho (2,S1|T_{1f})& =0\hfill & \hfill \varrho (1,P2|T_{1f})& =1/11\hfill & \hfill \\ \hfill \varrho (1,f|T_{1f})& =5/11\hfill & \hfill \varrho (S1,D2|T_{1f})& =0\hfill & \hfill \\ \hfill \varrho (2,f|T_{1f})& =5/33\hfill & \hfill \varrho (D1,S2|T_{1f})& =-1/11\hfill & \hfill \varrho \left(0\right|T_{1f})=1/33.\end{array}$$

(44)

The state $\left|T_{1f}\right.〉$ satisfies the conditions given by Equation (35) by compensating the enhancement of $\varrho (1,f)$ above its eigenstate value of $1/3$ with two negative KD terms: $\varrho (2,P1)$ and $\varrho (D1,S2)$. Note that the paths $[2,P1]$ and $[S2,D1]$ represented by these two KD terms do not share any outcome with $[1,f]$ in Table 1. This means that simultaneous control of five contexts is achieved at least in part by suppressing the joint contributions of the unintended outcomes. We expect to see this effect more clearly in the relation between observed outcomes and contextual outcome values.

We can now consider the statistics of outcome values for f, $S2$, and $P2$ in the measurement context $\{\left|1\right.〉,\left|2\right.〉,\left|3\right.〉\}$. The probabilities of each outcome and the contextual distribution of the total outcome value of 1 over f, $S2$, and $P2$ are given by

$$\begin{array}{ccccccccc}\hfill P\left(1\right|T_{1f})& =9/11\hfill & \hfill W\left(f\right|T_{1f};1)& =5/9\hfill & \hfill W\left(S2\right|T_{f1};1)& =1/3\hfill & \hfill W\left(P2\right|T_{f1};1)& =1/9\hfill & \hfill \\ \hfill P\left(2\right|T_{1f})& =1/11\hfill & \hfill W\left(f\right|T_{1f};2)& =5/3\hfill & \hfill W\left(S2\right|T_{1f};2)& =0\hfill & \hfill W\left(P2\right|T_{1f};2)& =-2/3\hfill & \hfill \\ \hfill P\left(3\right|T_{1f})& =1/11\hfill & \hfill W\left(f\right|T_{1f};3)& =5/3\hfill & \hfill W\left(S2\right|T_{1f};3)& =-1\hfill & \hfill W\left(P2\right|T_{1f};3)& =1/3.\hfill \end{array}$$

(45)

The first thing that should be noted is the way in which the high average of $P\left(f\right)=25/33$ for the outcome f is obtained. Instead of assigning a high value to the most likely outcome of 1, the Hilbert space formalism determines a value of only $5/9$ ($11/15$ of the average of $25/33$) to this outcome. Oppositely, the less likely outcomes 2 and 3 have a contextual value of $5/3$ for f, exceeding the eigenvalue limit of 1 of the projector. This is a direct consequence of the relations in Equation (35), where the high value of $P\left(f\right)$ is achieved by the suppressing joint contributions of $(2,3)$ and $(S2,P2)$. This suppression is directly visible in the negative outcome values of $W\left(P2\right)$ for outcome 2 and of $W\left(S2\right)$ for outcome 3. If the distributions of outcome values are viewed together, the concentration of outcome values on f can be observed for each of the outcomes, but it is much weaker for the most likely outcome of 1 and extremely enhanced by the outcome values outside the eigenvalue range for the less likely outcomes of 2 and 3.

As noted above, the statistical fluctuations of the outcome values are all equal to the corresponding truth value fluctuations of the eigencontext statistics given by the probabilities of $P\left(f\right)=25/33$, $P\left(S2\right)=2/11$, and $P\left(P2\right)=2/33$. This can be confirmed by considering the differences between the outcome values and the average values represented by the probabilities:

$$\begin{array}{c}\hfill \Delta W_f^2={\left({\displaystyle \frac{20}{99}}\right)}^2{\displaystyle \frac{9}{11}}+{\left({\displaystyle \frac{10}{11}}\right)}^2{\displaystyle \frac{1}{11}}+{\left({\displaystyle \frac{10}{11}}\right)}^2{\displaystyle \frac{1}{11}}={\displaystyle \frac{25}{33}}\left(1-{\displaystyle \frac{25}{33}}\right),\\ \hfill \Delta W_{S2}^2={\left({\displaystyle \frac{5}{33}}\right)}^2{\displaystyle \frac{9}{11}}+{\left({\displaystyle \frac{2}{11}}\right)}^2{\displaystyle \frac{1}{11}}+{\left({\displaystyle \frac{13}{11}}\right)}^2{\displaystyle \frac{1}{11}}={\displaystyle \frac{2}{11}}\left(1-{\displaystyle \frac{2}{11}}\right),\\ \hfill \Delta W_{P2}^2={\left({\displaystyle \frac{5}{99}}\right)}^2{\displaystyle \frac{9}{11}}+{\left({\displaystyle \frac{8}{11}}\right)}^2{\displaystyle \frac{1}{11}}+{\left({\displaystyle \frac{3}{11}}\right)}^2{\displaystyle \frac{1}{11}}={\displaystyle \frac{2}{33}}\left(1-{\displaystyle \frac{2}{33}}\right).\end{array}$$

(46)

The statistical fluctuations of the outocomes f, $S2$, and $P2$ observed in their eigencontext are fully reproduced by the statistics of the outcome values associated with the outcomes 1, 2, and 3. No additional randomness is required—the fluctuation of the outcomes in the context $\{\left|1\right.〉,\left|2\right.〉,\left|3\right.〉\}$ has the same physical origin as the fluctuation of the outcomes in the context $\{\left|f\right.〉,\left|S2\right.〉,\left|P2\right.〉\}$.

As shown above, the inequality violations usually associated with quantum contextuality are a natural consequence of the dependence of the outcome values on the measurement context, where negative contextual outcome values appear as fluctuations associated with low probability outcomes. Inequality violations can be achieved when a specific set of KD terms is negative, as shown in Equation (37). In the case of the state $\left|N_x\right.〉$, five of the six $KD$ terms contributing to the inequality violation are negative. The complete set of eleven KD terms can be obtained from the five terms in Equation (28) and the relations in Equation (35). The results read as

$$\begin{array}{ccccc}\hfill \varrho (S1,S2|N_x)& =1/4\hfill & \hfill \varrho (3,f|N_x)& =-1/9\hfill & \hfill \\ \hfill \varrho (1,S2|N_x)& =1/3\hfill & \hfill \varrho (2,P1|N_x)& =-1/9\hfill & \hfill \\ \hfill \varrho (2,S1|N_x)& =1/3\hfill & \hfill \varrho (1,P2|N_x)& =-1/9\hfill & \hfill \\ \hfill \varrho (1,f|N_x)& =2/9\hfill & \hfill \varrho (S1,D2|N_x)& =-1/12\hfill & \hfill \\ \hfill \varrho (2,f|N_x)& =2/9\hfill & \hfill \varrho (D1,S2|N_x)& =-1/12\hfill & \hfill \varrho \left(0\right|N_x)=5/36.\end{array}$$

(47)

The value of the sum $\Sigma$ is given by

$$\Sigma \left(N_x\right)=2+3\times {\displaystyle \frac{1}{9}}+2\times {\displaystyle \frac{1}{12}}-2\times {\displaystyle \frac{5}{36}}=2+{\displaystyle \frac{2}{9}}.$$

(48)

The inequality (22) is violated, because the negative KD terms for paths with one shared outcome outweigh twice the positive KD term of the path with no shared outcome. The $\left|N_x\right.〉$ state is nearly symmetric in the five contexts, with maximal suppression of the outcomes unique to each context. For the measurement context $\{\left|1\right.〉,\left|2\right.〉,\left|3\right.〉\}$, the measurement probabilities and the contextual values $W\left(f\right)$, $W\left(S2\right)$, and $W\left(P2\right)$ are given by

$$\begin{array}{ccccccccc}\hfill P\left(1\right|N_x)& =4/9\hfill & \hfill W\left(f\right|N_x;1)& =1/2\hfill & \hfill W\left(S2\right|N_x;1)& =3/4\hfill & \hfill W\left(P2\right|N_x;1)& =-1/4\hfill & \hfill \\ \hfill P\left(2\right|N_x)& =4/9\hfill & \hfill W\left(f\right|N_x;2)& =1/2\hfill & \hfill W\left(S2\right|N_x;2)& =0\hfill & \hfill W\left(P2\right|N_x;2)& =1/2\hfill & \hfill \\ \hfill P\left(3\right|N_x)& =1/9\hfill & \hfill W\left(f\right|N_x;3)& =-1\hfill & \hfill W\left(S2\right|N_x;3)& =3/2\hfill & \hfill W\left(P2\right|N_x;3)& =1/2\hfill \end{array}$$

(49)

The first thing to note regarding the dependence of the contextual values on the measurement outcomes is the dominance of suppression effects over enghancement effects. For an outcome of 1, the contextual values of f and $S2$ are nearly equal, but the contextual value of $P2$ is negative. For an outcome of 3, the value of $S2$ is greater than one, but the value of $P2$ is also much larger than its average of $P\left(P2\right)=1/12$. More impressive is the suppression of the value of f to $-1$, especially since the average value of f is $P\left(f\right)=1/3$. For the outcome 2, the value of $S2$ is the eigenvalue of zero, but it is worth noting that the conditional values of f and $P2$ are the same. Each outcome has high values for two of the three outcomes, even though the statistics of these outcome values fully explain the input state uncertainties, leaving no room for any uncertainties in the conditional values themselves. Negative contextual values describe the deterministic relations between different contexts as extreme anti-correlations, exceeding the limits based on the assumption that the maximal correlation is a joint reality represented by a conditional value of one for a specific outcome and a value of zero for the two others.

It is important to remember that a non-contextual reality can only assign truth values of zero or one to each outcome—whether the outcome is actually observed in the measurement or not. The operator formalism contradicts this assumption on a seemingly technical level by assigning weak values defined by the combination of deterministic input state properties with the additional information provided by the measurement context. The statistical analysis of the distribution over weak values for a specific measurement shows that weak values represent the same uncertainties that are also described by the eigenvalues, even though the individual values are different. In the case of the outcome f, the average value defined by the initial condition $\left|N_x\right.〉$ is $P\left(f\right)=1/3$ in every context. In the eigencontext, a value of 1 is obtained with a probability of $1/3$, and a value of zero is obtained elsewise. In the context $\{\left|1\right.〉,\left|2\right.〉,\left|3\right.〉\}$, a value of $1/2$ is obtained with a probability of $8/9$, and a value of $-1$ is obtained with a probability of $1/9$. The fluctuations of the two contexts are the same:

$$\Delta W_f^2={\left({\displaystyle \frac{1}{6}}\right)}^2{\displaystyle \frac{8}{9}}+{\left({\displaystyle \frac{4}{3}}\right)}^2{\displaystyle \frac{1}{9}}={\displaystyle \frac{1}{3}}\left(1-{\displaystyle \frac{1}{3}}\right).$$

(50)

The negative contextual value associated with a rare outcome of 3 describes the fluctuations in terms of sudden deviations from a value much closer to the average than the truth values of zero and one. Both the values and the distributions of quantum fluctuations depend on the measurement context, even though their total magnitude is completely detemined by the initial state.

The outcome $S2$ has an average value of $P\left(S2\right)=1/2$, which corresponds to a maximal uncertainty of $1/4$ in any context. Since the outcome $\left|2\right.〉$ is orthogonal to $\left|S2\right.〉$, an eigenvalue of zero is obtained with a probability of $4/9$. The remaining values are $3/4$ with a probability of $4/9$ and $3/2$ with a probability of $1/9$. The fluctuations of $W\left(S2\right)$ are given by

$$\Delta W_{S2}^2={\left({\displaystyle \frac{1}{2}}\right)}^2{\displaystyle \frac{4}{9}}+{\left({\displaystyle \frac{1}{4}}\right)}^2{\displaystyle \frac{4}{9}}+{\left(1\right)}^2{\displaystyle \frac{1}{9}}={\displaystyle \frac{1}{2}}\left(1-{\displaystyle \frac{1}{2}}\right).$$

(51)

Here too, an extremal value is obtained for the low probability outcome. This time, the fluctuation is up toward an outcome value greater than one.

The lowest average value is that of $P2$, with only $P\left(P2\right)=1/6$. Similar to the statistics of f, there are only two possible outcome values of $P2$ in the context $\{\left|1\right.〉,\left|2\right.〉,\left|3\right.〉\}$. A value of $1/2$ is obtained with a probability of $5/9$, and a value of $-1/4$ is obtained with a probability of $4/9$. The fluctuations are

$$\Delta W_{P2}^2={\left({\displaystyle \frac{1}{3}}\right)}^2{\displaystyle \frac{5}{9}}+{\left({\displaystyle \frac{5}{12}}\right)}^2{\displaystyle \frac{4}{9}}={\displaystyle \frac{1}{6}}\left(1-{\displaystyle \frac{1}{6}}\right).$$

(52)

Here, the effect of the context on the fluctuations is opposite to that observed for f. Since the low probability result 3 and the high probability result 2 have the same value, the probabilities for the two possible values are very similar. Due to this absence of rare events, the fluctuations are almost equally distributed over the different outcomes. Even the negative value of $-1/4$ is closer to the average of $1/6$ than the eigenvalue of 1 obtained in the eigencontext.

The relations between different measurement contexts used for quantum state reconstructions cannot be interpreted by statistical arguments alone. Pure states represent deterministic conditions that can be used to identify contextual values for unobserved outcomes by combining each measurement outcome with the initial conditions represented by the quantum state. Weak values describe this deterministic relation between measurement contexts, explaining in detail how each measurement resolves the uncertainties of measurement outcomes from different contexts in a consistent manner. Inequality violations and quantum paradoxes can all be explained as a natural consequence of this well-defined dependence of the physical properties on the specific measurement context.

8. Discussion and Outlook

The relations between five contexts in a three-dimensional Hilbert space can be derived using suitable non-orthogonal expansions of quantum states. As the analysis in this paper has shown, the structure of contextuality can be separated from the statistics of Kirkwood–Dirac distributions by identifying relations between different KD terms that limit the number of independent elements in the Kirkwood–Dirac distribution to five. At the same time, it is possible to relate the KD terms to a simultaneous representation of all five contexts, where a total of eleven KD terms represent the eleven classical paths through the five contexts. The importance of representing more than two contexts in a single statistical expression cannot be stressed enough. Quantum phenomena are difficult to understand, because we usually look at only one context at a time. Correlations between two contexts often appear to be sufficiently ambiguous to insert a wide variety of interpretations into the gaps. Two contexts are not sufficient for a description of deterministic relations, since there are no conditions that these relations must satisfy. The cyclic relation between the five contexts shown in Figure 1 and Figure 2 solves this problem. We can then take a look at the differences between the classical determinism of joint realities and the quantum mechanical relations expressed by operators and state vectors.

Each of the eleven KD terms in the five-context scenario contributes its probability to exactly one outcome in each context. The relation between two different contexts can be described by a conditional distribution, where a different distribution of context b is associated with each outcome in context a. Negative contextual values ensure that the uncertainties of the initial quantum state can be explained by observable fluctuations in the values of b for different outcomes of a. This result suggests that the contextual values $W\left(b\right|a)$ describe conditional realities of b associated with the initial state and the outcomes of a of the measurement context. It is important to remember that eigenvalues cannot be observed unless a strong measurement is actually performed [48]. This means that there is no information about the outcomes of b available anywhere in the universe whenever the measurement of a is conducted instead. KD terms do not represent joint realities, since the outcomes in the argument cannot be obtained jointly. Instead, the KD terms characterize the relation between different measurements and the deterministic relations between physical properties that determine them.

The results presented above can be used to settle a number of important questions regarding the foundations of quantum mechanics. First, they address the physics of quantum states and their representations. For each measurement context, a quantum state provides the correct probability distribution. However, the quantum state also defines the context dependence of physical properties by implicitly introducing deterministic conditions. In the end, quantum states represent the past of a system in a context-independent manner. This means that quantum states neither describe the physical reality of the system nor are an ensemble of possible physical realities. Quantum states without measurements are fundamentally incomplete.

Second, the results address causality and determinism. The relation between different measurement contexts is fully determined for any pure state input. These relations are faithfully represented by the weak values that define contextual outcome values for each measurement context. It may be interesting to consider what the origin of the fluctuations in a pure state are. Since pure states cannot be divided into different possibilities, the physics of state preparation must erase all external information that could distinguish between different possibilities. A close analysis of interaction dynamics suggests that this problem combines aspects of entanglement with measurement theory [48].

Most importantly, the careful analysis of relations between different measurement contexts and their observable statistics confronts us with the difficulty of distinguishing between description and fact. It is clearly problematic that we identify quantum state components with measurement outcomes, regardless of whether the measurement actually happens or not. The temptation of fantasizing an imagined reality into such expressions is probably greater than we realize. The idea that a mathematical representation is an image of the object can only be maintained if the representation stays close to the actual observations recorded in an experiment. The original formulation of quantum mechanics strays too far into the realm of formal abstraction to permit such a naive identification. Hopefully, the present analysis of the relation between different quantum phenomena will pave the way toward a better practical understanding of the theory.