Nonparametric Causal Structure Learning in High Dimensions

Abstract

The PC and FCI algorithms are popular constraint-based methods for learning the structure of directed acyclic graphs (DAGs) in the absence and presence of latent and selection variables, respectively. These algorithms (and their order-independent variants, PC-stable and FCI-stable) have been shown to be consistent for learning sparse high-dimensional DAGs based on partial correlations. However, inferring conditional independences from partial correlations is valid if the data are jointly Gaussian or generated from a linear structural equation model—an assumption that may be violated in many applications. To broaden the scope of high-dimensional causal structure learning, we propose nonparametric variants of the PC-stable and FCI-stable algorithms that employ the conditional distance covariance (CdCov) to test for conditional independence relationships. As the key theoretical contribution, we prove that the high-dimensional consistency of the PC-stable and FCI-stable algorithms carry over to general distributions over DAGs when we implement CdCov-based nonparametric tests for conditional independence. Numerical studies demonstrate that our proposed algorithms perform nearly as good as the PC-stable and FCI-stable for Gaussian distributions, and offer advantages in non-Gaussian graphical models.

1. Introduction

Directed acyclic graphs (DAGs) are commonly used to represent causal relationships among random variables [1,2,3]. The PC algorithm [3] is the most popular constraint-based method for learning DAGs from observational data under the assumption of causal sufficiency, i.e., when there are no unmeasured common causes and no selection variables. It first estimates the skeleton of a DAG by recursively performing a sequence of conditional independence tests, and then uses the information from the conditional independence relations to partially orient the edges, resulting in a completed partially directed acyclic graph (CPDAG). In Section 2, we provide a review of these and other notions commonly used in the graphical modeling literature that are relevant to our work. In addition, we refer to estimating the CPDAG as structure learning of the underlying DAG throughout the rest of the paper.

Observational studies often involve latent and selection variables, which complicate the causal structure learning problem. Ignoring such unmeasured variables can make the causal inference based on the PC algorithm erroneous; see, e.g., Section 1.2 in [4] for some illustrations. The Fast Causal Inference (FCI) algorithm and its variants [3,4,5,6] utilize similar strategies as the PC algorithm to learn the DAG structure in the presence of latent and selection variables.

Both PC and FCI algorithms adopt a hierarchical search strategy—they recursively perform conditional independence tests given subsets of increasingly larger cardinalities in some appropriate search pool. The PC algorithm is usually order-dependent, in the sense that its output depends on the order in which pairs of adjacent vertices and subsets of their adjacency sets are considered. The FCI algorithm suffers from a similar limitation. To overcome this limitation, Ref. [7] proposed two variants of the PC and FCI algorithms, namely the PC-stable and FCI-stable algorithms that resolve the order dependence at different stages of the algorithms.

In general, testing for conditional independence is a problem of central importance in the causal structure learning. The literature on the PC and FCI algorithms predominantly uses partial correlations to infer conditional independence relations. It is well-known that the characterization of conditional independence by partial correlations, or, in other words, equivalence between conditional independence and zero partial correlations only holds for multivariate normal random variables. Therefore, the high-dimensional consistency results for the PC and FCI algorithms [4,8] are limited to Gaussian graphical models, where the nodes correspond to random variables with a joint Gaussian distribution. Although the Gaussian graphical model is the standard parametric model for continuous data, it may not hold in many real data applications. Although this limitation can be somewhat relaxed by considering linear structural equation models (SEMs) with general noise distributions [9], linear SEMs and joint Gaussianity are essentially equivalent [10]. Moreover, neither approach is appropriate when the observations are categorical, discrete, or are supported on a subset of the real line. In Section 4.3, for example, we present a real application where all the observed variables are categorical, and therefore far from being Gaussian. As an improvement, ref. [11] used rank-based partial correlations to test for conditional independence relations, showing that the high-dimensional consistency of the PC algorithm holds for a broader class of Gaussian copula models. Some nonparametric versions of the PC algorithm have been also proposed in the literature via kernel-based tests for conditional independence [12,13]; however, they lack theoretical justifications of the correctness of the algorithms, and are not studied in high dimensions.

This work aims to broaden the applicability of the PC-stable and FCI-stable algorithms to general distributions by employing a nonparametric test for conditional independence relationships. To this end, we utilize recent developments on dependence metrics that quantify nonlinear and non-monotone dependence between multivariate random variables. More specifically, our work builds on the idea of distance covariance (dCov) proposed by [14] and its extension to conditional distance covariance (CdCov) by [15] as a nonparametric measure of nonlinear and non-monotone conditional independence between two random vectors of arbitrary dimensions given a third. Utilizing this flexibility, we use the conditional distance covariance (CdCov) to test for conditional independence relationships in the sample versions of the PC-stable and FCI-stable algorithms. The resulting algorithms—which, for distinction, are termed nonPC and nonFCI—facilitate causal structure learning from general distributions over DAGs and are shown to be consistent in sparse high-dimensional settings. We establish the consistency of the proposed algorithms using some moment and tail conditions on the variables, without requiring strict distributional assumptions. To our knowledge, the proposed generalizations of PC/PC-stable or the FCI/FCI-stable algorithms provide the first general nonparametric framework for causal structure learning with theoretical guarantees in high dimensions.

The rest of the paper is organized as follows: In Section 2, we review the relevant background, including preliminaries on graphical modeling (Section 2.1), an outline of the PC-stable and FCI-stable algorithms (Section 2.2) and a brief overview of dCov and CdCov (Section 2.3). The nonparametric version of the PC-stable algorithm is presented in Section 3.1. As a key contribution of the paper, we establish that the algorithm consistently estimates the skeleton and the equivalence class of the underlying sparse high-dimensional DAG in a general nonparametric framework. We then present the nonparametric version of the FCI-stable algorithm in Section 3.2 and establish its consistency in sparse high-dimensional settings. As the FCI involves the adjacency search of the PC algorithm, any improvement on the PC/PC-stable directly carries over to the FCI/FCI-stable as well. In Section 4, we compare the performances of our algorithms with the PC-stable and FCI-stable using both simulated datasets (involving both Gaussian and non-Gaussian examples), as well as a real dataset. These numerical studies clearly demonstrate that nonPC and nonFCI algorithms are comparable with PC-stable and FCI-stable for Gaussian data and offer improvements for non-Gaussian data.

2. Background

2.1. Preliminaries on Graphical Modeling

We start with introducing some necessary terminologies and background information. Our notations and terminologies follow standard conventions in graphical modeling (see, e.g., [3]). A graph $\mathcal{G}=(V,E)$ consists of a vertex set $V=\{1,\cdots ,p\}$ and an edge set $E\subseteq V\times V$. In a graphical model, the vertices or nodes are associated with random variables $X_a$ for $1\le a\le p$. Throughout, we index the nodes by the corresponding random variables. We also allow the edge set E of the graph $\mathcal{G}$ to contain (a subset of) the following six types of edges: → (directed), ↔ (bidirected), − (undirected), $\circ -\circ$ (nondirected), $\circ -$ (partially undirected) and $\circ \to$ (partially directed). The endpoints of an edge are called marks, which can be tails, arrowheads or circles. A “∘” at the end of an edge indicates it is not known whether an arrowhead should occur at that place. We use the symbol ‘★’ to denote an arbitrary edge mark; for example, the symbol $★\to$ represents an edge of the type →, ↔ or $\circ \to$ in the graph. A mixed graph is a graph containing directed, bidirected and undirected edges. A graph containing only directed edges $(\to )$ is called a directed graph, one containing only undirected edges $(-)$ is called an undirected graph, and one containing directed and undirected edges is called a partially directed graph.

The adjacency set of a vertex $X_a$ in the graph $\mathcal{G}=(V,E)$, denoted $\mathrm{adj}(\mathcal{G},X_a)$, is the set of all vertices in V that are adjacent to $X_a$, or, in other words, are connected to $X_a$ by an edge. The degree of a vertex $X_a$, $\left|\mathrm{adj}\right(\mathcal{G},X_a\left)\right|$, is defined as the number of vertices adjacent to it. A graph is complete if all pairs of vertices in the graph are adjacent. A vertex $X_b\in \mathrm{adj}(\mathcal{G},X_a)$ is called a parent of $X_a$ if $X_b\to X_a$, a child of $X_a$ if $X_a\to X_b$  and a neighbor of $X_a$ if $X_a-X_b$. The skeleton of the graph $\mathcal{G}$ is the undirected graph obtained by replacing all the edges of $\mathcal{G}$ by undirected edges, in other words, ignoring all the edge orientations. Three vertices $\langle X_a,X_b,X_c\rangle$ are called an unshielded triple if $X_a$ and $X_b$ are adjacent, $X_b$ and $X_c$ are adjacent, but $X_a$ and $X_c$ are not adjacent. A path is a sequence of distinct adjacent vertices. A node $X_a$ is an ancestor of its descendent$X_b$, if $\mathcal{G}$ contains a directed path $X_a\to \cdots \to X_b$. A non-endpoint vertex $X_a$ on a path is called a collider on the path if both the edges preceding and succeeding it have an arrowhead at $X_a$, or, in other words, the path contains $★\to X_a\leftarrow ★$. An unshielded triple $\langle X_a,X_b,X_c\rangle$ is called a v-structure if $X_b$ is a collider on the path $\langle X_a,X_b,X_c\rangle$.

A cycle occurs in a graph when there is a path from $X_a$ to $X_b$, and $X_a$ and $X_b$ are adjacent. A directed path from $X_a$ to $X_b$ forms a directed cycle together with the edge $X_b\to X_a$, and it forms an almost directed cycle together with the edge $X_b\leftrightarrow X_a$. Three vertices that form a cycle are called a triangle. A directed acyclic graph (DAG) is a directed graph that does not contain any cycle. A DAG entails conditional independence relationships via a graphical criterion called d-separation (Section 1.2.3 in [16]). Two vertices $X_a$ and $X_b$ that are not adjacent in a DAG $\mathcal{G}$ are d-separated in $\mathcal{G}$ by a subset $X_S\subseteq V\setminus \{X_a,X_b\}$. A probability distribution P on ${\mathbb{R}}^p$ is said to be faithful with respect to the DAG $\mathcal{G}$ if the conditional independence relationships in P can be inferred from $\mathcal{G}$ using d-separation and vice versa; in other words, $X_a⫫X_b|X_S$ if and only if $X_a$ and $X_b$ are d-separated in $\mathcal{G}$ by $X_S$.

A graph that is both (partially) directed and acyclic is called a partially directed acyclic graph (PDAG). DAGs that encode the same set of conditional independence relations form a Markov equivalence class [17]. Two DAGs belong to the same Markov equivalence class if and only if they have the same skeleton and the same v-structures. A Markov equivalence class of DAGs can be uniquely represented by a completed partially directed acyclic graph (CPDAG), which is a PDAG that satisfies the following: (i) $X_a\to X_b$ in the CPDAG if $X_a\to X_b$ in every DAG in the Markov equivalence class, and (ii) $X_a-X_b$ in the CPDAG if the Markov equivalence class contains a DAG in which $X_a\to X_b$ as well as a DAG in which $X_a\leftarrow X_b$.

2.2. The PC-Stable and FCI-Stable Algorithms

In this section, we provide an outline of the PC/PC-stable and FCI/FCI-stable algorithms. Estimation of the CPDAG by the PC algorithm involves two steps: (1) estimation of the skeleton and separating sets (also called the adjacency search step); and (2) partial orientation of edges; see Algorithms 1 and 2 in [8] for details.

Intuitively, the PC algorithm works as follows. In the first step (the adjacency search step), the algorithm starts with a complete undirected graph Then, for conditioning sets of increasing cardinality, $k=0,1,\dots$, the algorithm removed an edge $X_a-X_b$ if $X_a$ and $X_b$ are conditionally independent given a subset S of size k chosen among the current neighbors of nodes a and b. This process continues up to the order $q-1$, where q is the maximum degree of the underlying DAG. By searching over the neighboring nodes, the algorithm is adaptive and can efficiently infer sparse high-dimensional DAGs, where the sparsity is characterized by the maximum node degree, q.

In the presence of latent and selection variables, one needs a generalization of an DAG, called a maximal ancestral graph (MAG). A mixed graph is called an ancestral graph if it contains no directed or almost directed cycles and no subgraph of the type $X_a-X_b\leftarrow ★X_c$. DAGs form a subset of ancestral graphs. A MAG is an ancestral graph in which every missing edge corresponds to a conditional independence relationship via the m-separation criterion [18], a generalization of the notion of d-separation. Multiple MAGs may represent the same set of conditional independence relations. Such MAGs form a Markov equivalence class which can be represented by a partial ancestral graph (PAG) [19]; see [18] for additional details.

Under the faithfulness assumption, the Markov equivalence class of a DAG with latent and selection variables can be learned using the FCI algorithm (e.g., Algorithm 3.1 in [4]), which is a modification of the PC algorithm. The FCI algorithm first employs the adjacency search of the PC algorithm, and then performs additional conditional independence queries because of the presence of latent variables followed by partial orientation of the edges, resulting in an estimated PAG. The FCI algorithm adopts the same hierarchical search strategy as the PC algorithm: It starts with a complete undirected graph and recursively removes edges via conditional independence queries given subsets of increasingly larger cardinalities in some appropriate search pool.

The PC algorithm is usually order-dependent, in the sense that its output depends on the order in which pairs of adjacent vertices and subsets of their adjacency sets are considered. The FCI algorithm suffers from a similar limitation, as it shares the adjacency search step of the PC algorithm as its first step. To overcome this limitation, ref. [7] proposed variants of the PC and FCI algorithms, namely the PC-stable and FCI-stable algorithms that resolve the order dependence at different stages of the algorithms. The basic difference between the PC algorithm and the PC-stable algorithm is that, in the adjacency search step, the latter computes and stores the adjacency sets of all the variables after each new cardinality, $k=0,1,\dots$, of the conditioning sets. These stored adjacency sets are then used to search for conditioning sets of this given size k. As a consequence, the removal of an edge no longer affects which conditional independence relations need to be checked for other pairs of variables at this given size of the conditioning sets.

We would refer the reader to Appendix A, where we provide in full detail the pseudocodes of the oracle versions of the PC-stable and FCI-stable algorithms. In the oracle versions of the algorithms, it is assumed that perfect knowledge is available about all the necessary conditional independence relations. As such, conditional independence relations are not estimated from data. Of course, this perfect knowledge is not available in practice. Sample versions of the PC-stable and FCI-stable algorithms can be obtained by replacing the conditional independence queries by a suitable test for conditional independence at some pre-specified level. For example, if the variables are jointly Gaussian, one can test for zero partial correlations (see, e.g., [8]). The next subsection is devoted to discussions on nonparametric tests for independence and conditional independence.

2.3. Distance Covariance and Conditional Distance Covariance

We start by describing the notation used throughout the paper. We denote by ${\parallel ·\parallel }_p$ the Euclidean norm of ${\mathbb{R}}^p$ and use $\parallel ·\parallel$ when the dimension is clear from the context. We use $X⫫Y$ to denote the independence of X and Y and use ${\mathbb{E}}_U$ to denote expectation with respect to the probability distribution of the random variable U. For any set S, we denote its cardinality by $\left|S\right|$.

We use the usual asymptotic notation, ‘O’ and ‘o’, as well as their probabilistic counterparts, $O_p$ and $o_p$, which denote stochastic boundedness and convergence in probability, respectively. For two sequences of real numbers ${\left\{a_n\right\}}_{n=1}^{\infty }$ and ${\left\{b_n\right\}}_{n=1}^{\infty }$, $a_n\asymp b_n$ if and only if $a_n/b_n=O\left(1\right)$ and $b_n/a_n=O\left(1\right)$ as $n\to \infty$. We use the symbol “$a\lesssim b$” to indicate that $a\le Cb$  for some constant $C>0$. For a matrix $A={\left(a_{kl}\right)}_{k,l=1}^n\in {\mathbb{R}}^{n\times n}$, we denote its determinant by $\left|A\right|$ and define its  $\mathcal{U}$-centered version $\tilde{A}={\left({\tilde{a}}_{kl}\right)}_{k,l=1}^n$ as

$$\begin{array}{c}\hfill {\tilde{a}}_{kl}=\left\{\begin{array}{cc}{\displaystyle a_{kl}-\frac{1}{n-2}\sum _{j=1}^n{a}_{kj}-\frac{1}{n-2}\sum _{i=1}^n{a}_{il}+\frac{1}{(n-1)(n-2)}\sum _{i,j=1}^n{a}_{ij},}\hfill & k\ne l,\hfill \\ 0,\hfill & k=l,\hfill \end{array}\right.\end{array}$$

(1)

for $k,l=1,\cdots ,n$. We denote the indicator function of any set A by $\mathbf{1}\left(A\right)$. Finally, we denote the integer part of $a\in \mathbb{R}$ by $\lfloor a\rfloor$.

Ref. [14], in their seminal paper, introduced the notion of distance covariance (dCov, henceforth) to quantify nonlinear and non-monotone dependence between two random vectors of arbitrary dimensions. Consider two random vectors $X\in {\mathbb{R}}^p$ and $Y\in {\mathbb{R}}^q$ with ${\mathbb{E}\parallel X\parallel }_p<\infty$ and ${\mathbb{E}\parallel Y\parallel }_q<\infty$. The distance covariance between X and Y is defined as the positive square root of

$$\begin{array}{c}\hfill {\mathrm{dCov}}^2(X,Y)=\frac{1}{c_p{c}_q}{\int }_{{\mathbb{R}}^{p+q}}\frac{|f_{X,Y}(t,s)-f_X\left(t\right)f_Y{\left(s\right)|}^2}{{\parallel t\parallel }_p^{1+p}{\parallel s\parallel }_q^{1+q}}dtds\end{array}$$

where $f_X$, $f_Y$ and $f_{X,Y}$ are the individual and joint characteristic functions of X and Y, respectively, and $c_p={\pi }^{(1+p)/2}/\Gamma ((1+p)/2)$ is a constant with $\Gamma (·)$ being the complete gamma function.

The key feature of dCov is that it completely characterizes the independence between two random vectors, or in other words $\mathrm{dCov}(X,Y)=0$ if and only if $X⫫Y$. According to Remark 3 in [14], dCov can be equivalently expressed as

$$\begin{array}{cc}\hfill {\mathrm{dCov}}^2(X,Y)=& \mathbb{E}\parallel X-X^{\prime }{\parallel }_p\parallel Y-Y^{\prime }{\parallel }_q+\mathbb{E}\parallel X-X^{\prime }{\parallel }_p\mathbb{E}{\parallel Y-Y^{\prime }\parallel }_q\hfill \\ & -2\mathbb{E}\parallel X-X^{\prime }{\parallel }_p{\parallel Y-Y^{″}\parallel }_q.\hfill \end{array}$$

This alternate expression comes handy in constructing V or U-statistic type estimators for the quantity. For an observed random sample ${(X_i,Y_i)}_{i=1}^n$ from the joint distribution of X and Y, define the distance matrices $d^X={\left(d_{ij}^X\right)}_{i,j=1}^n$ and $d^Y={\left(d_{ij}^Y\right)}_{i,j=1}^n\in {\mathbb{R}}^{n\times n}$, where $d_{ij}^X:={\parallel X_i-X_j\parallel }_p$ and $d_{ij}^Y:={\parallel Y_i-Y_j\parallel }_q$. Following the $\mathcal{U}$-centering idea in [20], an unbiased U-statistic type estimator of ${\mathrm{dCov}}^2(X,Y)$ can be expressed as

$$\begin{array}{cc}\hfill {\mathrm{dCov}}_n^2(X,Y)& {\displaystyle :=({\tilde{d}}^X·{\tilde{d}}^Y):=\frac{1}{n(n-3)}\sum _{i\ne j}{\tilde{d}}_{ij}^X{\tilde{d}}_{ij}^Y,}\hfill \end{array}$$

(2)

where ${\tilde{d}}^X={\left({\tilde{d}}_{ij}^X\right)}_{i,j=1}^n$  and   ${\tilde{d}}^Y={\left({\tilde{d}}_{ij}^Y\right)}_{i,j=1}^n$ are the  $\mathcal{U}$-centered versions of the matrices $d^X$ and $d^Y$, respectively, as defined in (1).

Ref. [15] generalized the notion of dCov and introduced the conditional distance covariance (CdCov, henceforth) as a measure of conditional dependence between two random vectors of arbitrary dimensions given a third. CdCov essentially replaces the characteristic functions used in the definition of dCov by conditional characteristic functions. Consider a third random vector $Z\in {\mathbb{R}}^r$ with $\mathbb{E}(\parallel X{\parallel }_p+\parallel Y{\parallel }_q\mid Z)<\infty$. Denote by $f_{X,Y|Z}$ the conditional joint characteristic function of X and Y given Z, and by $f_{X|Z}$ and $f_{Y|Z}$ the conditional marginal characteristic functions of X and Y given Z, respectively. Then, CdCov between X and Y given Z is defined as the positive square root of

$$\begin{array}{c}\hfill {\mathrm{CdCov}}^2(X,Y|Z)=\frac{1}{c_p{c}_q}{\int }_{{\mathbb{R}}^{p+q}}\frac{|f_{X,Y|Z}(t,s)-f_{X|Z}\left(t\right)f_{Y|Z}{\left(s\right)|}^2}{{\parallel t\parallel }_p^{1+p}{\parallel s\parallel }_q^{1+q}}dtds.\end{array}$$

The key feature of CdCov is that $\mathrm{CdCov}(X,Y|Z)=0$ almost surely if and only if $X⫫Y|Z$, which is quite straightforward to see from the definition.

Similar to dCov, an equivalent alternative expression can be established for CdCov that avoids complicated integrations involving conditional characteristic functions. Let ${\{W_i=(X_i,Y_i,Z_i)\}}_{i=1}^n$ be an i.i.d. sample from the joint distribution of $W:=(X,Y,Z)$. Define $d_{ijkl}:=\left(d_{ij}^X+d_{kl}^X-d_{ik}^X-d_{jl}^X\right)\left(d_{ij}^Y+d_{kl}^Y-d_{ik}^Y-d_{jl}^Y\right)$, which is not symmetric with respect to $\{i,j,k,l\}$, and therefore necessitates defining the following symmetric form: $d_{ijkl}^S:=d_{ijkl}+d_{ijlk}+d_{ilkj}$. Lemma 1 in [15] establishes an equivalent representation of ${\mathrm{CdCov}}^2(X,Y|Z=z)$ as

$$\begin{array}{c}\hfill {\mathrm{CdCov}}^2(X,Y|Z=z)=\frac{1}{12}\mathbb{E}\left[d_{1234}^S|Z_1=z,Z_2=z,Z_3=z,Z_4=z\right].\end{array}$$

(3)

Remark 1.

In a recent work, [21] explore the connection between conditional independence measures induced by distances on a metric space and reproducing kernels associated with a reproducing kernel Hilbert space (RKHS). They generalize CdCov to arbitrary metric spaces of negative type—termed generalized CdCov (gCdCov)—and develop a kernel-based measure of conditional independence, namely the Hilbert–Schmidt conditional independence criterion (HSCIC). Theorem 1 in their paper establishes an equivalence between gCdCov and HSCIC, or, in other words, between distance and kernel-based measures of conditional independence.

For $w\in {\mathbb{R}}^r$, let $K_H\left(w\right):={\left|H\right|}^{-1}K\left(H^{-1}w\right)$ be a kernel function, where H is the diagonal matrix $\mathrm{diag}(h,\cdots ,h)$ determined by a bandwidth parameter h. $K_H$ is typically considered to be the Gaussian kernel $K_H\left(w\right)={\left(2\pi \right)}^{-\frac{r}{2}}{\left|H\right|}^{-1}exp\left(-\frac{1}{2}{w}^T{H}^{-2}w\right)$, where $w\in {\mathbb{R}}^r$.

Let $K_{iu}:=K_H(Z_i-Z_u)={\left|H\right|}^{-1}K\left(H^{-1}(Z_i-Z_u)\right)$ and $K_i\left(Z\right):=K_H(Z-Z_i)$ for $1\le i,u\le n$. Then, by virtue of the equivalent representation of CdCov in (3), a V-statistic type estimator of ${\mathrm{CdCov}}^2(X,Y|Z)$ can be constructed as

$$\begin{array}{c}\hfill {\displaystyle {\mathrm{CdCov}}_n^2(X,Y|Z):=\sum _{i,j,k,l}\frac{K_i\left(Z\right)K_j\left(Z\right)K_k\left(Z\right)K_l\left(Z\right)}{12{\left({\sum }_{i=1}^n{K}_i\left(Z\right)\right)}^4}{d}_{ijkl}^S.}\end{array}$$

(4)

Under certain regularity conditions, Theorem 4 in [15] shows that, conditioned on Z, ${\mathrm{CdCov}}_n^2(X,Y|Z)\stackrel{P}{⟶}{\mathrm{CdCov}}^2(X,Y|Z)$ as $n\to \infty$.

3. Methodology and Theory

3.1. The Nonparametric PC Algorithm in High Dimensions

To obtain a measure of conditional independence between X and Y given Z that is free of Z, we define

$$\begin{array}{c}\hfill {\rho }_0^{*}(X,Y|Z):=\mathbb{E}\left[{\mathrm{CdCov}}_n^2(X,Y|Z)\right].\end{array}$$

(5)

Clearly, ${\rho }_0^{*}(X,Y|Z)=0$ if and only if $X⫫Y|Z$. Consider a plug-in estimate of ${\rho }_0^{*}(X,Y|Z)$ as

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill & {\displaystyle {\widehat{\rho }}^{*}(X,Y|Z):=\frac{1}{n}\sum _{u=1}^n{\mathrm{CdCov}}_n^2(X,Y|Z_u)=\frac{1}{n}\sum _{u=1}^n{\Delta }_{i,j,k,l;u}}\hfill \\ \hfill \mathrm{where}& {\displaystyle {\Delta }_{i,j,k,l;u}:=\sum _{i,j,k,l}\frac{K_{iu}{K}_{ju}{K}_{ku}{K}_{lu}}{12{\left({\sum }_{i=1}^n{K}_{iu}\right)}^4}{d}_{ijkl}^S.}\hfill \end{array}\end{array}$$

(6)

We reject $H_0:X⫫Y|Z$  vs  $H_A:X\cancel{⫫}Y|Z$ at level $\alpha \in (0,1)$ if  ${\widehat{\rho }}^{*}(X,Y|Z)>{\xi }_{\alpha }$, for a suitably chosen threshold ${\xi }_{\alpha }$. In Appendix A, we present a local bootstrap procedure for choosing ${\xi }_{\alpha }$ in practice, which is also used in our numerical studies. Henceforth, we will often denote ${\rho }_0^{*}(X,Y|Z)$ and ${\widehat{\rho }}^{*}(X,Y|Z)$ simply by ${\rho }_0^{*}$ and ${\widehat{\rho }}^{*}$ respectively for notational simplicity, whenever there is no confusion.

In view of the complete characterization of conditional independence by ${\rho }_0^{*}$, we propose testing for conditional independence relations nonparametrically in the sample version of the PC-stable algorithm based on ${\rho }_0^{*}$, rather than partial correlations. We coin the resulting algorithm the ‘nonPC’ algorithm, to emphasize that it is a nonparametric generalization of parametric PC-stable algorithms.

The oracle version of the first step of nonPC, or the skeleton estimation step, is exactly the same as that of the PC-stable algorithm (Algorithm A1 in Appendix A). The second step, which extends the skeleton estimated in the first step to a CPDAG (Algorithm A2 in Appendix A), is comprised of some purely deterministic rules for edge orientations, and is exactly the same for both the nonPC and PC-stable as well. The only difference lies in the implementation of the tests for conditional independence relationships in the sample versions of the first step. Specifically, we replace all the conditional independence queries in the first step by tests based on ${\rho }_0^{*}(X,Y|Z)$. At some pre-specified significance level $\alpha$, we infer that $X_a⫫X_b|X_S$ when ${\widehat{\rho }}^{*}(X_a,X_b|X_S)\le {\xi }_{n,\alpha }$, where $a,b\in V$ and $S\subseteq V$, $\left|S\right|\ne \varphi$. When $\left|S\right|=\varphi$, ${\widehat{\rho }}^{*}(X_a,X_b|X_S)={\mathrm{dCov}}_n^2(X_a,X_b)$ and ${\rho }_0^{*}(X,Y|Z)={\mathrm{dCov}}^2(X,Y)$. The critical value ${\xi }_{n,\alpha }$ in this case is obtained by a bootstrap procedure (see, e.g., Section 4 in [22] with $d=2$).

Given that the equivalence between conditional independence and zero partial correlations only holds for multivariate normal random variables, our generalization broadens the scope of applicability of causal structure learning by the PC/PC-stable algorithm to general distributions over DAGs. This nonparametric approach is thus a natural extension of Gaussian and Gaussian copula models. It enables capturing nonlinear and non-monotone conditional dependence relationships among the variables, which partial correlations fail to detect.

Next, we establish theoretical guarantees on the correctness of the nonPC algorithm in learning the true underlying causal structure in sparse high-dimensional settings. Our consistency results only require mild moment and tail conditions on the set of variables, without making any strict distributional assumptions. Denote by $m_p$ the maximum cardinality of the conditioning sets considered in the adjacency search step of the PC-stable algorithm. Clearly, $m_p\le q$, where $q:={max}_{1\le a\le p}\left|\mathrm{adj}(\mathcal{G},a)\right|$ is the maximum degree of the DAG $\mathcal{G}$. For a fixed pair of nodes $a,b\in V$, the conditioning sets considered in the adjacency search step are elements of $J_{a,b}^{m_p}:=\{S\subseteq V\setminus \{a,b\}:|S|\le m_p\}$.

We first establish a concentration inequality that gives the rate at which the absolute difference of ${\rho }_0^{*}(X_a,X_b|X_S)$ and its plug-in estimate ${\widehat{\rho }}^{*}(X_a,X_b|X_S)$ decays to zero, for any fixed pair of nodes a and $b\in V$ and a fixed conditioning set S. Towards that, we impose the following regularity conditions.

(A1)

There exists $s_0>0$ such that, for $0\le s<s_0$, ${\displaystyle \underset{p}{sup}\underset{1\le a\le p}{max}\mathbb{E}exp\left(s{X}_a^2\right)<\infty }$.

(A2)

The kernel function $K(·)$ is non-negative and uniformly bounded over its support.

Condition (A1) imposes a sub-exponential tail bound on the squares of the random variables. This is a quite commonly used condition, for example, in the high-dimensional feature screening literature (see, for example, [23]). Condition (A2) is a mild condition on the kernel function $K(·)$ that is guaranteed by many commonly used kernels, including the Gaussian kernel. Under conditions (A1) and (A2), the next result shows that the plug-in estimate ${\widehat{\rho }}^{*}(X_a,X_b|X_S)$ converges in probability to its population counterpart ${\rho }_0^{*}(X_a,X_b|X_S)$ exponentially fast.

Theorem 1.

Under conditions (A1) and (A2), for any  $ϵ>0$, there exist positive constants A, B and  $\gamma \in (0,1/4)$  such that

$$\begin{array}{c}\hfill \mathbb{P}\left(|{\widehat{\rho }}^{*}(X_a,X_b|X_S)-{\rho }_0^{*}(X_a,X_b|X_S)|>ϵ\right)\le O\left(2exp\left(-A{n}^{1-2\gamma }{ϵ}^2\right)+n^{4}exp\left(-B{n}^{\gamma }\right)\right).\end{array}$$

The proof of Theorem 1 is long and somewhat technical; it is thus relegated to Appendix B. Theorem 1 serves as the main building block towards establishing the consistency of the nonPC algorithm in sparse high-dimensional settings.

For notational convenience, henceforth, we denote ${\rho }_0^{*}(X_a,X_b|X_S)$ and ${\widehat{\rho }}^{*}(X_a,X_b|X_S)$ by ${\rho }_{0;ab|S}^{*}$ and ${\widehat{\rho }}_{ab|S}^{*}$, respectively. In Theorem 2 below, we establish a uniform bound for the errors in inferring conditional independence relationships using the ${\rho }_0^{*}$-based test in the skeleton estimation step of the sample version of the nonPC algorithm.

Theorem 2.

Under conditions (A1) and (A2), for any  $ϵ>0$, there exist positive constants A, B and  $\gamma \in (0,1/4)$  such that

$$\begin{array}{cc}\hfill \underset{\begin{array}{c}a,b\in V\\ S\in J_{a,b}^{m_p}\end{array}}{sup}\mathbb{P}\left(|{\widehat{\rho }}_{ab|S}^{*}-{\rho }_{0;ab|S}^{*}|>ϵ\right)& \le \mathbb{P}(\underset{\begin{array}{c}a,b\in V\\ S\in J_{a,b}^{m_p}\end{array}}{sup}|{\widehat{\rho }}_{ab|S}^{*}-{\rho }_{0;ab|S}^{*}|>ϵ)\hfill \\ \hfill & \le O\left(p^{m_p+2}\left[2exp\left(-A{n}^{1-2\gamma }{ϵ}^2\right)+n^{4}exp\left(-B{n}^{\gamma }\right)\right]\right).\hfill \end{array}$$

(7)

Next, we turn to proving the consistency of the nonPC algorithm in the high-dimensional setting where the dimension p can be much larger than the sample size n, but the DAG is considered to be sparse. We impose the following regularity conditions, which are similar to the assumptions imposed in Section 3.1 of [8] in order to prove the consistency of the PC algorithm for Gaussian graphical models. We let the number of variables p grow with the sample size n and consider $p=p_n$, and also the DAG $\mathcal{G}={\mathcal{G}}_n:=(V_n,E_n)$ and the distribution $P=P_n$.

(A3)

The dimension $p_n$ grows at a rate such that the right-hand side of (7) tends to zero as $n\to \infty$. In particular, this is satisfied when $p_n=O\left(n^r\right)$ for any $0\le r<\infty$.

(A4)

The maximum degree of the DAG ${\mathcal{G}}_n$, denoted by $q_n:={max}_{1\le a\le p_n}\left|adj({\mathcal{G}}_n,a)\right|$, grows at the rate of $O\left(n^{1-b}\right)$, where $0<b\le 1$.

(A5)

The distribution $P_n$ is faithful to the DAG  ${\mathcal{G}}_n$  for all n. In other words, for any $a,b\in V_n$ and $S\in J_{a,b}^{m_{p_n}},$

$$\begin{array}{c}\hfill X_a\mathrm{and}{X}_b\mathrm{are}\mathrm{d}-\mathrm{separated}\mathrm{by}{X}_S⟺X_a⫫X_b|X_S⟺{\rho }_{0;ab|S}^{*}=0.\end{array}$$

Moreover, ${\rho }_{0;ab|S}^{*}$ values are uniformly bounded both from above and below. Formally,

$$\begin{array}{cc}\hfill C_{min}:& =\underset{\begin{array}{c}a,b\in V_n\\ S\in J_{a,b}^{m_{p_n}}\\ {\rho }_{0;ab|S}^{*}\ne 0\end{array}}{inf}{\rho }_{0;ab|S}^{*}\ge {\lambda }_{min}{\lambda }_{min}^{-1}=O\left(n^v\right)\hfill \\ \hfill \mathrm{and}{C}_{max}:& =\underset{\begin{array}{c}a,b\in V_n\\ S\in J_{a,b}^{m_{p_n}}\end{array}}{sup}{\rho }_{0;ab|S}^{*}\le {\lambda }_{max}\hfill \end{array}$$

where ${\lambda }_{max}$ is a positive constant and $0<v<1/4$.

Condition (A3) allows the dimension to grow at any arbitrary polynomial rate of the sample size. Condition (A4) is a sparsity assumption on the underlying true DAG, allowing the maximum degree of the DAG to also grow, but at a slower rate than n. Since $m_p\le q_n$, we also have $m_p=O\left(n^{1-b}\right)$. Finally, Condition (A5) is the strong faithfulness assumption (Definition 1.3 in [24]) on $P_n$ and is similar to condition (A4) in [8]. This essentially requires ${\rho }_{0;ab|S}^{*}$ to be bounded away from zero when the vertices $X_a$ and $X_b$ are not d-separated by $X_S$. It is worth noting that the faithfulness assumption alone is not enough to prove the consistency of the PC/PC-stable/nonPC algorithms in high-dimensional settings, and the more stringent strong faithfulness condition is required.

Remark 2.

For notational convenience, treat $X_a,X_b$ and $X_S$ as X, Y and Z, respectively, for any $a,b\in V_n$ and $S\in J_{a,b}^{m_{p_n}}$. From Equation (3), we have

$$\begin{array}{c}\hfill {\mathrm{CdCov}}^2(X,Y|Z)=\frac{1}{12}\mathbb{E}\left[d_{1234}^S|Z_1=Z,\cdots ,Z_4=Z\right],\end{array}$$

which implies

$$\begin{array}{c}\hfill {\rho }_0^{*}=\mathbb{E}\left[{\mathrm{CdCov}}^2(X,Y|Z)\right]=\frac{1}{12}\mathbb{E}\left[d_{1234}^S\right]=\frac{1}{12}\mathbb{E}\left[d_{1234}+d_{1243}+d_{1432}\right].\end{array}$$

Condition (A1) implies  ${\displaystyle \underset{p}{sup}\underset{1\le a\le p}{max}\mathbb{E}{X}_a^2<\infty }$. With this and the definition of $d_{ijkl}$ in Section 2.3, it follows from some simple algebra and the Cauchy–Schwarz inequality that  ${\rho }_0^{*}<\infty$. This provides a justification for the second part of Assumption (A5) that ${\displaystyle \underset{\begin{array}{c}a,b\in V_n\\ S\in J_{a,b}^{m_{p_n}}\end{array}}{sup}{\rho }_{0;ab|S}^{*}\le {\lambda }_{max}}$  for some positive constant ${\lambda }_{max}$.

The next theorem establishes that the nonPC algorithm consistently estimates the skeleton of a sparse high-dimensional DAG, thereby providing the necessary theoretical guarantees to our proposed methodology. It is worth noting that, in the sample version of the PC-stable and hence the nonPC algorithm, all the inference is done during the skeleton estimation step. The second step that involves appropriately orienting the edges of the estimated skeleton is purely deterministic (see Sections 4.2 and 4.3 in [7]). Therefore, to prove the consistency of the nonPC algorithm in estimating the equivalence class of the underlying true DAG, it is enough to prove the consistency of the estimated skeleton. We include the detailed proof of Theorem 3 in Appendix B.

Theorem 3.

Assume that Conditions (A1)–(A5) hold. Let ${\mathcal{G}}_{\mathrm{skel},n}$ be the true skeleton of the graph ${\mathcal{G}}_n$, and ${\widehat{\mathcal{G}}}_{\mathrm{skel},n}$ be the skeleton estimated by the nonPC algorithm. Then, as $n\to \infty$, $\mathbb{P}\left({\widehat{\mathcal{G}}}_{\mathrm{skel},n}={\mathcal{G}}_{\mathrm{skel},n}\right)\to 1$.

Remark 3.

In the proof of Theorem 3, we consider the threshold ${\xi }_{\alpha }$ to be of constant order. However, the proof continues to work as long as ${\xi }_{\alpha }$ is of the same order as $C_{min}$ as $n\to \infty$.

3.2. The Nonparametric FCI Algorithm in High Dimensions

The FCI is a modification of the PC algorithm that accounts for latent and selection variables. Thus, generalizations of the PC algorithm naturally extend to the FCI as well. Similar to nonPC, we propose testing for conditional independence relations nonparametrically in the sample version of the FCI-stable algorithm (Algorithm A3 in Appendix A) based on ${\rho }_0^{*}$, instead of partial correlations. We coin the resulting algorithm the ‘nonFCI’ algorithm, to emphasize that it is a generalization of parametric FCI-stable algorithms. Again, the oracle version of the nonFCI is exactly the same as that of the FCI-stable algorithm. The difference is in the implementation of the tests for conditional independence relationships in their sample versions. This broadens the scope of the FCI algorithm in causal structural learning for observational data in the presence of latent and selection variables when Gaussianity is not a viable assumption. More specifically, it enables capturing nonlinear and non-monotone conditional dependence relationships among the variables that partial correlations would fail to detect.

Equipped with the theoretical guarantees we established for the nonPC in Section 3.1, we establish below in Theorem 4 the consistency of the nonFCI algorithm for general distributions in sparse high-dimensional settings. Let $\mathcal{H}=(V,E)$ be a DAG with the vertex set partitioned as $V=V_X\cup V_L\cup V_T$, where $V_X$ indexes the set of p observed variables, $V_L$ denotes the set of latent variables and $V_T$ stands for the set of selection variables. Let $\mathcal{M}$ be the unique MAG over $V_X$. We let p grow with n and consider $p=p_n$, $\mathcal{H}={\mathcal{H}}_n$ and $Q=Q_n$, where Q is the distribution of $(U_1,\cdots ,U_p):=(X_1|V_T,\cdots ,X_p\left|V_T\right)$. We provide below the definition of possible-D-SEP sets (Definition 3.3 in [4]).

Definition 1.

Let $\mathcal{C}$ be a graph with any of the following edge types: $\circ -\circ$, $\circ \to$ and ↔. A possible-D-SEP $(X_a,X_b)$ in $\mathcal{C}$, denoted $\mathrm{pds}(\mathcal{C},X_a,X_b)$, is defined as follows: $X_c\in$$\mathrm{pds}(\mathcal{C},X_a,X_b)$ if and only if there is a path π between $X_a$ and $X_c$ in $\mathcal{C}$ such that, for every subpath $\langle X_e,X_f,X_g\rangle$ of π, $X_f$ is a collider on the subpath in $\mathcal{C}$ or $\langle X_e,X_f,X_g\rangle$ is a triangle in $\mathcal{C}$.

To prove the consistency of the nonFCI algorithm in sparse high-dimensional settings, we impose the following regularity conditions, which are similar to the assumptions imposed in Section 4 in [4].

(C3)

The distribution  $Q_n$ is faithful to the underlying MAG ${\mathcal{M}}_n$ for all n.

(C4)

The maximum size of the possible-D-SEP sets for finding the final skeleton in the FCI-stable algorithm (Algorithm A6 in Appendix A), $q_n^{\prime }$, grows at the rate of $O\left(n^{1-b}\right)$, where $0<b\le 1$.

(C5)

For any $U_i,U_j\in \{U_1,\cdots ,U_{p_n}\}$ and $U_S\subseteq \{U_1,\cdots ,U_{p_n}\}\setminus \{U_i,U_j\}$ with $|U_S|\le q_n^{\prime }$, assume

$$\begin{array}{cc}\hfill & inf\left\{|{\rho }_0^{*}(U_i,U_j|U_S)|:{\rho }_0^{*}(U_i,U_j|U_S)\ne 0\right\}\ge {\lambda }_{min}^{\prime }{\left({\lambda }_{min}^{\prime }\right)}^{-1}=O\left(n^v\right)\hfill \\ \hfill \mathrm{and}& sup|{\rho }_0^{*}(U_i,U_j|U_S)|\le {\lambda }_{max}^{\prime }\hfill \end{array}$$

where ${\lambda }_{max}^{\prime }$ is a positive constant  and   $0<v<1/4$.

Theorem 4.

Suppose conditions (A1)–(A3) and (C3)–(C5) hold. Denote by ${\mathcal{C}}_n$ and ${\mathcal{C}}_n^{*}$ the true underlying FCI-PAG and the output of the nonFCI algorithm, respectively. Then, as $n\to \infty$, $\mathbb{P}\left({\mathcal{C}}_n^{*}={\mathcal{C}}_n\right)\to 1$.

4. Numerical Studies

4.1. Performance of the NonPC Algorithm

In this subsection, we compare the performances of the nonPC and the PC-stable algorithms in finding the skeleton and the CPDAG for various simulated datasets. We simulate random DAGs in the following examples and sample from probability distributions faithful to them.

Example 1

(Linear SEM). We first fix a sparsity parameter $s\in (0,1)$ and enumerate the vertices as $V=\{1,\cdots ,p\}$. We then construct a $p\times p$ adjacency matrix Λ as follows. First, initialize Λ as a zero matrix. Next, fill every entry in the lower triangle (below the diagonal) of Λ by independent realizations of Bernoulli random variables with success probability s. Finally, replace each nonzero entry in Λ by independent realizations of a Uniform$(0.1,1)$ random variable.

In this scheme, each node has the same expected degree $\mathbb{E}\left(m\right)=(p-1)s$, where m is the degree of a node and follows a Binomial $(p-1,s)$ distribution. Using the adjacency matrix $\Lambda$, the data are then generated from the following linear structural equation model (SEM):

$$X=\Lambda X+ϵ$$

where $ϵ=({ϵ}_1,\cdots ,{ϵ}_p)$ and ${ϵ}_1,\cdots ,{ϵ}_p$ are jointly independent. To obtain samples ${\{X_1^k,\cdots ,X_p^k\}}_{k=1}^n$ on $\{X_1,\cdots ,X_p\}$, we first sample ${\{{ϵ}_1^k,\cdots ,{ϵ}_p^k\}}_{k=1}^n$ from the three following data-generating schemes. For $1\le k\le n$ and $1\le i\le p$,

1.

Normal: Generate ${ϵ}_i^k$ ’s independently from a standard normal distribution.

2.

Copula: Generate ${ϵ}_i^k$ ’s as in (1) and then transform the marginals to a $F_{1,1}$ distribution.

3.

Mixture: Generate ${ϵ}_i^k$ ’s independently from a 50–50 mixture of a standard normal and a standard Cauchy distribution.

Example 2

(Nonlinear SEM). In this example, we first generate a $p\times p$ adjacency matrix Λ in the similar way as in Example 1 and then generate the data from the following nonlinear SEM (similar to [10]): $X_i={\sum }_{j:{\Lambda }_{ij}\ne 0}{f}_{ij}\left(X_j\right)+{ϵ}_i$  with  ${ϵ}_i\stackrel{i.i.d.}{\sim }N(0,1)$, where $1\le j<i\le p$. If the functions $f_{ij}$’s are chosen to be nonlinear, then the data will typically not correspond to a well-known multivariate distribution. We consider $f_{ij}\left(x_j\right)=b_{ij1}{x}_j+b_{ij2}{x}_j^2$, where $b_{ij1}$ and $b_{ij1}$ are independently sampled from $N(0,1)$ and $N(0,0.5)$ distributions, respectively.

With the exception of Example 1.1, the above examples are all non-Gaussian graphical models. We would thus expect the nonPC to perform better than the PC-stable in learning the unknown causal structure in these examples. For each of the four data generating methods considered above, we compare the Structural Hamming Distance (SHD) [25] between the estimated and the true skeletons of the underlying DAGs using the nonPC and PC-stable algorithms. The SHD between two undirected graphs is the number of edge additions or deletions necessary to make the two graphs match. Therefore, larger SHD values between the estimated and the true skeleton correspond to worse estimates.

We consider 199 bootstrap replicates for the CdCov-based conditional independence tests in the implementation of our nonPC algorithm and the significance level $\alpha =0.05$.

Table 1. Comparison of the average structural Hamming distances (SHD) of nonPC and PC-stable algorithms across simulation studies.

			Normal		Copula
n	p	$\mathbb{E}\left(m\right)$	nonPC	PC-stable	nonPC	PC-stable
50	9	1.4	3.35	3.05	5.55	5.75
100	27	2.0	14.55	11.00	25.6	28.6
150	81	2.4	53.70	43.45	97.3	121.3
200	243	2.8	186.2	183.4	331.00	471.45
			Mixture		Nonlinear SEM
n	p	$\mathbb{E}\left(m\right)$	nonPC	PC-stable	nonPC	PC-stable
50	9	1.4	3.8	3.5	2.9	3.7
100	27	2.0	17.75	18.00	15.05	20.05
150	81	2.4	69.05	77.75	62.583	95.083
200	243	2.8	250.3	336.1	213.70	375.45

presents the average SHD for the different data generating schemes over 20 simulation runs, for different choices of $n,p$ and $\mathbb{E}\left(m\right)$.

The results in Table 1 demonstrate that the nonPC performs nearly as good as the PC-stable for the Gaussian data example, in terms of the average SHD. However, for each of the non-Gaussian data examples, the nonPC performs better than the PC-stable in estimating the true skeleton of the underlying DAGs. The improvement in SHD becomes more substantial as the dimension grows. The superior performance of the nonPC over PC-stable for the non-Gaussian graphical models is expected, as the characterization of conditional independence by partial correlations is only valid under the assumption of joint Gaussianity.

4.2. Performance of the NonFCI Algorithm

In this subsection, we compare the performances of the nonFCI and the FCI-stable algorithms over various simulated datasets. We first generate random DAGs as in Examples 1 and 2. To assess the impact of latent variables, we randomly define half of the variables with no parents and at least one child as latent. We do not consider selection variables. We run both the nonFCI and the FCI-stable algorithms on the above data examples with $n=200$, $p=\{10,20,30,100,200\}$ and $\alpha =0.01$, using 199 bootstrap replicates for the CdCov-based conditional independence tests. We consider 20 simulation runs for each of the data generating models.

Table 2. Comparison of the average structural Hamming distances (SHD) of nonFCI and FCI-stable algorithms across simulation studies.

		Normal		Copula		Mixture		Nonlinear SEM
p	$\mathbb{E}\left(\mathit{m}\right)$	nonFCI	FCI-Stable	nonFCI	FCI-Stable	nonFCI	FCI-Stable	nonFCI	FCI-Stable
10	2.0	7.15	7.60	1.3	1.8	5.65	6.80	7.15	8.20
20	2.0	14.55	17.60	4.55	6.85	13.65	18.55	19.0	20.8
30	2.0	27.65	33.95	5.25	10.15	19.3	27.8	33.40	37.85
100	3.0	109.30	150.35	26.95	60.05	62.25	111.10	115.2	149.0
200	3.0	287.75	371.40	76.733	157.267	136.05	255.10	289.6	354.1

reports the average SHD between the estimated and true PAG skeleton by the nonFCI and FCI-stable algorithms.

The results in Table 2 demonstrate that, in both the Gaussian and non-Gaussian examples, the nonFCI algorithm outperforms the FCI-stable in estimating the true PAG skeleton.

4.3. Real Data Example

A major difficulty in assessing whether nonPC and nonFCI provide more reasonable estimates compared to the parametric versions of the algorithms in high-dimensional real data settings is that the true causal graph is not known in most of the cases. In absence of the truth, we may only be able to draw some conclusions about sensible causal mechanisms by examining known or logical relationships among pairs of variables. However, this becomes increasingly difficult for larger networks, where even visualization becomes challenging. This is why we first choose a relatively smaller dataset in Section 4.3.1, where we can draw upon background knowledge to glean insight into potential causal mechanisms in a setting where the data are clearly non-Gaussian. This example highlights the main focus of the paper that, with non-Gaussian data (categorical, as in this example), nonPC is expected to perform better than the PC-stable in learning the true causal structure of the underlying DAG. In Section 4.3.2, we consider a larger example and examine the performance of PC-stable and nonPC in learning the DAG from both seemingly Gaussian data as well as a categorized version of the same data. This example clearly illustrates the potential limitations of PC-stable: in contrast to nonPC, the output of PC-stable can be strikingly different when applied to a categorized version of the original data.

4.3.1. Montana Poll Dataset

To demonstrate the flexibility of our proposed framework, we first apply the nonPC algorithm to the Montana Economic Outlook Poll dataset. The poll was conducted in May 1992 where a random sample of 209 Montana residents were asked whether their personal financial status was worse, the same or better than a year ago, and whether they thought the state economic outlook was better than the year before. Accompanying demographic information on the respondents’ age, income, political orientation, and area of residence in the state were also recorded. We obtained the dataset from the Data and Story Library (DASL), available at https://math.tntech.edu/e-stat/DASL/page4.html (accessed on 25 March 2021). The study is comprised of the following seven categorical variables: AGE = 1 for under 35, 2  for 35–54, 3  for 55 and over;  SEX = 0   for male, 1  for female;  INC = yearly income: 1 for under $20 K, 2  for $20–35 K, 3  for over $35 K;  POL = 1 for Democrat, 2  for Independent, 3  for Republican;  AREA = 1 for Western, 2  for Northeastern, 3  for Southeastern Montana;  FIN (=Financial status): 1 for worse, 2  for same, 3  for better than a year ago;  and  STAT (=State economic outlook): 1 for better, 0  for not better than a year ago.

After removing the cases with missing values, we are left with $n=163$ samples. Since all the variables are categorical, the Gaussianity assumption is outrightly violated. Thus, we would expect the nonPC to perform better than the PC-stable in learning the true causal structure among the variables in this case. Figure 1 below presents the CPDAGs estimated by the nonPC and PC-stable algorithms at a significance level $\alpha =0.1$. We consider 199 bootstrap replicates for the CdCov-based conditional independence tests in the implementation of the nonPC algorithm.

It is quite intuitive that age and sex are likely to affect the income; one’s financial status and the area of residence might also influence their political inclination; and improvements or downturns in the state economic outlook might impact an individual’s financial status. The CPDAG estimated by the nonPC algorithm in Figure 1a affirms such common-sense understanding of these causal influences. However, in the CPDAG estimated by the PC-stable in Figure 1b, the edge between age and income is missing. In addition, the directed edges POL → AREA and POL → FIN seem to make little sense in this case.

4.3.2. Protein Expression Data

We next consider a protein expression dataset of 410 patients with breast cancer from The Cancer Genome Atlas (TCGA). The dataset consists of $p=118$ genes, and we randomly select a subset of $n=100$ patients with PR-negative status. Since the true causal structure of the genes in the cancer cells may be different than that of normal cells [26], we apply both the nonPC and PC-stable algorithms to learn the causal structure. To put the performances of the nonPC and PC-stable under scrutiny as the data depart farther away from Gaussianity, we categorize the protein expression data for each of the p genes, denoted by ${\left\{X_a^k\right\}}_{k=1}^n$, $1\le a\le p$, as follows. We compute the three quartiles $Q_{1;a},Q_{2;a}$ and $Q_{3;a}$ of the protein expression values for every $1\le a\le p$. Consequently, we obtain categorized protein expressions ${\left\{X_{C;a}^k\right\}}_{k=1}^n$ for $1\le a\le p$, where

$$\begin{array}{c}\hfill X_{C;a}^k:=\left\{\begin{array}{cc}0\hfill & \mathrm{if}{X}_a^k\le Q_{1;a}\hfill \\ 1\hfill & \mathrm{if}{Q}_{1;a}<X_a^k\le Q_{2;a}\hfill \\ 2\hfill & \mathrm{if}{Q}_{2;a}<X_a^k\le Q_{3;a}\hfill \\ 3\hfill & \mathrm{if}{X}_a^k>Q_{3;a}.\hfill \end{array}\right.\end{array}$$

We apply the nonPC and PC-stable algorithms to both the original and the categorized protein expression data at a significance level $\alpha =0.01$. We consider 199 bootstrap replicates for the CdCov-based conditional independence tests in the implementation of the nonPC algorithm.

Table 3. Comparison of the SHD between the skeletons estimated from the original and the categorized protein expression data by the nonPC and PC-stable algorithms.

nonPC	PC-Stable
22	79

below shows the SHD between the skeletons estimated from the original and the categorized data by the nonPC and PC-stable algorithms. It can be seen that the SHD between the skeletons estimated from the original and categorized data by the PC-stable algorithm is much larger than that for nonPC. This example highlights the potential limitation of parametric implementations of the PC algorithm: when the data deviate farther away from Gaussianity (in this case, being categorical), the estimates produced by the PC-stable may deviate considerably more from the estimates from the original data. In contrast, the nonparametric test in nonPC delivers more stable estimates regardless of the data distribution.

5. Discussion

We proposed nonparametric variants of the widely popular PC-stable and FCI-stable algorithms, which employ conditional distance covariance (CdCov) to test for conditional independence relationships in their sample versions. Our proposed algorithms broaden the applicability of the PC/PC-stable and FCI/FCI-stable algorithms to general distributions over DAGs, and enable taking into account nonlinear and non-monotone conditional dependence among the random variables, which partial correlations fail to capture. We show that the high-dimensional consistency of the PC-stable and FCI-stable algorithms carry over to more general distributions over DAGs when we implement CdCov-based nonparametric tests for conditional independence. These results are obtained without imposing any strict distributional assumptions and only require moment and tail conditions on the variables.

There are several intriguing potential directions for future research. First, it is generally difficult to select the tuning parameter (i.e., the significance threshold for the CdCov test) in causal structure learning. One possible strategy is to use ideas based on stability selection [27,28]. By assessing the stability of the estimated graphs in multiple subsamples, this strategy allows us to choose the tuning parameter in order to control the false positive error. However, the repeated subsampling increases the computational burden. Second, the computational and sample complexities of the PC and FCI algorithms (and hence those of the nonPC and nonFCI) scale with the maximum degree of the DAG, which is assumed to be small relative to the sample size. However, in many applications, one encounters sparse graphs containing a small number of highly connected ‘hub’ nodes. In such cases, ref. [29] proposed a low-complexity variant of the PC algorithm, namely the reduced PC (rPC) algorithm that exploits the local separation property of large random networks [30]. The rPC is shown to consistently estimate the skeleton of a high-dimensional DAG by conditioning only on sets of small cardinality. More recently, ref. [31] have generalized this approach to account for unobserved confounders. In this light, it would be intriguing to develop computationally faster variants of the nonPC and nonFCI in the future by exploiting the idea of local separation.