Robust Permutation Tests for Penalized Splines

Abstract

Penalized splines are frequently used in applied research for understanding functional relationships between variables. In most applications, statistical inference for penalized splines is conducted using the random effects or Bayesian interpretation of a smoothing spline. These interpretations can be used to assess the uncertainty of the fitted values and the estimated component functions. However, statistical tests about the nature of the function are more difficult, because such tests often involve testing a null hypothesis that a variance component is equal to zero. Furthermore, valid statistical inference using the random effects or Bayesian interpretation depends on the validity of the utilized parametric assumptions. To overcome these limitations, I propose a flexible and robust permutation testing framework for inference with penalized splines. The proposed approach can be used to test omnibus hypotheses about functional relationships, as well as more flexible hypotheses about conditional relationships. I establish the conditions under which the methods will produce exact results, as well as the asymptotic behavior of the various permutation tests. Additionally, I present extensive simulation results to demonstrate the robustness and superiority of the proposed approach compared to commonly used methods.

1. Introduction

1.1. Penalized Spline Prevalence

Penalized splines are used within modern multiple and generalized nonparametric regression frameworks [1,2], such as the smoothing spline analysis of variance models [3,4,5] and generalized additive models [6,7,8], to discover unknown functional relationships between a response variable Y and a collection of predictors $X_1,\dots ,X_d$. Penalized splines and their variants have been applied to understand functional relationships in data from a variety of different disciplines. For example, recent applications include the use of penalized splines to model spatiotemporal patterns in US homelessness [9], biomechanical analysis of locomotion data [10,11], fear learning curves in veterans with PTSD [12], functional properties of successful smiles [13], self-esteem trajectories across the lifespan [14], and spatiotemporal trends in social media [15].

In addition to being a commonly used tool in applied research, penalized splines are also frequently the focus of theoretical and computational statistics research. For example, there has been recent interest in fitting penalized spline models with ordinal predictors and responses [16,17], mixed-effects models with penalized spline components [18,19,20], efficient approximation for large samples [21,22], and efficient algorithms for fitting penalized spline models to big data [23,24]. Furthermore, given the frequent usage of penalized splines for the analysis of real world data, there has been a growing interest in the robustness of penalized spline tuning and inference methods under model misspecification [25]. Recently, there has also been the development of alternative spline penalization approaches for nonparametric function estimation from noisy data [26].

1.2. Penalized Spline Definition

Consider a multiple nonparametric regression model [1,2] of the form

$$Y_i=\eta \left({\mathbf{X}}_i\right)+{ϵ}_i$$

(1)

where $Y_i$ is the i-th realization of the response variable $Y\in \mathbb{R}$, ${\mathit{X}}_i={(X_{i1},\dots ,X_{id})}^{\top }$ is the i-th realization of the predictor variable $\mathit{X}\in \mathcal{X}={\mathcal{X}}_1\times \cdots \times {\mathcal{X}}_d$, and ${ϵ}_i$ is the i-th realization of the error term $ϵ\in \mathbb{R}$. The error terms are assumed to satisfy $E\left({ϵ}_i\right)=0$ and $E\left({ϵ}_i^2\right)={\sigma }_i^2$ with ${\sigma }_i^2<\infty$ denoting the i-th observation’s error variance. To estimate $\eta$, it is typical to minimize the penalized least squares (PLS) functional

$$\frac{1}{n}\sum _{i=1}^n{\left(Y_i-\eta \left({\mathbf{X}}_i\right)\right)}^2+\lambda J\left(\eta \right)$$

where $J(·)$ is the penalty functional, and $\lambda \ge 0$ is the smoothing parameter, which controls the balance between fitting and smoothing.

Given $\lambda$, the Kimeldorf-Wahba representer theorem [27] reveals that the minimizer of the PLS functional has the form

$${\eta }_{\lambda }\left(\mathit{X}\right)=a+\sum _{v=1}^{m-1}{b}_v{N}_v\left(\mathit{X}\right)+\sum _{i=1}^r{c}_{i}R(\mathit{X},{\mathit{X}}_i^{*})$$

where ${\left\{N_v\right\}}_{v=0}^{m-1}$ are known functions that span the null space ${\mathcal{H}}_0=\{\eta :J\left(\eta \right)=0\}$ with $N_0\left(\mathbf{X}\right)=1$, the symmetric and bivariate function $R(·,·)$ is the known reproducing kernel of the contrast space ${\mathcal{H}}_1=\{\eta :J\left(\eta \right)<\infty \}$, the collection of predictor scores ${\left\{{\mathit{X}}_i^{*}\right\}}_{i=1}^r$ are the selected spline knots, $a\in \mathbb{R}$ is the unknown intercept, and $\mathbf{b}={(b_1,\dots ,b_{m-1})}^{\top }$ and $\mathbf{c}={(c_1,\dots ,c_r)}^{\top }$ are the unknown basis function coefficient vectors. The representer theorem uses all design points as knots (i.e., $r=n$ and ${\mathit{X}}_i^{*}={\mathit{X}}_i)$, but reasonable approximations can be obtained using $r<n$ knots [21,22,28].

1.3. Penalized Spline Estimation

Using the representer theorem, the nonparametric regression model can be written as

$$Y_i=a+{\mathit{N}}_i^{\top }\mathbf{b}+{\mathit{R}}_i^{\top }\mathbf{c}+{ϵ}_i$$

where ${\mathit{N}}_i^{\top }=(N_1\left({\mathit{X}}_i\right),\dots ,N_{m-1}\left({\mathit{X}}_i\right))$ is the null space basis function vector for the i-th observation, and ${\mathit{R}}_i^{\top }=(R({\mathit{X}}_i,{\mathit{X}}_1^{*}),\dots ,R({\mathit{X}}_i,{\mathit{X}}_r^{*}))$ is the reproducing kernel function evaluated at ${\mathit{X}}_i$ and the selected knots. Let $Y_i^c=Y_i-\overline{Y}$ denote the centered response and let ${\mathit{N}}_i^c={\mathit{N}}_i-\overline{\mathit{N}}$ and ${\mathit{R}}_i^c={\mathit{R}}_i-\overline{\mathit{R}}$ denote the centered basis functions, where $\overline{Y}=\frac{1}{n}{\sum }_{i=1}^n{Y}_i$, $\overline{\mathit{N}}=\frac{1}{n}{\sum }_{i=1}^n{\mathit{N}}_i$, and $\overline{\mathit{R}}=\frac{1}{n}{\sum }_{i=1}^n{\mathit{R}}_i$. Then, the PLS functional can be rewritten as

$$\frac{1}{n}{\parallel {\mathit{Y}}_c-{\mathbf{N}}_c\mathbf{b}-{\mathbf{R}}_c\mathbf{c}\parallel }^2+\lambda {\mathbf{c}}^{\top }\mathbf{Q}\mathbf{c}$$

where ${\mathit{Y}}_c={(Y_1^c,\dots ,Y_n^c)}^{\top }$, ${\mathbf{N}}_c={[{\mathit{N}}_1^c,\dots ,{\mathit{N}}_n^c]}^{\top }$ and ${\mathbf{R}}_c={[{\mathit{R}}_1^c,\dots ,{\mathit{R}}_n^c]}^{\top }$ are the mean centered basis function matrices, and $\mathbf{Q}=\left[R({\mathit{X}}_i^{*},{\mathit{X}}_j^{*})\right]$ is the penalty matrix.

Let $\mathbf{d}={(b_1,\dots ,b_{m-1},c_1,\dots ,c_r)}^{\top }$ denote the combined basis function coefficient vector, and let ${\mathbf{K}}_c=({\mathbf{N}}_c,{\mathbf{R}}_c)$ denote the combined (centered) basis function matrix. Given the smoothing parameter $\lambda$, it is well known that the basis function coefficients that minimize the PLS functional can be written as

$${\widehat{\mathbf{d}}}_{\lambda }={\left({\mathbf{K}}_c^{\top }{\mathbf{K}}_c+n\lambda {\mathbf{Q}}^{*}\right)}^{†}{\mathbf{K}}_c^{\top }{\mathit{Y}}_c$$

where ${\mathbf{Q}}^{*}=\mathrm{bdiag}({\mathbf{0}}_{m-1},\mathbf{Q})$ is the block diagonal penalty matrix, and ${(·)}^{†}$ denotes the Moore-Penrose pseudo-inverse [29,30]. The least squares estimate of the intercept term can then be written as $\widehat{a}=\overline{Y}-\overline{\mathit{K}}{}^{\top }{\widehat{\mathbf{d}}}_{\lambda }$, where $\overline{\mathit{K}}{}^{\top }=(\overline{\mathit{N}}{}^{\top },\overline{\mathit{R}}{}^{\top })$. The fitted values have the form ${\widehat{\eta }}_{\lambda }=\widehat{a}+{\widehat{\eta }}_{0\lambda }+{\widehat{\eta }}_{1\lambda }$, where ${\widehat{\eta }}_{0\lambda }={\mathbf{N}}_c{\widehat{\mathbf{b}}}_{\lambda }$ is the (non-constant) null space contribution, and ${\widehat{\eta }}_{1\lambda }={\mathbf{R}}_c{\widehat{\mathbf{c}}}_{\lambda }$ is the contrast space contribution with ${\widehat{\mathbf{d}}}_{\lambda }^{\top }=({\widehat{\mathbf{b}}}_{\lambda }^{\top },{\widehat{\mathbf{c}}}_{\lambda }^{\top })$.

1.4. Bayesian Interpretation

Given the estimated function ${\widehat{\eta }}_{\lambda }$, the statistical inference about the unknown function $\eta$ is often conducted using the Bayesian interpretation of a smoothing spline [31,32]. This approach assumes that $\eta ={\eta }_0+{\eta }_1$, where the null space function ${\eta }_0$ has a vague prior, and the contrast space function ${\eta }_1$ has a Gaussian process prior with a mean zero and covariance matrix proportional to ${\mathbf{Q}}^{†}$. Given these prior assumptions, it can be demonstrated that the posterior distribution of $\eta$ given $\mathit{Y}$ is multivariate normal with mean vector ${\widehat{\eta }}_{\lambda }$ and covariance matrix ${\left({\mathbf{K}}^{\top }\mathbf{K}+n\lambda {\mathbf{Q}}^{*}\right)}^{†}$, where $\mathbf{K}=({\mathbf{1}}_n,\mathbf{N},\mathbf{R})$. When the smoothing parameter $\lambda$ is chosen via the generalized cross-validation (GCV) criterion [33], confidence intervals formed using the Bayesian covariance matrix tend to have an “across-the-function” coverage property [32]. See [25] for an investigation of the Bayesian CI coverage properties.

The Bayesian confidence intervals can be used to assess the uncertainty of the fitted values and the component functions, i.e., the main and interaction effects of the d predictors [34,35]. However, testing hypotheses about the nature of the function $\eta$ is a more difficult problem. Various different approaches have been proposed for testing hypotheses about penalized smoothers [36,37,38,39,40,41,42,43]. However, these methods are primarily designed for testing specific hypotheses (e.g., $H_0:\eta \in {\mathcal{H}}_0$) under the assumption of homoscedastic Gaussian errors. The exception is Wood’s (2013a) approach, which is designed to be more general, but the validity of Wood’s approach depends on having a correctly specified parametric (exponential family) distribution. As a result, the generalizability of the existing inference methods is limited, and the validity of these methods is suspect when the parametric assumptions are questionable.

1.5. Proposed Approach

When working with real data, it can be difficult to determine whether or not the parametric assumptions that are required for valid hypothesis testing are reasonably met. Furthermore, if the error terms are non-Gaussian and/or heteroscedastic, the proposed hypothesis tests may produce substantially misleading inferential results. To overcome this important practical issue, I propose a flexible permutation testing framework for robust inference in nonparametric regression models. The proposed approach extends recent advances in robust permutation tests for linear models [44] to generalized ridge regression (GRR) and penalized smoothing problems. As I demonstrate in the following sections, the proposed framework can be used for overall (omnibus) tests about functional relationships, as well as more specific (conditional) tests.

The remainder of this paper is organized as follows: Section 2 develops the foundations and theory for omnibus permutation tests using GRR estimators, Section 3 extends the permutation testing framework to conditional tests of effects, Section 4 presents extensive simulation results to validate the theoretical results derived in the previous sections, and Section 5 discusses how the proposed framework can be flexibly adapted for testing a variety hypotheses about semi- and non-parametric regression models.

2. Omnibus Regression Tests

2.1. Model and Estimation

Given an independent sample of n observations, consider the linear regression model

$$Y_i=\alpha +{\mathit{X}}_i^{\top }\mathit{\beta }+{ϵ}_i$$

(2)

for $i\in \{1,\dots ,n\}$, where $Y_i$ is the i-th realization of the response variable $Y\in \mathbb{R}$, ${\mathit{X}}_i^{\top }=(X_{i1},\dots ,X_{ip})$ is the i-th realization of the predictor vector ${\mathit{X}}^{\top }=(X_1,\dots ,X_p)\in {\mathbb{R}}^p$, and ${ϵ}_i$ is the i-th realization of the error term $ϵ\in \mathbb{R}$. The error terms are assumed to satisfy $E\left({ϵ}_i\right)=0$ and $E\left({ϵ}_i^2\right)={\sigma }_i^2$ with ${\sigma }_i^2<\infty$ denoting the i-th observation’s (finite) error variance. Without a loss of generality, we can assume that the response and predictors are centered, which implies that the intercept $\alpha$ can be dropped from the model for estimation purposes. Let $Y_i^c=Y_i-\overline{Y}$ and ${\mathit{X}}_i^c={\mathit{X}}_i-\overline{\mathit{X}}$ denote the mean centered response and predictor vector for the i-th observation, where $\overline{Y}=\frac{1}{n}{\sum }_{i=1}^n{Y}_i$ and $\overline{\mathit{X}}=\frac{1}{n}{\sum }_{i=1}^n{\mathit{X}}_i$.

To estimate the coefficients in $\mathit{\beta }$, consider minimizing the GRR loss function [45]

$$\frac{1}{n}{({\mathit{Y}}_c-{\mathbf{X}}_c\mathit{\beta })}^{\top }({\mathit{Y}}_c-{\mathbf{X}}_c\mathit{\beta })+{\mathit{\beta }}^{\top }\mathsf{\Delta }\mathit{\beta }$$

(3)

where ${\mathit{Y}}_c={(Y_1^c,\dots ,Y_n^c)}^{\top }$ is the centered response vector, ${\mathbf{X}}_c={[{\mathit{X}}_1^c,\dots ,{\mathit{X}}_n^c]}^{\top }$ is the centered design matrix, and $\mathsf{\Delta }$ is a $p\times p$ symmetric and positive semi-definite penalty matrix. The coefficients that minimize the GRR problem in Equation (3) have the form

$${\widehat{\mathit{\beta }}}_{\mathsf{\Delta }}={\left(\frac{1}{n}{\mathbf{X}}_c^{\top }{\mathbf{X}}_c+\mathsf{\Delta }\right)}^{-1}\left(\frac{1}{n}{\mathbf{X}}_c^{\top }{\mathit{Y}}_c\right)$$

(4)

which is subscripted to emphasize that the estimated coefficients depend on the penalty matrix $\mathsf{\Delta }$. Given the estimated slope vector ${\widehat{\mathit{\beta }}}_{\mathsf{\Delta }}$, the least squares estimate of the intercept has the form $\widehat{\alpha }=\overline{Y}-{\overline{\mathit{X}}}^{\top }{\widehat{\mathit{\beta }}}_{\mathsf{\Delta }}$.

2.2. Asymptotic Distributions

Consider the linear regression model from Equation (2) with the assumptions:

A1.

$Y_i=\alpha +{\mathit{X}}_i^{\top }\mathit{\beta }+{ϵ}_i$ with $E\left({ϵ}_i\right)=0$ and $E\left({ϵ}_i^2\right)={\sigma }_i^2<\infty$ for $i=1,\dots ,n$

A2.

$(Y_i,{\mathit{X}}_i)$ are iid from a distribution satisfying $E\left({ϵ}_i{\mathit{X}}_i\right)=0$

A3.

${\mathsf{\Sigma }}_X=E\left(({\mathit{X}}_i-{\mu }_X){({\mathit{X}}_i-{\mu }_X)}^{\top }\right)$ and ${\mathsf{\Omega }}_X=E\left({ϵ}_i^2({\mathit{X}}_i-{\mu }_X){({\mathit{X}}_i-{\mu }_X)}^{\top }\right)$ are nonsingular, where ${\mu }_X=E\left({\mathit{X}}_i\right)$, and $\frac{1}{n}{\mathbf{X}}_c^{\top }{\mathbf{X}}_c+\mathsf{\Delta }$ is almost surely invertible. In the fixed predictors case, the mean vector is ${\mu }_X=\frac{1}{n}{\sum }_{i=1}^n{\mathit{X}}_i$ and the the covariance matrix terms are defined as ${\mathsf{\Sigma }}_X=\frac{1}{n}{\mathit{X}}_c^{\top }{\mathit{X}}_c$ and ${\mathsf{\Omega }}_X=\frac{1}{n}{\mathit{X}}_c^{\top }\mathsf{\Psi }{\mathit{X}}_c$, where $\mathsf{\Psi }=\mathrm{diag}({\sigma }_1^2,\dots ,{\sigma }_n^2)$.

Given assumptions A1–A3, the GRR estimator provides an estimate of ${\mathit{\beta }}_{\mathsf{\Delta }}={\mathsf{\Sigma }}_{X\mathsf{\Delta }}^{-1}{\sigma }_{XY}$, where ${\mathsf{\Sigma }}_{X\mathsf{\Delta }}={\mathsf{\Sigma }}_X+\mathsf{\Delta }$ and ${\sigma }_{XY}=E\left(({\mathit{X}}_i-{\mu }_X)(Y_i-{\mu }_Y)\right)$ is the covariance between ${\mathit{X}}_i$ and $Y_i$. Note that ${\mathit{\beta }}_{\mathsf{\Delta }}={\mathsf{\Sigma }}_{X\mathsf{\Delta }}^{-1}{\mathsf{\Sigma }}_X\mathit{\beta }$, where $\mathit{\beta }={\mathsf{\Sigma }}_X^{-1}{\sigma }_{XY}$ is estimated by the ordinary least squares (OLS) estimator $\widehat{\mathit{\beta }}=\widehat{\mathsf{\Sigma }}{}_X^{-1}{\widehat{\sigma }}_{XY}$ with ${\widehat{\mathsf{\Sigma }}}_X=\frac{1}{n}{\mathit{X}}_c^{\top }{\mathit{X}}_c$ and ${\widehat{\sigma }}_{XY}=\frac{1}{n}{\mathit{X}}_c^{\top }{\mathit{Y}}_c$. This implies that the GRR estimator can be written as ${\widehat{\mathit{\beta }}}_{\mathsf{\Delta }}=\widehat{\mathsf{\Sigma }}{}_{X\mathsf{\Delta }}^{-1}{\widehat{\mathsf{\Sigma }}}_X\widehat{\mathit{\beta }}$ where $\widehat{\mathsf{\Sigma }}{}_{X\mathsf{\Delta }}={\widehat{\mathsf{\Sigma }}}_X+\mathsf{\Delta }$.

Lemma 1.

Given assumptions A1–A3, the GRR estimator ${\widehat{\mathit{\beta }}}_{\mathsf{\Delta }}$ from Equation (4) is asymptotically normal with mean vector ${\mathit{\beta }}_{\mathsf{\Delta }}$ and covariance matrix $\frac{1}{n}{\mathsf{\Sigma }}_{X\mathsf{\Delta }}^{-1}{\mathsf{\Omega }}_X{\mathsf{\Sigma }}_{X\mathsf{\Delta }}^{-1}$, i.e.,

$$\sqrt{n}\left({\widehat{\mathit{\beta }}}_{\mathsf{\Delta }}-{\mathit{\beta }}_{\mathsf{\Delta }}\right)\stackrel{d}{\to }N\left(\mathbf{0},{\mathsf{\Sigma }}_{X\mathsf{\Delta }}^{-1}{\mathsf{\Omega }}_X{\mathsf{\Sigma }}_{X\mathsf{\Delta }}^{-1}\right)$$

as $n\to \infty$, where the notation $\stackrel{d}{\to }$ denotes convergence in distribution.

Lemma 1 can be proved using results of White [46], who demonstrated that the OLS estimator $\widehat{\mathit{\beta }}$ is asymptotically normal with a mean vector $\mathit{\beta }$ and covariance matrix $\frac{1}{n}{\mathsf{\Sigma }}_X^{-1}{\mathsf{\Omega }}_X{\mathsf{\Sigma }}_X^{-1}$ under assumptions A1–A3. Noting that ${\widehat{\mathit{\beta }}}_{\mathsf{\Delta }}=\widehat{\Sigma }{}_{X\mathsf{\Delta }}^{-1}{\widehat{\mathsf{\Sigma }}}_X\widehat{\mathit{\beta }}$ and ${\mathit{\beta }}_{\mathsf{\Delta }}={\mathsf{\Sigma }}_{X\mathsf{\Delta }}^{-1}{\mathsf{\Sigma }}_X\mathit{\beta }$ completes the proof of Lemma 1, given that ${\widehat{\mathsf{\Sigma }}}_X$ is a consistent estimator of ${\mathsf{\Sigma }}_X$. See Appendix A.1 for details.

Lemma 2.

Consider the linear model in Equation (2) with the assumptions $\mathit{\beta }\sim N({\mathbf{0}}_p,\frac{{\sigma }^2}{n}{\mathsf{\Delta }}^{-1})$ and ${ϵ}_i\stackrel{\mathrm{iid}}{\sim }N(0,{\sigma }^2)$, and suppose that β and ${ϵ}_i$ are independent of one another. Under these assumptions, the posterior distribution of β given $\mathit{Y}$ is multivariate normal with mean vector ${\widehat{\mathit{\beta }}}_{\mathsf{\Delta }}$ and covariance matrix $\frac{{\sigma }^2}{n}\widehat{\mathsf{\Sigma }}{}_{X\mathsf{\Delta }}^{-1}$. The asymptotic mean vector is ${\mathit{\beta }}_{\mathsf{\Delta }}$ and the asymptotic covariance matrix is $\frac{{\sigma }^2}{n}{\mathsf{\Sigma }}_{X\mathsf{\Delta }}^{-1}$.

Lemma 2 can be proved by using the results of Henderson [47,48], who derived the covariance matrix of the best linear unbiased estimator (BLUE) and the best linear unbiased predictor (BLUP) in linear mixed models (e.g., see [49]). The asymptotic mean vector and covariance matrix result from the facts that ${\widehat{\mathsf{\Sigma }}}_X$ and ${\widehat{\sigma }}_{XY}$ are consistent estimators of ${\mathsf{\Sigma }}_X$ and ${\sigma }_{XY}$, respectively. See Appendix A.2 for details.

2.3. Test Statistics

Consider the linear model in Equation (2), and suppose that we want to test the null hypothesis $H_0:\mathit{\beta }={\mathbf{0}}_p$ versus the alternative hypothesis $H_1:\mathit{\beta }\ne {\mathbf{0}}_p$, where the notation ${\mathbf{0}}_p$ denotes a $p\times 1$ vector of zeros. If $\mathit{\beta }\sim N({\mathbf{0}}_p,\frac{{\sigma }^2}{n}{\mathsf{\Delta }}^{-1})$ and ${ϵ}_i\stackrel{\mathrm{iid}}{\sim }N(0,{\sigma }^2)$ are independent of one another, one may consider using the F statistic

$$F=\frac{n}{p{\widehat{\sigma }}^2}{\widehat{\mathit{\beta }}}_{\mathsf{\Delta }}^{\top }\widehat{\mathsf{\Sigma }}{}_{X\mathsf{\Delta }}{\widehat{\mathit{\beta }}}_{\mathsf{\Delta }}$$

(5)

where ${\widehat{\sigma }}^2=\frac{1}{n-p-1}{\sum }_{i=1}^n{\widehat{ϵ}}_i^2$ is the estimated error variance, and ${\widehat{ϵ}}_i=Y_i-{\widehat{Y}}_i$ is the residual with ${\widehat{Y}}_i=\widehat{\alpha }+{\mathbf{X}}_i^{\top }{\widehat{\mathit{\beta }}}_{\mathsf{\Delta }}$ denoting the fitted value for the i-th observation. Note that if $H_0$ is true and $\mathit{\beta }$ is fixed, then the F statistic would approach an F distribution with degrees of freedom parameters p and $n-p-1$ as $\mathsf{\Delta }\to \mathbf{0}$. However, under the assumptions of Lemma 2, the F statistic in Equation (5) will not follow an $F_{p,n-p-1}$ distribution under $H_0$, and it may produce asymptotically invalid results when used in a permutation test.

The assumption ${ϵ}_i\stackrel{\mathrm{iid}}{\sim }N(0,{\sigma }^2)$ may be questionable in many real data situations, where the error terms may be non-Gaussian and/or heteroscedastic. If the error terms are heteroscedastic, i.e., if $E\left({ϵ}_i^2\right)={\sigma }_i^2$, then the F statistic may not produce valid results even in a permutation test (see [44,50,51]). Note that even if ${ϵ}_i\stackrel{\mathrm{iid}}{\sim }(0,{\sigma }^2)$ for some non-Gaussian distribution, the F statistic may produce asymptotically invalid results when used in a permutation test. Instead, consider the Wald test statistic

$$W=n{\widehat{\mathit{\beta }}}_{\mathsf{\Delta }}^{\top }{\left[\widehat{\mathsf{\Sigma }}{}_{X\mathsf{\Delta }}^{-1}{\widehat{\mathsf{\Omega }}}_X\widehat{\mathsf{\Sigma }}{}_{X\mathsf{\Delta }}^{-1}\right]}^{-1}{\widehat{\mathit{\beta }}}_{\Delta }$$

(6)

where ${\widehat{\mathsf{\Omega }}}_X=\frac{1}{n}{\mathbf{X}}_c^{\top }{\mathbf{D}}_Y^2{\mathbf{X}}_c$ with ${\mathbf{D}}_Y=\mathrm{diag}(Y_1^c,\dots ,Y_n^c)$. Under assumptions A1–A3, the W statistic asymptotically follows a ${\chi }_p^2$ distribution when $H_0$ is true, which is a result of Lemma 1 and the consistency of the estimators ${\widehat{\mathsf{\Sigma }}}_X$ and ${\widehat{\mathsf{\Omega }}}_X$ under $H_0$.

2.4. Permutation Inference

Let the vector $\pi =({\pi }_1,\dots ,{\pi }_n)$ denote some permutation of the integers $\{1,\dots ,n\}$, and define ${\mathit{Y}}_{\pi }=(Y_{{\pi }_1}^c,\dots ,Y_{{\pi }_n}^c)$ to be the permuted (and centered) response vector using the permutation vector $\mathit{\pi }$. Furthermore, let $F(Y_{\pi },X)$ and $W(Y_{\pi },X)$ denote the test statistics from Equations (5) and (6) calculated using the permuted response vector ${\mathbf{Y}}_{\pi }$. When ${\mathit{X}}_i$ is independent of ${ϵ}_i$, the permutation test conducted using $W(Y_{\pi },X)$ will be exact, given that ${\mathsf{\Omega }}_X=E\left({\mathsf{\sigma }}_i^2\right){\mathsf{\Sigma }}_X$ when ${\mathit{X}}_i$ and ${ϵ}_i$ are independent. However, the permutation test using $F(Y_{\pi },X)$ is not guaranteed to be exact or asymptotically valid, given that the asymptotic sampling distribution of $F(Y,X)$ may not be the same as the permutation distribution (see [44]). When there is dependence between ${\mathit{X}}_i$ and ${ϵ}_i$, the permutation test conducted using $F(Y_{\pi },X)$ will be inexact and asymptotically invalid, whereas the permutation test conducted using $W(Y_{\pi },X)$ will be inexact and asymptotically valid. Define the additional assumption A4: $E\left(Y_i^4\right)<\infty$ and $E\left(X_{ij}^4\right)<\infty$ for all j.

Theorem 1.

Consider the linear model with assumptions A1–A4. When $\mathit{\beta }={\mathbf{0}}_p$, the permutation distribution of $W(Y_{\pi },X)$ converges to a ${\chi }_p^2$ distribution as $n\to \infty$.

Theorem 1 can be proved by combining the results in Lemma 1 with the results in Theorem 3.1 of DiCiccio and Romano [44], who derived the asymptotic nature of the permutation distribution $W(Y_{\pi },X)$ for the OLS estimator $\widehat{\mathit{\beta }}$. Note that Theorem 1 reveals that a permutation test using $W(Y_{\pi },X)$ will produce asymptotically level $\alpha$ rejection rates when the null hypothesis $H_0:\mathit{\beta }={\mathbf{0}}_p$ is true.

Let the vector $\mathbf{\psi }=({\psi }_1,\dots ,{\psi }_n)$ denote a resigning vector where ${\psi }_i\in \{-1,1\}\forall i$, and define ${\mathit{Y}}_{\psi }=({\psi }_1{Y}_1^c,\dots ,{\psi }_n{Y}_n^c)$ to be the resigned (and centered) response vector using the resigning vector $\mathbf{\psi }$. Furthermore, let $F(Y_{\psi },X)$ and $W(Y_{\psi },X)$ denote the test statistics from Equations (5) and (6) calculated using the resigned response vector ${\mathit{Y}}_{\psi }$. When ${\mathit{X}}_i$ is independent of ${ϵ}_i$ and the errors are symmetric, the permutation test conducted using $W(Y_{\psi },X)$ will be exact, but the permutation test using $F(Y_{\psi },X)$ may be invalid. When ${\mathbf{X}}_i\cancel{\perp \perp }{ϵ}_i$, the permutation test conducted using $F(Y_{\psi },X)$ will be inexact and asymptotically invalid, whereas the permutation test conducted using $W(Y_{\psi },X)$ will be inexact and asymptotically valid, regardless of whether or not errors are symmetric.

Theorem 2.

Consider the linear model with assumptions A1–A4. When $\mathit{\beta }={\mathbf{0}}_p$, the permutation distribution of $W(Y_{\psi },X)$ converges to a ${\chi }_p^2$ distribution as $n\to \infty$.

Theorem 2 can be proved by combining the results in Lemma 1 with the results in Theorem 3.2 of DiCiccio and Romano [44], who derived the asymptotic nature of the permutation distribution $W(Y_{\psi },X)$ for the OLS estimator $\widehat{\mathit{\beta }}$. Note that Theorem 2 reveals that a permutation test using $W(Y_{\psi },X)$ will produce asymptotically level $\alpha$ rejection rates when the null hypothesis $H_0:\mathit{\beta }={\mathbf{0}}_p$ is true.

Corollary 1.

Define ${\mathit{Y}}_{\pi \psi }=({\psi }_1{Y}_{{\pi }_1}^c,\dots ,{\psi }_n{Y}_{{\pi }_n}^c)$ to be the permuted and resigned (centered) response vector using the permutation vector π and resigning vector ψ. Consider the linear model with assumptions A1–A4. When $\mathit{\beta }={\mathbf{0}}_p$, the permutation distribution of $W(Y_{\pi \psi },X)$ converges to a ${\chi }_p^2$ distribution as $n\to \infty$, where $W(Y_{\pi \psi },X)$ is the test statistic in Equation (6), calculated using the permuted and resigned response vector ${\mathit{Y}}_{\pi \psi }$.

Corollary 1 follows directly from the results of Theorems 1 and 2.

3. Conditional Regression Tests

3.1. Model and Estimation

Given an independent sample of n observations, consider the linear regression model

$$Y_i=\alpha +{\mathit{X}}_i^{\top }\mathit{\beta }+{\mathit{Z}}_i^{\top }\gamma +{ϵ}_i$$

(7)

for $i\in \{1,\dots ,n\}$, where $Y_i$ is the i-th realization of the response variable $Y\in \mathbb{R}$, ${\mathit{X}}_i^{\top }=(X_{i1},\dots ,X_{ip})$ and ${\mathit{Z}}_i^{\top }=(Z_{i1},\dots ,Z_{iq})$ are the i-th realizations of the predictor vectors ${\mathit{X}}^{\top }=(X_1,\dots ,X_p)\in {\mathbb{R}}^p$ and ${\mathit{Z}}^{\top }=(Z_1,\dots ,Z_q)\in {\mathbb{R}}^q$, and ${ϵ}_i$ is the i-th realization of the error term $ϵ\in \mathbb{R}$. The predictor variables are assumed to be partitioned into two sets, such that $\mathit{X}$ contains the variables of interest for inference purposes, and $\mathit{Z}$ contains the covariates that will be conditioned on. Let ${\mathit{M}}_i^{\top }=({\mathit{X}}_i^{\top },{\mathit{Z}}_i^{\top })$ denote the combined predictor vector, and let $\theta ={({\mathit{\beta }}_1,\dots ,{\mathit{\beta }}_p,{\gamma }_1,\dots ,{\gamma }_q)}^{\top }$ denote the combined coefficient vector. Furthermore, let ${\mathit{M}}_i^c={\mathit{M}}_i-\overline{\mathit{M}}$ denote the mean centered (combined) predictor vector, where $\overline{\mathit{M}}=\frac{1}{n}{\sum }_{i=1}^n{\mathit{M}}_i$ is the sample average of the combined predictor vector.

To estimate the coefficients in $\theta$, consider minimizing the GRR loss function

$$\frac{1}{n}{({\mathit{Y}}_c-{\mathbf{M}}_c\theta )}^{\top }({\mathit{Y}}_c-{\mathbf{M}}_c\theta )+{\theta }^{\top }\mathsf{\Delta }\theta$$

(8)

where ${\mathbf{M}}_c={[{\mathit{M}}_1^c,\dots ,{\mathit{M}}_n^c]}^{\top }$ is the centered design matrix, and $\mathsf{\Delta }$ is an $r\times r$ symmetric and positive semi-definite penalty matrix (where $r=p+q$ is the total number of slope coefficients). The coefficients that minimize Equation (8) can be written as

$${\widehat{\theta }}_{\mathsf{\Delta }}={\left(\frac{1}{n}{\mathbf{M}}_c^{\top }{\mathbf{M}}_c+\mathsf{\Delta }\right)}^{-1}\left(\frac{1}{n}{\mathbf{M}}_c^{\top }{\mathbf{Y}}_c\right)$$

(9)

which is subscripted to emphasize that the estimated coefficients depend on the penalty matrix $\mathsf{\Delta }$. Given the estimated slope vector ${\widehat{\theta }}_{\mathsf{\Delta }}$, the least squares estimate of the intercept has the form $\widehat{\alpha }=\overline{Y}-{\overline{\mathit{M}}}^{\top }{\widehat{\theta }}_{\mathsf{\Delta }}$.

3.2. Asymptotic Distributions

Consider the linear regression model from Equation (7) with the assumptions:

B1.

$Y_i=\alpha +{\mathit{M}}_i^{\top }\theta +{ϵ}_i$ with $E\left({ϵ}_i\right)=0$ and $E\left({ϵ}_i^2\right)={\sigma }_i^2<\infty$ for $i=1,\dots ,n$

B2.

$(Y_i,{\mathit{M}}_i)$ are iid from a distribution satisfying $E\left({ϵ}_i\right|{\mathit{M}}_i)=0$

B3.

${\mathsf{\Sigma }}_M=E\left(({\mathit{M}}_i-{\mu }_M){({\mathit{M}}_i-{\mu }_M)}^{\top }\right)$ and ${\mathsf{\Omega }}_M=E\left({ϵ}_i^2({\mathit{M}}_i-{\mu }_M){({\mathit{M}}_i-{\mu }_M)}^{\top }\right)$ are nonsingular, where ${\mu }_M=E\left({\mathit{M}}_i\right)$, and $\frac{1}{n}{\mathbf{M}}_c^{\top }{\mathbf{M}}_c+\mathsf{\Delta }$ is almost surely invertible. In the fixed predictors case, the mean vector is ${\mu }_M=\frac{1}{n}{\sum }_{i=1}^n{\mathit{M}}_i$ and the covariance matrix terms are defined as ${\mathsf{\Sigma }}_M=\frac{1}{n}{\mathbf{M}}_c^{\top }{\mathbf{M}}_c$ and ${\mathsf{\Omega }}_M=\frac{1}{n}{\mathbf{M}}_c^{\top }\mathsf{\Psi }{\mathbf{M}}_c$, where $\mathsf{\Psi }=\mathrm{diag}({\sigma }_1^2,\dots ,{\sigma }_n^2)$.

Given assumptions B1–B3, the GRR estimator provides an estimate of ${\theta }_{\mathsf{\Delta }}={\mathsf{\Sigma }}_{M\mathsf{\Delta }}^{-1}{\sigma }_{MY}$, where ${\mathsf{\Sigma }}_{M\mathsf{\Delta }}={\mathsf{\Sigma }}_M+\mathsf{\Delta }$ and ${\sigma }_{MY}=E\left(({\mathit{M}}_i-{\mu }_M)(Y_i-{\mu }_Y)\right)$ is the covariance between ${\mathit{M}}_i$ and $Y_i$. Note that ${\theta }_{\mathsf{\Delta }}={\mathsf{\Sigma }}_{M\mathsf{\Delta }}^{-1}{\mathsf{\Sigma }}_M\theta$, where $\theta ={\mathsf{\Sigma }}_M^{-1}{\sigma }_{MY}$ is estimated by the OLS estimator $\widehat{\theta }=\widehat{\mathsf{\Sigma }}{}_M^{-1}{\widehat{\sigma }}_{MY}$ with ${\widehat{\mathsf{\Sigma }}}_M=\frac{1}{n}{\mathbf{M}}_c^{\top }{\mathbf{M}}_c$ and ${\widehat{\sigma }}_{MY}=\frac{1}{n}{\mathbf{M}}_c^{\top }{\mathbf{Y}}_c$. This implies that the GRR estimator can be written as ${\widehat{\theta }}_{\mathsf{\Delta }}=\widehat{\mathsf{\Sigma }}{}_{M\mathsf{\Delta }}^{-1}{\widehat{\mathsf{\Sigma }}}_M\widehat{\theta }$ where $\widehat{\mathsf{\Sigma }}{}_{M\mathsf{\Delta }}={\widehat{\mathsf{\Sigma }}}_M+\mathsf{\Delta }$.

Lemma 3.

Given assumptions B1–B3, the GRR estimator ${\widehat{\theta }}_{\mathsf{\Delta }}$ from Equation (9) is asymptotically normal with mean vector ${\theta }_{\mathsf{\Delta }}$ and covariance matrix $\frac{1}{n}{\mathsf{\Sigma }}_{M\mathsf{\Delta }}^{-1}{\mathsf{\Omega }}_M{\mathsf{\Sigma }}_{M\mathsf{\Delta }}^{-1}$, i.e.,

$$\sqrt{n}\left({\widehat{\theta }}_{\mathsf{\Delta }}-{\theta }_{\mathsf{\Delta }}\right)\stackrel{d}{\to }N\left(\mathbf{0},{\mathsf{\Sigma }}_{M\mathsf{\Delta }}^{-1}{\mathsf{\Omega }}_M{\mathsf{\Sigma }}_{M\mathsf{\Delta }}^{-1}\right)$$

as $n\to \infty$, where the notation $\stackrel{d}{\to }$ denotes convergence in distribution.

Lemma 4.

Consider the linear model in Equation (7) with the assumptions $\theta \sim N({\mathbf{0}}_r,\frac{{\sigma }^2}{n}{\mathsf{\Delta }}^{-1})$ and ${ϵ}_i\stackrel{\mathrm{iid}}{\sim }N(0,{\sigma }^2)$, and suppose that θ and ${ϵ}_i$ are independent of one another. Under these assumptions, the posterior distribution of θ given $\mathit{Y}$ is multivariate normal with mean vector ${\widehat{\theta }}_{\mathsf{\Delta }}$ and covariance matrix $\frac{{\sigma }^2}{n}\widehat{\mathsf{\Sigma }}{}_{M\mathsf{\Delta }}^{-1}$. The asymptotic mean vector is ${\theta }_{\mathsf{\Delta }}$ and the asymptotic covariance matrix is $\frac{{\sigma }^2}{n}{\mathsf{\Sigma }}_{M\mathsf{\Delta }}^{-1}$.

Note that Lemmas 3 and 4 can be proved using analogues of the results that were used to prove Lemmas 1 and 2. Specifically, for Lemma 3, we can use a direct analogue of the proof for Lemma 1 with ${\theta }_{\mathsf{\Delta }}$ replacing ${\mathit{\beta }}_{\mathsf{\Delta }}$ and the matrices ${\mathsf{\Omega }}_M$ and ${\mathsf{\Sigma }}_{M\mathsf{\Delta }}$ replacing the matrices ${\mathsf{\Omega }}_X$ and ${\mathsf{\Sigma }}_{X\mathsf{\Delta }}$. For Lemma 4, we can use a direct analogue of the proof for Lemma 2 with ${\theta }_{\mathsf{\Delta }}$ replacing ${\mathit{\beta }}_{\mathsf{\Delta }}$ and ${\mathsf{\Sigma }}_{M\mathsf{\Delta }}$ replacing ${\mathsf{\Sigma }}_{X\mathsf{\Delta }}$. For both cases, we need to replace the unpenalized coefficient vector $\mathit{\beta }$ with the unpenalized coefficient vector $\theta$.

3.3. Test Statistics

Consider the linear model in Equation (7), and suppose that we want to test the null hypothesis $H_0:\mathit{\beta }={\mathbf{0}}_p$ versus the alternative hypothesis $H_1:\mathit{\beta }\ne {\mathbf{0}}_p$. This is the same null hypothesis that was considered in the previous section, but now the nuisance effects ${\mathit{Z}}_i^{\top }\gamma$ are included in the model (i.e., conditioned on) while testing the significance of the $\mathit{\beta }$ vector. Assuming that $\theta \sim N({\mathbf{0}}_r,\frac{{\sigma }^2}{n}{\mathsf{\Delta }}^{-1})$ and ${ϵ}_i\stackrel{\mathrm{iid}}{\sim }N(0,{\sigma }^2)$ are independent of one another, we could use the F test statistic

$$F=\frac{n}{p}{\widehat{\mathit{\beta }}}_{\mathsf{\Delta }}^{\top }{\left({\widehat{\sigma }}^2\mathbf{S}\widehat{\mathsf{\Sigma }}{}_{M\mathsf{\Delta }}^{-1}{\mathbf{S}}^{\top }\right)}^{-1}{\widehat{\mathit{\beta }}}_{\mathsf{\Delta }}$$

(10)

where $\mathbf{S}=[{\mathbf{I}}_p,{\mathbf{0}}_{p\times q}]$ is a $p\times r$ selection matrix such that $\mathbf{S}{\theta }_{\mathsf{\Delta }}={\mathit{\beta }}_{\mathsf{\Delta }}$ (note that ${\mathbf{I}}_p$ is the $p\times p$ identity matrix and ${\mathbf{0}}_{p\times q}$ is a $p\times q$ matrix of zeros), ${\widehat{\sigma }}^2=\frac{1}{n-r-1}{\sum }_{i=1}^n{\widehat{ϵ}}_i^2$ is the estimated error variance, and ${\widehat{ϵ}}_i=Y_i-{\widehat{Y}}_i$ is the residual with ${\widehat{Y}}_i=\widehat{\alpha }+{\mathit{M}}_i^{\top }{\widehat{\theta }}_{\mathsf{\Delta }}$ denoting the fitted value for the i-th observation.

When $H_0$ is true and the assumptions in Lemma 4 are met, the F statistic approaches an F distribution with degrees of freedom parameters p and $n-r-1$ as $\mathsf{\Delta }\to \mathbf{0}$. For non-zero penalties, the F statistic in Equation (10) will not follow an $F_{p,n-r-1}$ distribution, and may produce asymptotically invalid results when used in a permutation test, especially when the error terms are heteroscedastic (see [44,50,51]). In such cases, the Wald test statistic should be preferred

$$W=n{\widehat{\mathit{\beta }}}_{\mathsf{\Delta }}^{\top }{\left[\mathbf{S}\widehat{\mathsf{\Sigma }}{}_{M\mathsf{\Delta }}^{-1}{\widehat{\mathsf{\Omega }}}_M\widehat{\mathsf{\Sigma }}{}_{M\mathsf{\Delta }}^{-1}{\mathbf{S}}^{\top }\right]}^{-1}{\widehat{\mathit{\beta }}}_{\mathsf{\Delta }}$$

(11)

where ${\widehat{\mathsf{\Omega }}}_M=\frac{1}{n}{\mathbf{M}}_c^{\top }{\mathbf{D}}_{\widehat{ϵ}}^2{\mathbf{M}}_c$ with ${\mathbf{D}}_{\widehat{ϵ}}=\mathrm{diag}({\widehat{ϵ}}_1,\dots ,{\widehat{ϵ}}_n)$. Under assumptions B1–B3, the W statistic asymptotically follows a ${\chi }_p^2$ distribution when $H_0$ is true, which is a result of Lemma 3 (and the consistency of the estimators ${\widehat{\mathsf{\Sigma }}}_M$ and ${\widehat{\mathsf{\Omega }}}_M$).

3.4. Permutation Inference

Table 1. Permutation methods for testing $H_0:\mathit{\beta }={\mathbf{0}}_p$ in the presence of the nuisance parameters $\gamma$. The intercept $\alpha$ is excluded from each model for notational simplicity.

Code	Method	Permutation Method
DS	Draper-Stoneman (1966)	Y	$=PX\mathit{\beta }+Z\gamma +ϵ$
OS	O’Gorman-Smith (2005/8)	Y	$=P{R}_{Z}X\mathit{\beta }+Z\gamma +ϵ$
MA	Manly (1986)	$PY$	$=X\mathit{\beta }+Z\gamma +ϵ$
FL	Freedman-Lane (1983)	$(H_Z+P{R}_Z)Y$	$=X\mathit{\beta }+Z\gamma +ϵ$
TB	ter Braak (1992)	$(H_M+P{R}_M)Y$	$=X\mathit{\beta }+Z\gamma +ϵ$
SW	Still-White (1981)	$P{R}_{Z}Y$	$=X\mathit{\beta }+ϵ$
KC	Kennedy-Cade (1996)	$P{R}_{Z}Y$	$=R_{Z}X\mathit{\beta }+ϵ$
HJ	Huh-Jhun (2001)	$P{Q}^{\prime }{R}_{Z}Y$	$=Q^{\prime }{R}_{Z}X\mathit{\beta }+ϵ$

Notes. P is a permutation matrix. $R_Z=I-H_Z$ where $H_Z=Z{(Z^{\top }Z+n{\mathsf{\Delta }}_Z)}^{-1}{Z}^{\top }$ is the hat matrix with Z in the model. $R_M=I-H_M$ where $H_M=M{(M^{\top }M+n\mathsf{\Delta })}^{-1}{M}^{\top }$ is the hat matrix with $M=(X,Z)$ in the model. For the Huh-Jhun method, $R_Z=Q{Q}^{\top }$ where the columns of Q are mutually orthogonal.

depicts eight different permutation methods that have been proposed for testing the significance of regression coefficients in the presence of nuisance parameters. The eight methods can be split into three different groups: (i) methods that permute the rows of $\mathbf{X}$ [52,53,54], (ii) methods that permute $\mathit{Y}$ with $\mathbf{Z}$ included in the model [55,56,57], and (iii) methods that permute $\mathit{Y}$ after partialling out $\mathbf{Z}$ [58,59,60]. All of these methods were originally proposed for use with the OLS estimator $\widehat{\theta }$ and the F test statistic. Recent works have incorporated the use of the robust W test statistic with these permutation methods [44,50,51]. However, these authors only considered a theoretical analysis of the DS and FL permutation methods, and no previous works seem to have studied these methods using the GRR estimators from Equations (4) and (9).

To understand the motivation of the various permutation methods in Table 1, assume that the penalty matrix is a block diagonal such as $\mathsf{\Delta }=\mathrm{bdiag}({\mathsf{\Delta }}_X,{\mathsf{\Delta }}_Z)$, where ${\mathsf{\Delta }}_X$ and ${\mathsf{\Delta }}_Z$ denote the penalty matrices for $\mathit{\beta }$ and $\gamma$, respectively. Using the well-known form for the inverse of a block matrix [61,62,63,64], the coefficient estimates from Equation (9) have the form

$$\begin{array}{cc}\hfill {\widehat{\mathit{\beta }}}_{\mathsf{\Delta }}& ={\left({\mathbf{X}}_c^{\top }{\mathbf{R}}_Z{\mathbf{X}}_c+n{\mathsf{\Delta }}_X\right)}^{-1}{\mathbf{X}}_c^{\top }{\mathbf{R}}_Z{\mathit{Y}}_c\hfill \\ \hfill {\widehat{\gamma }}_{\mathsf{\Delta }}& ={\left({\mathbf{Z}}_c^{\top }{\mathbf{Z}}_c+n{\mathsf{\Delta }}_Z\right)}^{-1}{\mathbf{Z}}_c^{\top }\left({\mathit{Y}}_c-{\mathbf{X}}_c{\widehat{\mathit{\beta }}}_{\mathsf{\Delta }}\right)\hfill \end{array}$$

where ${\mathbf{R}}_Z={\mathbf{I}}_n-{\mathbf{Z}}_c{({\mathbf{Z}}_c^{\top }{\mathbf{Z}}_c+n{\mathsf{\Delta }}_Z)}^{-1}{\mathbf{Z}}_c^{\top }$ is the residual forming matrix for the model that only includes the nuisance effects in the model, i.e., $\mathit{Y}=\alpha +{\mathit{Z}}^{\top }\gamma +ϵ$. This implies that the (centered) fitted values can be written as ${\widehat{\mathit{Y}}}_c={\mathbf{X}}_c{\widehat{\mathit{\beta }}}_{\mathsf{\Delta }}+{\mathbf{Z}}_c{\widehat{\gamma }}_{\mathsf{\Delta }}={\mathbf{R}}_Z{\mathbf{X}}_c{\widehat{\mathit{\beta }}}_{\mathsf{\Delta }}+{\mathbf{H}}_Z{\mathit{Y}}_c$, where ${\mathbf{H}}_Z={\mathbf{Z}}_c{\left({\mathbf{Z}}_c^{\top }{\mathbf{Z}}_c+n{\mathsf{\Delta }}_Z\right)}^{-1}{\mathbf{Z}}_c^{\top }$ is the hat matrix for the linear model that only includes the nuisance effects. Thus, when ${\mathsf{\Delta }}_Z={\mathbf{0}}_{q\times q}$, all of the permutation methods except SW will produce the same observed F statistic (when the permutation matrix is $\mathbf{P}={\mathbf{I}}_n$).

Consider the additional assumption that the response and predictor variables have finite fourth moments, i.e., B4: $E\left(Y_i^4\right)<\infty$, $E\left(X_{ij}^4\right)<\infty \forall j$, and $E\left(Z_{ik}^4\right)<\infty \forall k$. Assume B1–B4 and that the null hypothesis $H_0:\mathit{\beta }={\mathbf{0}}_p$ is true. Using the W test statistic from Equation (11), the following can be said about the finite sample and asymptotic properties of the various permutation methods in Table 1: the DS method is exact when $\mathit{X}\perp \perp (Y,\mathit{Z})$ and asymptotically valid otherwise; the MA method is exact when $Y\perp \perp (\mathit{X},\mathit{Z})$ and asymptotically valid otherwise; the SW method is inexact and asymptotically valid only when $E\left((\mathit{X}-{\mu }_X){(\mathit{Z}-{\mu }_Z)}^{\top }\right)={\mathbf{0}}_{p\times q}$; the other five methods (OS, FL, TB, KC, HJ) are inexact and asymptotically valid.

The asymptotic behaviors of the DS and FL methods were proved by DiCiccio and Romano [44]. The asymptotic validity of the OS method can be proved using a similar result as used for the DS method, given that $\frac{1}{n}{\mathbf{X}}_c^{\top }{\mathbf{R}}_Z{\mathbf{X}}_c$ is a consistent estimator of ${\mathsf{\Sigma }}_X-{\mathsf{\Sigma }}_{XZ}{\mathsf{\Sigma }}_{Z\mathsf{\Delta }}^{-1}{\mathsf{\Sigma }}_{ZX}$ and $\frac{1}{n}{\mathbf{X}}_c^{\top }{\mathbf{R}}_Z{\mathit{Y}}_c$ is a consistent estimator of ${\sigma }_{XY}-{\mathsf{\Sigma }}_{XZ}{\mathsf{\Sigma }}_{Z\mathsf{\Delta }}^{-1}{\sigma }_{ZY}$. The asymptotic validity of the MA method can also be proved using a similar result as used for the DS method. The asymptotic validity of the TB method can be proved using a similar result as used for the FL method, given that $\frac{1}{n}{\mathbf{M}}_c^{\top }{\mathbf{R}}_M{\mathbf{M}}_c$ is a consistent estimator of ${\mathsf{\Sigma }}_M-{\mathsf{\Sigma }}_M{\mathsf{\Sigma }}_{M\mathsf{\Delta }}^{-1}{\mathsf{\Sigma }}_M$ and $\frac{1}{n}{\mathbf{M}}_c^{\top }{\mathbf{R}}_M{\mathit{Y}}_c$ is a consistent estimator of ${\sigma }_{MY}-{\mathsf{\Sigma }}_M{\mathsf{\Sigma }}_{M\mathsf{\Delta }}^{-1}{\sigma }_{MY}$. Finally, note that the KC and HJ methods are asymptotically equivalent, and the SW method is asymptotically equivalent to the KC and HJ methods when $\mathit{X}$ and $\mathit{Z}$ are uncorrelated. The asymptotic validity of the KC and HJ methods follow from the results in Theorem 1, given that these methods permute the response after partialling out the nuisance effects. It is important to note that if $\mathit{X}$ and $\mathit{Z}$ are correlated, the SW method will produce asymptotically invalid results because $\mathit{Z}$ is partialled out of $\mathit{Y}$ but not $\mathit{X}$.

4. Simulation Studies

4.1. Simulation A

The first simulation study was designed to explore the validity of the claims made about the omnibus permutation tests discussed in Section 2. Simulation A was a fully crossed design that manipulated three design factors: (i) the data generating distribution (three levels: left skew normal, standard normal, right skew normal), (ii) the error standard deviation (three levels: constant, increasing, and parabolic), and (iii) the sample size (five levels: $n\in \{10,25,50,100,200\}$), see Figure 1. For each of the 45 combinations of simulation design parameters (3 $F_{ϵ}$× 3 $\sigma$× 5 n), I generated 10,000 independent copies of the data from the model in Equation (1) with $X_i=(i-1)/(n-1)$ and $\eta \left(X_i\right)=0$ (so that $Y_i={ϵ}_i$). For each generated sample of data, the ss() function in the npreg R package [65] was used to fit a cubic smoothing spline with $r=5$ knots, which were placed at the quantiles of the predictor scores. The generalized cross-validation (GCV) criterion [33] was used to select the smoothing parameter $\lambda$.

Ten different inference methods (six permutation tests and four parametric tests) were used to test the null hypothesis of no functional relationship. The six permutation tests were formed using the two test statistics (F and W) combined with the three permutation methods discussed in Section 2: permute Y, sign-flip Y, and both permuting and sign flipping. The permutation tests were implemented using the np.reg.test() function in the nptest R package [66] using the default number of resamples (the default uses $R=min(R_0,9999)$ resamples, where $R_0$ is the number of elements of the exact null distribution: $R_0=n!$ for the permute method, $R_0=2^n$ for the sign-flip method, and $R_0=n!2^n$ for the combined method). The first two parametric tests were formed by comparing the F statistic from Equation (5) to an $F_{p,n-p-1}$ distribution, and the W statistic from Equation (6) to a ${\chi }_p^2$ distribution. The other two parametric tests were the F tests that are implemented by the mgcv R package [67] and the npreg R package [65]. Note that these F tests compare the F statistic from Equation (5) to an $F_{\nu ,n-\nu -1}$ distribution, where $\nu$ is an estimate of the effective degrees of freedom. The npreg package defines $\nu$ as the trace of the smoothing matrix, whereas the mgcv package uses a more complex estimate of $\nu$ (see [42]).

Figure 2 displays the type I error rate for each inference method in each combination of Simulation A conditions. All of the inference methods perform similarly across the three data generating distributions, so the following discussions of the results apply to each of the three data generating distributions. When using the W test statistic, the three permutation methods produced accurate type I error rates for all combinations of data generating conditions (see red square, blue circle, and green triangle), and the parametric ${\chi }_p^2$ approximation produced asymptotically accurate results (see purple plus sign). When using the F test statistic, the permutation methods produced inflated type I error rates (see orange x, yellow diamond, and brown upside-down triangle), and the parametric $F_{p,n-p-1}$ approximation produced inconsistent results across the different error standard deviation conditions (see pink square with x). The parametric $F_{\nu ,n-\nu -1}$ tests implemented by the mgcv and npreg packages also produce inconsistent results across the different error standard deviation conditions, such that the type I error rate is inflated in the increasing and parabolic conditions (see gray asterisk and black diamond with plus sign).

4.2. Simulation B

The second simulation study was designed to explore the validity of the claims made about the conditional permutation tests discussed in Section 3. Simulation B was a fully crossed design that manipulated two design factors: (i) the error standard deviation (three levels: constant, increasing, and parabolic), and (ii) the sample size (five levels: $n\in \{10,25,50,100,200\}$). Given that the results in Simulation A did not noticeably differ across the three data generating distributions, errors were generated from a normal distribution throughout Simulation B. For each of the 15 combinations of data generating parameters (3 $\sigma$× 5 n), I generated 10,000 independent copies of the data from the model in Equation (1) with $X_i=(i-1)/(n-1)$ and $\eta \left(X_i\right)=X_i$ (unlike Simulation A, the data generating mean function now includes a linear effect). As in the previous simulation, the ss() function in the npreg R package [65] was used to fit a cubic smoothing spline with $r=5$ knots, and the GCV criterion was used to select the smoothing parameter.

The null hypothesis of a linear relationship was tested using a total of 19 inference methods: 16 permutation tests and 3 parametric tests. The 16 permutations tests were formed using the two test statistics (F and W) combined with the eight permutation methods in Table 1. As in Simulation A, the permutation tests were implemented using the np.reg.test() function in the nptest R package [66] using the default number of resamples (i.e., $R=9999$) to form the permutation distribution. The first two parametric tests were formed by comparing the F statistic from Equation (10) to an $F_{p,n-r-1}$ distribution, and the W statistic from Equation (11) to a ${\chi }_p^2$ distribution. The other parametric test is the F test that is implemented by the npreg package, which compares the F statistic from Equation (10) to an $F_{\nu -m,n-\nu -1}$ distribution. Note that the “cardinal” spline parameterization used in the mgcv R package does not separate the linear and non-linear portions of the function; thus, a test of linearity is not possible using this package.

Figure 3 displays the type I error rate for each inference method in each combination of Simulation B conditions. When the errors have constant variance, using the F statistic produces (i) inflated type I error rates using all permutation methods, (ii) deflated type I error rates using the parametric test, and (iii) asymptotically accurate error rates using the npreg test. In contrast, when using the W test statistic, all of the inference methods except SW produce asymptotically accurate error rates when the errors have constant variance. As expected, the F statistic produces asymptotically invalid results when the errors have non-constant variance, with the performance being the worst in the parabolic condition. In contrast, the W statistic produced asymptotically valid type I error rates when the errors have non-constant variance (using all methods except SW).

5. Discussion

5.1. Summary of Findings

Penalized splines are frequently used in applied research for understanding functional relationships between variables. Although there has been a considerable body of work on the estimation and computation of penalized splines, there have been relatively few papers on statistical inference about the nature of the functional relation. In most applications, the statistical inference for penalized splines is conducted using the random effects or Bayesian interpretation of a smoothing spline. These inferential frameworks rely on parametric assumptions about the unknown function and error terms, i.e., that $\eta$ is a Gaussian process and ${ϵ}_i$ are iid Gaussian variables. Even when these parametric assumptions are met, valid statistical inference can be challenging due to the need for a reasonable estimate of the degrees of freedom of the penalized spline estimate. Furthermore, when these parametric assumptions are incorrect (e.g., due to heteroscedastic and/or non-Gaussian errors), the standard inferential tools can produce substantially misleading results.

In this paper, I developed a flexible and robust permutation testing framework for inference using penalized splines. Unlike the existing methods for statistical inference with penalized splines, the proposed methods can provide asymptotically valid inferential results under a variety of data generating situations. Furthermore, unlike a majority of the existing methods for hypothesis testing with penalized splines, the proposed approach can be flexibly adapted to test a variety of standard and non-standard hypotheses about the nature of the functional relationships between variables. The omnibus test in Section 2 is frequently of interest in practical applications, but has been ignored by most inferential tests for penalized splines (e.g., see [42]). The conditional test in Section 3 can be used to test the classic hypothesis $\eta \in {\mathcal{H}}_0$ (which has been considered in many papers), as well as the more specialized hypotheses about the main and/or interaction effects of predictors.

The simulation results in Section 4 clearly demonstrate the benefits of the proposed approach over standard methods used for inference with penalized splines. In particular, the simulation results demonstrate that classic (parametric) methods that rely on an F test statistic can produce substantially inflated type I error rates when the error terms are heteroscedastic. Furthermore, the simulation results demonstrate that the F test statistic can even produce inaccurate results when used in a permutation test. In contrast, the permutation tests using the robust W test statistic can produce exact results for the omnibus tests and asymptotically valid results for the conditional tests—even when the errors depend on the predictor(s). Moreover, the ${\chi }_p^2$ approximation using the robust W statistic produced asymptotically valid results that were reasonably close to the nominal $\alpha =0.05$ rate with only $n=100$ (for the omnibus test) or $n=200$ (for the conditional test).

5.2. Future Directions

Although the simulation studies only explored the omnibus and conditional tests with a single predictor, the theoretical results in Section 2 and Section 3 can be readily applied to penalized spline models with multiple predictors. With $d>1$ predictors, the number of possible hypotheses that could be tested increases, given that it is possible to test omnibus or conditional tests about any of the main or interaction effects. The simulation results for a single predictor revealed that the permutation methods of Kennedy and Cade [59] and Huh and Jhun [60] performed the best for conditional tests with a single predictor, so I hypothesize that these procedures will be ideal for tests of main and/or interaction effects with $d>1$ predictors. However, future theoretical and Monte Carlo simulation work is needed to determine which permutation strategies should be preferred for conditional tests of main and/or interaction effects of multiple predictors.

The theoretical results in Section 2 and Section 3 were developed for the classic formulation of penalized splines, which solves a GRR problem to obtain the function estimate. Note that the penalized spline GRR problem is a special case of the elastic net penalty [68] that is used in the recently proposed kernel eigenvector smoothing and selection operator (kesso) regression method [26]. Although the theorems derived in Section 2 and Section 3 assume a ridge penalty, it is straightforward to develop extensions of these formulas for elastic net penalties used in the kesso regression. Specifically, the vector version of the coefficients, which is given after Equation (8) in [26], can be used to express the kesso coefficients as a modification of the least squares coefficients. However, future Monte Carlo research is needed to explore the accuracy of the various permutation strategies when using elastic net penalties to fit penalized spline regression models.

Finally, it is worth noting that this paper only considered the GCV tuning method, which is one of several smoothing parameter selection methods available in the ss() function. Recent work has demonstrated that the different tuning methods tend to produce similar results across a wide variety of data generating conditions—including non-normal and/or heteroscedastic errors (see [25]). However, it was noted that the maximum likelihood based tuning method tended to perform (i) worse than the cross-validation based methods when the sample size was small and (ii) better than the cross-validation based methods when the errors were correlated. Given these past findings, I would not expect the simulation results in this paper to significantly change if a different tuning criterion were used instead of the GCV. However, future research should explore the performance of the proposed permutation testing approach using different combinations of tuning methods and permutation strategies to analyze data from a wide variety of data-generating conditions.