A Surrogate Measure for Time-Varying Biomarkers in Randomized Clinical Trials

Abstract

Clinical trials with rare or distant outcomes are usually designed to be large in size and long term. The resource-demand and time-consuming characteristics limit the feasibility and efficiency of the studies. There are motivations to replace rare or distal clinical endpoints by reliable surrogate markers, which could be earlier and easier to collect. However, statistical challenges still exist to evaluate and rank potential surrogate markers. In this paper, we define a generalized proportion of treatment effect for survival settings. The measure’s definition and estimation do not rely on any model assumption. It is equipped with a consistent and asymptotically normal non-parametric estimator. Under proper conditions, the measure reflects the proportion of average treatment effect mediated by the surrogate marker among the group that would survive to mark the measurement time under both intervention and control arms.

1. Introduction

The HPTN (HIV Prevention Trial Network) 052 study is an HIV prevention trial conducted across several continents. The primary clinical endpoint of interest is that HIV infection is estimated to have a rate of 3–5% in modern clinical trial settings. In addition to this, the time from viral exposure to infection is long. The median infection time on average is more than one year. It is desirable to replace the clinically meaningful endpoint by an earlier and more easily accessible alternative endpoint.

A surrogate marker in clinical trials is considered to be “a laboratory measurement or physical sign used as a substitute for a clinically meaningful endpoint that measures directly how a patient feels, functions, or survives and that is expected to predict the effect of the therapy” [1]. It is considered to be valid if one could correctly conclude treatment effect on the clinical endpoint by using the marker [2,3]. In this context, how to validate surrogate markers for a clinically meaningful endpoint are especially important. Zhuang and Chen [4] review the surrogate measures in clinical research on their strengths and limitations in details.

A surrogate marker is considered to be valid if one could correctly conclude the treatment effect on the clinical endpoint by using that marker. In the language of hypothesis testing, that is, departure from the null hypothesis $P(T\mid Z)=P\left(T\right)$ is captured by departure from the null hypothesis $P(S\mid Z)=P\left(S\right)$, where Z, S, and T represent the intervention, marker, and clinical endpoint, respectively. Prentice [5] operationalized the idea to test $P(T\mid S,Z)=P(T\mid S).$ The conditional independence of T and Z represents an ideal situation when the marker fully mediates the treatment effect. However, many candidate markers may only capture part of the treatment effect, so that to which extent a marker captures the treatment effect is of great practical importance. Freedman et al. [6] further extended Prentice’s criterion and evaluated the strength of surrogate markers by comparing the treatment effect with and without adjusting for the marker. For a binary endpoint T and logistic models:

$$\begin{array}{ccc}\hfill logit(T\mid Z)& =& {\mu }_1+\beta Z,\hfill \\ \hfill logit(T\mid Z,S)& =& {\mu }_2+{\beta }_{s}Z+{\varphi }_{z}S,\hfill \end{array}$$

the proportion of treatment effect explained (PTE) is defined as $PTE=1-{\beta }_s/\beta .$ However, the adjusted and unadjusted models do not hold simultaneously in general model classes [7,8,9], and the assumption of no interaction in the adjusted model is not necessarily true. To avoid model dependence, Wang and Taylor [9] proposed the F-measure in a general setting as $F=(AA-AB)/(AA-BB),$ where $AA=h\left({\int }_s{g}_A\left(s\right)d{P}_A\left(s\right)\right),AB=h\left({\int }_s{g}_A\left(s\right)d{P}_B\left(s\right)\right),$ and $BB=h\left({\int }_s{g}_B\left(s\right)d{P}_B\left(s\right)\right).$ Here, $P_A\left(s\right)$ and $P_B\left(s\right)$ are the distributions of surrogate marker S in the treatment group A and the control group B, respectively. The functions $g_A\left(s\right)$ and $g_B\left(s\right)$ are functions of the conditional distribution of the primary endpoint given S in the two groups. The functions $h(·)$, $g_A\left(s\right)$ and $g_B\left(s\right)$ are chosen such that $AA-BB$ is the desired measure of treatment effect on the primary endpoint. The F-measure framework is flexible while preserving the flavor of comparing the marginal treatment effect and adjusted treatment effect.

In this paper, we bring in the time dimension and define a generalized F-measure for time-to-event outcomes and time-varying internal surrogate markers explicitly. In Section 2, we introduce the time-varying F-measure. We show the measure can be estimated using a non-parametric estimator that is consistent and asymptotically normal. In Section 3, we give examples to visualize the change of F-measure with time and conduct Monte Carlo simulation studies to evaluate the proposed non-parametric estimation and inference. In Section 4, we apply the time-varying F-measure to an HIV prevention trial for illustration. Finally, we conclude the paper with a discussion in Section 5 and a conclusion in Section 6.

2. The Time-Varying F-Measure

2.1. Definition

We introduce the time-varying F-measure in this section. The new measure does not rely on any model assumption. In addition, it reflects Prentice’s criterion and describes the degree to which a marker captures the treatment effect on the clinical endpoints.

We consider intervention groups $Z=1$ and $Z=0$. Let T represent the time-to-event outcome and $X_t$ represents the value of a candidate marker measured at time point t (after randomization). The time-varying F-measure is formulated to evaluate the marker when survival status at time point $c(c>t)$ is of primary interest. We choose $h\left(u\right)=u,g_z\left(x\right)=P(T\ge c\mid X_t=x,T\ge t,Z=z),P_z\left(s\right)=P(T\ge c\mid T\ge t,Z=z)$. Then, the F-measure for a time-to-event outcome T is:

$$F(c,t)=\frac{AA-AB}{AA-BB},$$

where

$$\begin{array}{cc}\hfill AA& =P(T\ge c\mid T\ge t,Z=1),\hfill \\ \hfill BB& =P(T\ge c\mid T\ge t,Z=0),\hfill \\ \hfill AB& =\sum _{x}P(T\ge c\mid X_t=x,T\ge t,Z=1)P(X_t=x\mid T\ge t,Z=0).\hfill \end{array}$$

It is a function of time point c when the survival status is of primary interest, and time point t when the surrogate marker is measured. The definition and estimation do not necessarily rely on any model assumption and are exempt from model misspecification.

The time-varying F-measure reflects Prentice’s criterion. Namely, the scenarios of perfect markers, in which a marker mediates all the treatment effect, lead to $F(c,t)=1$; the scenarios of useless markers, in which a marker does not mediate any treatment effect or is independent of intervention in the group of interest, leads to $F(c,t)=0.$ In addition, when the treatment effect mediated by the marker is consistent with the direct treatment effect, the F-measure for a partial marker is guaranteed to be bounded within (0,1). A value outside the ideal bound indicates treatment effects via different pathways are not in the same direction so that the marker is not an appropriate surrogate. (Theoretic results are deferred to Section 2.3.)

In summary, the time-varying F-measure evaluates the relative position of the survival probability adjusted by eliminating the treatment effect on a biomarker. It serves as a model-free metric for assessing the proportion of treatment effect explained by the marker.

2.2. Estimation and Inference

In the time-varying F-measure, survival probabilities can be estimated by the non-parametric Kaplan–Meier estimator [10]. Under the assumption of random censoring, the conditional probability $p_{x|0}:=P(X_t=x\mid T\ge t,Z=0)$ can be estimated by the empirical distribution. Naturally, we propose a plug-in estimator for the defined time-varying F-measure:

$$\widehat{F}=\frac{\widehat{s_1}-{\sum }_x\widehat{s_{1x}}·\widehat{p_{x|0}}}{\widehat{s_1}-\widehat{s_0}},$$

(1)

where $\widehat{s_z}(z=0,1)$ and $\widehat{s_{1x}}$ are the Kaplan–Meier estimator for $P(T\ge c\mid T\ge t,Z=z)$ and $P(T\ge c\mid X_t=x,T\ge t,Z=1)$, respectively. Let $u_{z1}<u_{z2}<\dots$ be the ordered, distinct times observed on arm z; $n_z\left(\tau \right)$ be the number of subjects at risk set at time $\tau$ on arm z; and $d_z\left(\tau \right)$ be the number of events at time $\tau$ on arm z. The Kaplan–Meier estimator of survival probabilities reads:

$$\widehat{s_z}=\prod _{t<u_{zk}\le c}\left(1-\frac{d_z\left(u_{zk}\right)}{n_z\left(u_{zk}\right)}\right).$$

Similarly, $s_{1x}:=Pr(T\ge c\mid X_t=x,T\ge t,Z=1)$ can be estimated by the Kaplan–Meier estimator in the strata by $Z=1$ and $X_t=x$ as:

$$\widehat{s_{1x}}=\prod _{t<u_{1xk}\le c}\left(1-\frac{d_{1x}\left(u_{1xk}\right)}{n_{1x}\left(u_{1xk}\right)}\right).$$

Under the assumption of random censoring, $p_{x|0}:=P(X_t=x\mid T\ge t,Z=0)$ can be estimated by the empirical distribution as:

$$\widehat{p_{x|0}}=\frac{n_{0x}\left(t\right)}{n_0\left(t\right)},$$

where $n_{0x}\left(t\right)={\sum }_{i=1}^{n}I(X_t=x,T\ge t,Z=0)$.

We show the proposed estimator converges weakly to a Gaussian process under the following regularity assumptions (Proof of the theorem is deferred to Appendix A).

Assumption A1.

The time c is in a range of $(t,\tau )$ for some constant $t>0$, $0<\tau <\infty$ such that $s_1\left(\tau \right)s_0\left(\tau \right)>0$ and $1-(1-H\left({\tau }_{-}\right))(1-G\left({\tau }_{-}\right))<1$, where H is the distribution function of time-to-event T and G is the distribution function of censoring time U.

Assumption A2.

Survival probabilities $s_1(·)\ne s_0(·)$ on $(0,\tau )$.

Assumption A3.

Random censoring: The censoring time U is independent of both the failure time T and time-varying covariates $X_t$ on $(0,\tau )$.

Theorem 1.

Under regularity Assumptions A1–A3, given a time t, $\sqrt{n_t}(\widehat{F}\left(c\right)-F\left(c\right))$ converges weakly to a zero-mean Gaussian process with covariance function $E\left(\zeta \left(c\right)\zeta \left(c^{\prime }\right)\right)$ between time points c and $c^{\prime }$, where:

$$\zeta \left(c\right)=\frac{s_1-{\sum }_x{p}_{x|0}{s}_{1x}}{{(s_1-s_0)}^2}·{\eta }_0+\frac{{\sum }_x{p}_{x|0}{s}_{1x}-s_0}{{(s_1-s_0)}^2}·{\eta }_1+\frac{1}{s_0-s_1}\sum _x{p}_{x|0}·{\eta }_{1x}+\frac{1}{s_0-s_1}\sum _x{s}_{1x}{\eta }_{0x}^p,$$

and

$$\begin{array}{ccc}\hfill {\eta }_0& =& -s_0\left(c\right)I(Z=0\mid T\ge t){\int }_t^c\frac{dN\left(u\right)-Y\left(u\right)d{\mathsf{\Lambda }}_0\left(u\right)}{E\left(I(Z=0\mid T\ge t)Y\left(u\right)\right)},\hfill \\ \hfill {\eta }_1& =& -s_1\left(c\right)I(Z=1\mid T\ge t){\int }_t^c\frac{dN\left(u\right)-Y\left(u\right)d{\mathsf{\Lambda }}_1\left(u\right)}{E\left(I(Z=1\mid T\ge t)Y\left(u\right)\right)},\hfill \\ \hfill {\eta }_{1x}& =& -s_1\left(c\right|X_t=x,T\ge t)I(Z=1,X_t=x\mid T\ge t){\int }_t^c\frac{dN\left(u\right)-Y\left(u\right)d{\mathsf{\Lambda }}_{1x}\left(u\right)}{E\left(I(Z=1,X_t=x\mid T\ge t)Y\left(u\right)\right)},\hfill \\ \hfill {\eta }_{0x}^p& =& \frac{1}{p_0}\left(I(X_t=x,Z=0\mid T\ge t)-p_{0x}\right)-\frac{p_{0x}}{p_0^2}\left(I(Z=0\mid T\ge t)-p_0\right).\hfill \end{array}$$

In the above equations, $N\left(u\right):=I(T\le U,T\le u)$ denote the observed counting process and $Y\left(u\right):=I(T\ge u,U\ge u)$ the at-risk process. The covariance function $E\left(\zeta \left(c\right)\zeta \left(c^{\prime }\right)\right)$ can be consistently estimated by $1/n_t{\sum }_{i=1}^{n_t}\widehat{{\zeta }_i}\left(c\right)\widehat{{\zeta }_i}\left(c^{\prime }\right),$ where $\widehat{{\zeta }_i}(·)$ is the sample versions of $\zeta (·)$.

2.3. Ranges of F-Measure

2.3.1. Perfect Marker

When the marker mediates the entire treatment effect, we have $P(T\ge c\mid X_t=x,T\ge t,Z=1)=P(T\ge c\mid X_t=x,T\ge t,Z=0).$ It implies ${\sum }_{x}P(T\ge c\mid X_t=x,T\ge t,Z=1)P(X_t=x\mid T\ge t,Z=0)={\sum }_{x}P(T\ge c\mid X_t=x,T\ge t,Z=0)P(X_t=x\mid T\ge t,Z=0),$ and furthermore, $F(c,t)=1.$

2.3.2. Useless Marker

When the marker does not mediate any treatment effect, we have $P(T\ge c\mid X_t=x_1$, $T\ge t,Z=1)=P(T\ge c\mid X_t=x_2,T\ge t,Z=1);$ when the intervention is independent of $X_t$ in the risk set at time point t, we have $P(X_t=x\mid T\ge t,Z=1)=P(X_t=x\mid T\ge t,Z=0).$ Either of the above useless marker conditions leads to ${\sum }_{x}P(T\ge c\mid X_t=x,T\ge t,Z=1)P(X_t=x\mid T\ge t,Z=1)={\sum }_{x}P(T\ge c\mid X_t=x,T\ge t,Z=1)P(X_t=x\mid T\ge t,Z=0),$ and furthermore, $F(c,t)=0.$

2.3.3. Partial Marker

Without loss of generality, we consider the case $AA-BB>0.$ Theorem 2 stated below and its proof at Appendix B are naturally extendable for the counterpart case $AA-BB<0.$ To give interpretability and links to common instances in clinical trials, we impose three mild assumptions:

Assumption A4.

$X_t$ in the treatment group and that in the control group are stochastically ordered, $P(X_t\le x\mid T\ge t,Z=1)\prec P(X_t\le x\mid T\ge t,Z=0)\forall x,$ or $P(X_t\le x\mid T\ge t,Z=1)\succ P(X_t\le x\mid T\ge t,Z=0)\forall x.$

Assumption A5.

$P(T\ge c\mid X_t=x,T\ge t,Z=z)$ is monotone with x in the same direction for any given z.

Assumption A6.

$P(T\ge c\mid X_t=x,T\ge t,Z=z)$ is monotone with z in the same direction for any given x.

In addition, we formulate three conditions:

C1.

$P(T\ge c\mid X_t=x,T\ge t,Z=1)-P(T\ge c\mid X_t=x,T\ge t,Z=0)>0$.

C2.

$P(X_t\le x\mid T\ge t,Z=1)\prec P(X_t\le x\mid T\ge t,Z=0)\forall x$ and $P(T\ge c\mid X_t=x,T\ge t,Z=1)$ is increasing with $x.$

C3.

$P(X_t\le x\mid T\ge t,Z=1)\succ P(X_t\le x\mid T\ge t,Z=0)\forall x$ and $P(T\ge c\mid X_t=x,T\ge t,Z=1)$ is decreasing with $x.$

Theorem 2.

With Assumptions A4–A6, if Condition C1 is satisfied, then $F<1$; if either Condition C2 or C3 is satisfied, then $F>0.$

2.4. Causal Interpretation

The F-measure is closely related with the concept of natural indirect effect, which is defined in the counterfactual framework [11,12]. We formulate Theorem 3 revealing the link with detailed proof in Appendix C.

Assumption A7.

$\{T(Z=1,X_t=x),X_t(Z=1)\}\perp \{T(Z=0,X_t=x),X_t(Z=0)\}$.

Assumption A8.

$Z\perp \{T(Z=z,X_t=x),X_t(Z=z)\}$.

Assumption A9.

$X_t(Z=1)\perp T(Z=1,X_t=x)\mid T\ge t,Z=1$.

Theorem 3.

Under Assumptions A7–A9, it holds that:

$$\begin{array}{c}\hfill F=\frac{P(T\left(1\right)\ge c\mid T\left(1\right)\ge t,T\left(0\right)\ge t)-P(T(1,X_t=X_t\left(0\right))\ge c\mid T\left(1\right)\ge t,T\left(0\right)\ge t)}{P\left(T\right(1)\ge c\mid T(1)\ge t,T(0)\ge t)-P\left(T\right(0)\ge c\mid T(1)\ge t,T(0)\ge t)}.\end{array}$$

For the subgroup of $T\left(1\right)\ge t,T\left(0\right)\ge t,$ F-measure’s numerator describes the natural indirect effect mediated by the surrogate marker while the denominator describes the average treatment effect. The ratio reflects the proportion of the average treatment effect mediated by the surrogate measure (in the sense of natural direct effect) for the subgroup surviving to marker measurement anyway. However, we also note that the causal interpretation does not apply in general [13].

3. Numerical Studies

To assess the proposed surrogate measure, we conduct numerical studies motivated by the HIV Prevention Trial Network. The plasma HIV-1 viral load represents the degree of viral burden and is believed to play a crucial role in mediating the benefit of antiretroviral therapy (ART) on HIV-related disease progression and transmission. We consider a viral load measurement dichotomized by a threshold of 1000 copies per cubic millimeter as the biomarker of interest. In a hypothetical scenario, participants have some HIV-1 exposure at the enrollment. The viral load level may increase fast in the follow-up while an effective intervention could delay the virus proliferation and further suspend the failure time. We express the above scenario in the following mathematical models. The dichotomous viral load level at time t is modeled as $X_t=I(t\ge t_s)$, where $t_s$ denotes the time when one’s viral load shifts from level 0 to 1 after enrollment. We assume $t_s$ follows an exponential distribution with mean ${\mu }_z$ in intervention group z, and a time-varying Cox–Weibull model:

$$\begin{array}{ccc}\hfill h(t\mid X_t,Z)& =& h_0\left(t\right)exp(b_{1}Z+b_2{X}_t),\hfill \end{array}$$

(2)

$$\begin{array}{ccc}\hfill h_0\left(t\right)& =& \lambda v{t}^{v-1},\hfill \end{array}$$

(3)

where Z is Bernoulli with success probability of 0.5.

3.1. Numerical Examples

In Figure 1, we explore the numerical behavior of the time-varying F-measure under the motivation scenario described above. In particular, we assume Model (2) with a constant baseline hazard where $h_0\left(t\right)=0.2.$ Without loss of generality, we assume $b_1\le 0,b_2\ge 0,$ and $c=5$. With the model assumption, the F-measure has a closed-form formula with details in the Appendix D. Figure 1a has $b_1=-1,t_0=0.5,t_1=2,$ varying $b_2$ visualized F-measures curves from an useless marker with $b_2=0$ to partial markers with $b_2>0.$ Figure 1b has $b_2=1,t_0=0.5,t_1=2,$ varying $b_1$ gives the F-measure curves from a perfect marker with $b_1=1$ to partial markers with $b_1<0.$

3.2. Monte–Carlo Simulation

In this section, we describe our Monte–Carlo simulation to evaluate the proposed non-parametric estimator. We generate failure times for each z-group based on a closed-form approach described in Austin [14]: First, a random value u is generated from the Uniform(0, 1) distribution and the subject-specific shift time $t_s$ is generated from an exponential distribution with mean ${\mu }_z$; second, if $-log\left(u\right)<\lambda exp\left(b_{1}z\right)t_s^v$, we let failure time $T={\left(-log\left(u\right)/\left(\lambda exp\left(b_{1}z\right)\right)\right)}^{1/v};$ otherwise, $T={\left(\left(-log\left(u\right)-\lambda exp\left(b_{1}z\right)t_s^v+\lambda exp\left(b_2\right)exp\left(b_{1}z\right)t_s^v\right)/\left(\lambda exp\left(b_2\right)exp\left(b_{1}z\right)\right)\right)}^{1/v}.$ In addition, we generate the censoring times from Uniform $(0,\tau )$, in which $\tau$ is chosen to give a censoring rate of $20\%$. The censoring is independent of the failure time T, the covariates Z and $X_t$.

Table 1. Simulation results under Cox–Weibull distribution. The sample size of the study is 20,000 subjects and the coverage probability is obtained by 1000 replicates.

c = 5, v = 0.8
Scenario	Marker time ${}^{\mathrm{a}}$	True value	Bias	Sampling SE	Mean of SE	Coverage
Perfect	0.25	0.747	0.003	0.052	0.051	0.941
	0.5	0.932	0.002	0.054	0.055	0.956
	1	0.995	0.003	0.059	0.059	0.953
	2	1.000	0.007	0.081	0.080	0.948
Useless	0.25	0.000	0.001	0.034	0.034	0.948
	0.5	0.000	0.001	0.031	0.030	0.951
	1	0.000	0.001	0.023	0.023	0.949
	2	0.000	0.001	0.017	0.016	0.944
Partial	0.25	0.197	0.003	0.051	0.051	0.955
	0.5	0.229	0.001	0.047	0.047	0.953
	1	0.213	0.002	0.038	0.037	0.952
	2	0.167	0.003	0.030	0.030	0.957
c = 5, v = 1
Scenario	Marker time ${}^{\mathrm{a}}$	True value	Bias	Sampling SE	Mean of SE	Coverage
Perfect	0.25	0.743	0.002	0.043	0.044	0.951
	0.5	0.931	0.002	0.044	0.046	0.954
	1	0.995	0.002	0.047	0.048	0.949
	2	1.000	0.004	0.062	0.063	0.945
Useless	0.25	0.000	0.001	0.033	0.034	0.964
	0.5	0.000	0.000	0.030	0.030	0.958
	1	0.000	0.000	0.022	0.022	0.954
	2	0.000	0.001	0.015	0.016	0.954
Partial	0.25	0.204	0.002	0.051	0.051	0.951
	0.5	0.241	0.000	0.046	0.046	0.953
	1	0.228	0.000	0.036	0.036	0.960
	2	0.181	0.001	0.028	0.029	0.951
c = 5, v = 1.2
Scenario	Marker time ${}^{\mathrm{a}}$	True value	Bias	Sampling SE	Mean of SE	Coverage
Perfect	0.25	0.742	0.002	0.036	0.036	0.949
	0.5	0.930	0.002	0.036	0.037	0.956
	1	0.995	0.001	0.037	0.038	0.952
	2	1.000	0.003	0.049	0.048	0.940
Useless	0.25	0.000	0.001	0.037	0.037	0.953
	0.5	0.000	0.000	0.032	0.032	0.955
	1	0.000	0.000	0.023	0.023	0.943
	2	0.000	0.000	0.016	0.016	0.953
Partial	0.25	0.219	0.003	0.055	0.054	0.948
	0.5	0.262	0.000	0.049	0.049	0.956
	1	0.252	0.000	0.037	0.038	0.947
	2	0.203	0.001	0.030	0.030	0.952

^a The time point (year) measuring the marker.

summarizes the simulation results. We consider the scale parameter v to be 0.8, 1, and 1.2, representing when the hazard is decreasing, constant, and increasing with time, respectively. We are interested in the surrogacy level of the t-th year marker measurement for the treatment effect on the c-th year survival probability. For each setting of v, we choose $c=5$ (years) and $t=0.25,0.5,1,2$ (years). We show typical scenarios when a surrogate marker is perfect, useless, or partial. A perfect surrogate marker explains all the treatment effect on the clinical endpoint, i.e., $b_1=0;$ an useless maker is conditionally independent of the failure time given Z, i.e., $b_2=0$; a partial marker, the most common scenario in practice, is beyond the above extreme situations. Without loss of generality, we consider a treatment delaying the failure time by both directly affecting the clinical endpoint and suppressing a harmful marker. That is, $b_1\le 0$, $b_2\ge 0, \mathrm{and} {\mu }_0\le {\mu }_1.$ Specifically, here are the configurations for the three scenarios in Table 1: (1) A perfect marker: $\lambda =0.02,b_1=0,b_2=3,t_0=3\mathrm{months},t_1=30\mathrm{months};$ (2) a useless marker: $\lambda =0.3,b_1=-1,b_2=0,t_0=3\mathrm{months},t_1=30\mathrm{months};$ and (3) a partial marker: $\lambda =0.2,b_1=-0.5,b_2=0.5,t_0=3\mathrm{months},t_1=30\mathrm{months}.$ We replicate 1000 times with 20,000 subjects. Under a large sample size, the estimator is unbiased; its variance accurately reflects the sampling variation; the coverage of 95% Wald-type confidence intervals is close to the nominal probability. One limitation for the non-parametric estimator is its lack of efficiency, which is the price for avoiding model misspecification.

4. Data Analysis

We apply the proposed time-varying F-measure to the HIV Prevention Trial Network (HPTN) 052 study [15]. The study enrolled 1763 serodiscordant couples in which one participant was HIV-positive, and the other was HIV-negative. The HIV-positive patients were randomly assigned to receive either immediate or delayed ART in a 1:1 ratio. Patients on the delayed arm started ART when two consecutive CD4+ cell count measurements fell below 250 per cubic millimeter or an indicator of AIDS developed. The study monitored the earlier occurrence of severe clinical outcomes in HIV-positive patients or HIV transmission to HIV-negative partners as a key endpoint. It is believed that plasma viral load mediates the effect of ART on HIV-associated disease progression and transmission [16].

In this application, we consider the plasma viral load as a candidate marker and evaluate its surrogacy level on the composite monitoring endpoint in a 3-year follow-up. To explain the idea in a simple way, we dichotomize the viral load using a threshold of 1000 copies per cubic millimeter. More specifically, we set the marker value to be 1 for a viral load greater than 1000. We estimate the time-varying F-measure for the viral load measured at each of the 2nd to 7th quarter after randomization.

Table 2. Application to an HIV prevention trial HPTN 052. The proposed time-varying F-measure captures the proportion of treatment effect explained by the plasma HIV-1 viral load.

Marker Time ${}^{\mathbf{a}}$	Delayed ART Arm		Immediate ART Arm		F-Measure
	Prevalence of	Hazard Ratio ${}^{\mathbf{b}}$	Prevalence of	Hazard Ratio ${}^{\mathbf{b}}$	Estimator	95% CI
	Viral Load $\ge 1000$		Viral Load $\ge 1000$
2	0.88	1.39	0.08	2.20	0.18	−0.03, 0.39
3	0.88	0.94	0.08	3.21 *	0.41	0.13, 0.70
4	0.87	1.00	0.09	4.49 *	0.52	0.09, 0.95
5	0.85	1.59	0.08	5.59 *	0.72	0.10, 1.34
6	0.81	2.51 *	0.07	4.49 *	1.12	−0.42, 2.67
7	0.75	2.11	0.08	6.55 *	0.81	−0.92, 2.54

^a The time point (quarter) when plasma HIV-1 viral load was measured. ^b Hazard ratio between groups with a viral load higher and lower than 1000 copies per cubic millimeter. Significant results at the level of 0.05 are marked with *.

shows results of the application. Comparing the prevalence of a high viral load between the two arms reveals that ART was very effective in suppressing viral proliferation. In addition, a low viral load significantly decreases the hazard of the composite endpoint on the immediate arm before the treatment effect kicks in on the delayed arm. The time-varying F-measure gradually increases until reaching its maximum at the 6th quarter. This temporal pattern reflects the fact that the surrogacy level is a combination of the treatment effect on the marker and the marker effect on the clinical endpoint. On the one hand, it takes time to realize the effect of viral load suppression. On the other hand, as an increasing number of patients on the delayed arm began ART, the difference in marker distribution between two arms become smaller. The time-varying F-measure correctly reflects the temporal pattern and the biological mechanism of ART.

5. Discussion

In this paper, we consider a definition of time-varying F-measure based on three aspects. First, there is the question of whether there is a sound interpretation for comparisons. Second, do the typical marker types, such as perfect or useless markers, correspond to reasonable values. Third, is the defined F-measure model-free and equipped with a non-parametric estimation? Guided by the three questions, we define the time-varying F-measure in Section 2. In addition, we explore two alternative definitions. Both of them do not conduct an appropriate comparison. With $g_z\left(x\right)=P(T\ge c\mid X_t=x,Z=z),$ the F-measure can be defined as:

$$F(c,t)=\frac{P(T\ge c\mid Z=1)-{\sum }_{x}P(T\ge c\mid X_t=x,Z=1)P(X_t=x\mid Z=0)}{P(T\ge c\mid Z=1)-P(T\ge c\mid Z=0)}.$$

When the availability of an internal marker depends on the failure time (e.g., event is death-related), $X_t$ should include “not applicable” as a possible value for subjects with $T<t.$ In this case, $P(X_t=x\mid Z=1)-P(X_t=x\mid Z=0)$ is determined by both the treatment effect on the marker and that on the primary endpoint. Compared to $P(T\ge c\mid Z=1)$, the adjusted survival probability actually removes a portion of the direct treatment effect. This definition does not reflect the proportion of treatment effect explained by the surrogate marker in general. With $g_z\left(x\right)=h(c\mid X_t=x,Z=z),$ the F-measure can be defined as:

$$F(c,t)=\frac{h(c\mid Z=1)-{\sum }_{x}h(c\mid X_t=x,Z=1)P(X_t=x\mid T\ge c,Z=0)}{h(c\mid Z=1)-h(c\mid Z=0)}.$$

A closer look at the marker distribution $P(X_t=x\mid T\ge c,Z=z)$ reveals that:

$$P(X_t=x\mid T\ge c,Z=z)={\left(1+\sum _{y\ne x}\frac{P(T\ge c\mid X_t=y,T\ge t,Z=z)P(X_t=y\mid T\ge t,Z=z)}{P(T\ge c\mid X_t=x,T\ge t,Z=z)P(X_t=x\mid T\ge t,Z=z)}\right)}^{-1}.$$

If there is no interaction between the marker and intervention, the independence of $X_t$ and Z in the risk set at time point t could translate to the independence at time point $c.$ In other words, only if $P(T\ge c\mid X_t=y,T\ge t,Z=1)/P(T\ge c\mid X_t=y,T\ge t,Z=1)=P(T\ge c\mid X_t=y,T\ge t,Z=0)/P(T\ge c\mid X_t=y,T\ge t,Z=0),$ then $P(X_t=x\mid T\ge c,Z=1)=P(X_t=x\mid T\ge c,Z=0)$ is equivalent to $P(X_t=y\mid T\ge t,Z=z)/P(X_t=x\mid T\ge t,Z=z)$ and is constant with $z.$ Assumption of no interaction is, unfortunately, necessary for the appropriateness of the definition with hazard functions. As a contrast, the time-varying F definition introduced in Section 2 has a sound interpretation, reasonable ranges, and model-free definition and estimation. Moreover, numerical studies and practical data analysis verify the measure’s numerical behavior.

The time-varying F-measure is a generalization of the PTE [6] and F-measure [9]. All three measures are quantitative ones based on the qualitative Prentice Criterion [5]. While Prentice Criterion tests $P\left(T\right|S,Z)=P(T\left|S\right)$ and requires a surrogate marker to capture the treatment effect fully, the three quantitative measures compare the treatment effects unadjusted and adjusted by the marker distribution on the treatment arm. Beyond the similarities, PTE is defined for binary endpoints and relies on logistic regressions for definition and estimation; the F-measure is a model-free version of PTE, however it does not cover how to assess surrogate markers for time-to-event outcomes. The time-varying F-measure brings in the time dimension and extends the measure for time-to-event outcomes in survival settings.

6. Conclusions

This paper introduces a generalized proportion of treatment effect for survival settings, called the time-varying F-measure. Without relying on any model assumption, the measure reflects the proportion of the average treatment effect mediated by the surrogate marker. In addition, the paper introduces a non-parametric estimator to maximize the measure’s model-free characteristics. One limitation of the current estimation method is its lack of efficiency, which can be a future research direction. We applied the generalized F-measure to assess the viral load as a surrogate marker for HIV progression and transmission in the HPTN052 study. The time-varying F-measure increased from 0.18 in the 2nd quarter after randomization and reached 1.12 in the 6th quarter. It correctly captured the temporal pattern and biological mechanism of how ART regulates HIV progression and transmission by suppressing viral replication.