Adjusting the Cut-Off and Maximum Pool Size in RT-qPCR Pool Testing for SARS-CoV-2

Abstract

Reverse transcription quantitative real-time polymerase chain reaction (RT-qPCR) to detect SARS-CoV-2 RNA is an essential test to monitor the occurrence of COVID-19. A methodology is proposed for the determination of maximum pool size and adjustments of cut-off values of cycle threshold ($C_t$ in RT-qPCR pool testing, to compensate for the dilution caused by pooling. The trade-off between pool size and test sensitivity is stated explicitly. The procedure was designed to ensure that samples that would be detectable in individual testing remain detectable in pool testing. The proposed relaxation in cut-off is dependent on the pool size, allowing a relatively tight correction to avoid loss of detection of positive samples. The methodology was evaluated in a study of pool testing of adults attending a public emergency care unit, reference for COVID-19 in Belo Horizonte, Brazil, and presenting flu-like symptoms. Even samples on the edge of detectability in individual testing were detected correctly. The proposed procedure enhances the consistency of RT-qPCR pool testing by enforcing that the scales of detectability in pool processing and in individual sample processing are compatible. This may enhance the contribution of pool testing to large-scale testing for COVID-19.

1. Introduction

Most SARS-CoV-2-infected patients either present symptoms indistinguishable from other flu-like syndromes or remain asymptomatic. Therefore, the effective control of the epidemics requires testing large numbers of people regularly. The reverse transcription quantitative real-time polymerase chain reaction (RT-qPCR) is the most sensitive test for detecting SARS-CoV-2 shortly after the infection. However, its widespread application is limited by the cost of reagents, the need for specific laboratory equipment, sample transport logistics, and the long processing time [1,2,3].

Pool testing is a method of grouping samples to be tested together, to reduce costs and quicken the process. The number of samples on each pool should vary according to the infection’s prevalence [1,2,3]. If viral RNA is detected in a pool, each sample must be tested individually to identify the virus-positive ones. If the virus is not detected, all samples are considered non-detectable [1,2,3]. Pool testing allows substantial savings in test average cost and average time for delivering results. These characteristics are especially relevant to expand the testing coverage in scenarios with limited resources. As a shortcoming, if a sample with low viral load is included in a pool with negative samples, the reduced concentration of viral RNA in the pool can produce a false-negative result [4,5].

The usual criterion for discrimination between positive and negative results is based on the definition of a maximum value for the $C_t$ (cycle threshold) necessary to amplify the viral RNA in the sample up to a detectable level. Samples that become detectable with a $C_t$ greater than a pre-determined cut-off value ${\overline{C}}_t$ are interpreted as negative since the detected fluorescence is likely to represent noise. The authors of Reference [5] report that, in their experiments, the $C_t$ value in pooled samples is increased by a factor of 1.24 for each increase of dilution by a factor of 2. Other works also report experiments in which changes of the $C_t$ value in pooled samples are observed [6,7]. Although the authors of Reference [6] advise the “need for careful experimental design” due to that effect, no explicit guidelines for adapting the detection procedures for pool testing are provided in any of those works.

The goal of this study was to propose a methodology to determine the maximum pool size and to adjust cut-off values of cycle threshold ($C_t$) in RT-qPCR pool testing in order to compensate for the dilution caused by pooling. This article is organized as follows: A procedure is presented for the estimation of a lower bound $\overline{\gamma }$ for the amplification factor $\gamma$, which is characteristic of each specific laboratory setting. Then, a formula for the cut-off value of cycle threshold to be applied in pooled samples, denoted by ${\overline{C}}_t^{*}$, is developed as a function of the pool size $N$, of $\overline{\gamma }$, and of the cut-off value adopted in individual testing, ${\overline{C}}_t$. This formula gives the cut-off that would be necessary for detecting viral RNA, in a pool testing, in a sample situated in the edge of detectability of individual testing. As the value of ${\overline{C}}_t^{*}$ may become greater than the upper limit of the equipment detection range, $C_M$, as the pool size grows, then an upper bound $\overline{N}$ for the pool size $N$ is established, as a function of ${\overline{C}}_t$, $C_M$, and $\overline{\gamma }$, to guarantee that the sensibility of pool testing remains compatible with individual testing. The results of a calibration assay for the determination of $\overline{\gamma }$ in the specific laboratory setting used in this study are presented, and the respective specific formulae for ${\overline{C}}_t^{*}$ and $\overline{N}$ are derived. Finally, the application of the overall procedure in the processing of pooled samples collected from patients that attended a public emergency care unit are discussed.

2. Materials and Methods

2.1. Ethical Approval

This study was conducted within the preliminary phase of the research project Evaluation of the COVID-19 Diagnosis in Patients with Flu-Like Syndromes Attended at the Specialized COVID-19 Centers in Belo Horizonte, MG, Brazil. The research was approved by the UFMG Ethics Committee, CAAE-35074720.3.0000.5149, 23 June 2020.

2.2. Amplification Factor Estimation

In RT-qPCR, the concentration P of viral RNA in an individual sample after C cycles of replication is given by:

$$P=P_0{\gamma }^C$$

(1)

with P₀ representing the initial concentration before starting replication, and $\gamma$ representing the amplification factor per cycle. Let $C_t$ represent the number of cycles that amplifies P up to the detection threshold. If the positive sample is diluted in a pool with (N-1) negative samples, there will be a reduction of the initial viral concentration by a factor N. Therefore, the increase in the number of cycles for reaching detection in a pool is given by:

$$∆ = C_t^{*} - C_t=\frac{\ln N}{\ln \gamma }$$

(2)

with $C_t^{*}$ representing the number of cycles needed to reach the detection threshold in pool testing. For a given $\gamma$, different pool sizes lead to different Δ increments. For a specific set of reagents and PCR instrument, an explicit formula for the amplification factor $\gamma$ can be stated:

$$\gamma = N^{\frac{1}{∆}}$$

(3)

The apparent value of parameter $\gamma$ will vary between different runs, in the same equipment, due to random differences in relative volumes of reagent and sample material, and in relative volumes of different samples in a pool. The least reasonable value of $\gamma$ should be employed in Equation (2), such that the relaxation Δ becomes greater than the expected empirical values of $C_t^{*}$ found in practice.

2.3. Cut-Off Adaptation for Pooled Samples

Let ${\overline{C}}_t$ denote the cut-off value adopted in individual sample testing (any $C_t$ value above it is assumed to indicate non-detection). The cut-off value ${\overline{C}}_t^{*}$ for pooled samples that allows the detection of samples that are on the edge of detection in individual testing, with $C_t = {\overline{C}}_t$, should be:

$${\overline{C}}_t^{*} = {\overline{C}}_t+\frac{\ln \left(N\right)}{\ln \left(\overline{\gamma }\right)}$$

(4)

2.4. Pool Size Upper Bound

Each laboratory setting detection range which is bounded by a maximum detectable $C_t$ value, denoted as $C_M$, needs to be determined. Using cut-off values above this limit would lead to false-positive results. Therefore, if the testing is expected to detect viral RNA in samples with $C_t$ up to ${\overline{C}}_t$, the pool size N must be chosen such that ${\overline{C}}_t^{*} \le C_M$, which establishes an upper bound $\overline{N}$ for the pool size that can be used:

$$N \le \overline{N} = exp\left[\left(C_M - {\overline{C}}_t\right)\ln \left(\overline{\gamma }\right)\right]$$

(5)

Assuming that $C_M$ and $\overline{\gamma }$ are fixed values, intrinsic to the specific laboratory setting, Equation (5) quantifies the trade-off between the maximum pool size $\overline{N}$ and the target detection threshold ${\overline{C}}_t$. A stringent requirement on the detection of samples with high $C_t$ is translated in high ${\overline{C}}_t$ values, leading to small pools. As that requirement is relaxed to smaller values of ${\overline{C}}_t$, larger pool sizes become admissible, at the cost of possibly not detecting samples with $C_t$ greater than ${\overline{C}}_t$.

2.5. RT-qPCR Testing Framework

All testing experiments reported here were conducted in the CT-Vacinas laboratory according to the following guidelines. RT-qPCR tests for SARS-CoV-2 were performed with RNA purified from nasopharyngeal swab samples. RNA was extracted from 140 µL of samples using the QIAamp Viral RNA Mini Kit (Qiagen, Hilden, Germany,), according to protocols provided by the manufacturer. RT-qPCR was performed using primers and probes described in the Berlin (Charité/Berlin, Germany) protocol [8], targeting the gene E from SARS-CoV-2 and the human RNAse P mRNA, used as endogenous amplification reaction control. Reactions were carried out with the Promega GoTaq^® Probe 1-Step RT-qPCR Kit (Promega, Charbonnières-les-Bains, France) according to manufacturer’s recommendations, and the QuantStudio™ 5 Real-Time PCR System (Thermo Fisher, Waltham, MA, USA). To determine the lower limit of detection, the primer–probe sets were tested using purified SARS-CoV-2 RNA diluted in RNA elution buffer. The samples were assigned as undetectable when no SARS-CoV-2 gene E amplification occurs or a $C_t$ value above 37 in individual tests was obtained. Pool samples were prepared by mixing individual samples to a final volume of 140 μL before RNA extraction.

2.6. Calibration Experiments

The calibration phase of this study was performed using samples that were collected from patients with severe symptoms, which were processed individually. To determine a lower bound for $\gamma$ in the specific setting used in this study, 14 samples with known $C_t$ values were diluted in pools of sizes 16 and 32, and Formula (3) was employed in each case. The resulting values of γ were adjusted by a lognormal distribution. The value $\overline{\gamma }= 1.46$ corresponds to the 90% percentile of that distribution, meaning that $Pr\left(\gamma \ge \overline{\gamma }\right)=0.9$.

The variability of the apparent values between different executions of the testing procedure in the same experimental apparatus depends on several factors, such as the accuracy of the volume measurements made during the preparation of the pools. Although this study did not examine variations considering different experimental settings, it is reasonable to expect that the use of different reagents, for example, will also cause differences. For this reason, it is recommended that each laboratory that uses the procedures proposed in this article carry out a calibration test similar to the one described here. Notice that the calibration assay is quite simple, typically requiring only one batch run of the PCR system with 10 to 20 pooled volumes, which can be performed within a few hours.

2.7. Cut-Off Adjustment and Pool Size Choice

In the testing procedures adopted in this study, a range of different target cut-off values for individual samples was examined, with ${\overline{C}}_t$ varying from ${\overline{C}}_{tmin}=33$ to ${\overline{C}}_{tmax}=37$ respectively, leading to the following range of instances of Equation (4) for cut-off values to be adopted as functions of the pool size $N$:

$$\begin{array}{c}{\overline{C}}_{tmin}^{*}=33+2.643 \ln \left(N\right)\\ {\overline{C}}_{tmax}^{*}=37+2.643 \ln \left(N\right)\end{array}$$

(6)

Those different target values mean that ${\overline{C}}_{tmax}^{*}$ should be used whenever the detection of samples with individual $C_t$ up to 37 is considered relevant, while ${\overline{C}}_{tmin}^{*}$ may be used if not detecting samples with individual $C_t$ above 33 is assumed to be admissible.

In the case of the laboratory setting employed in this study, the upper limit of the detection range is $C_M$ = 41. The following upper bounds for the pool size, ${\overline{N}}_{33}$ and ${\overline{N}}_{37}$, respectively corresponding to ${\overline{C}}_{tmin}=33$ and ${\overline{C}}_{tmax}=37$, are calculated from Equation (5):

$$\begin{array}{c}{\overline{N}}_{33}=20.65\\ {\overline{N}}_{37}=4.54\end{array}$$

(7)

Equations (6) and (7) represent the guiding information for the adaptation of testing procedures for pool testing. After choosing the target cut-off value for individual testing within the range ${\overline{C}}_{tmin}=33$ to ${\overline{C}}_{tmax}=37$, Equation (6) shows the correction to be performed in the pool cut-off value ${\overline{C}}_t^{*}$ for discriminating between positive pools and negative ones. Equation (7) shows the respective upper bounds for the pool sizes that still allow the detection of pools containing one positive sample on the edge of detectability. Those relations are represented in Figure 1.

2.8. Pool Testing for Patients with Mild Symptoms

From 9 September to 10 October 2020, adults attending a public emergency care unit, reference for COVID-19 in the city of Belo Horizonte, Brazil, and presenting flu-like symptoms were invited to participate. Those who agreed to participate (n = 220) signed an informed consent form, answered a questionnaire reporting clinical and socioeconomic aspects, and had nasopharyngeal swab material collected. The collected material was sent to CT-Vacinas for the execution of testing procedures.

The 220 samples were grouped into 40 pools, 23 of them were positive and 11 pools presented a single positive sample, corresponding to 50 patients. Those patients’ age ranges from 20 to 88 years (40 ± 16 years) and most of them were female (59%). Their symptoms had started from 1 to 15 days before the test (6 ± 4 days) and half of them had no comorbidities. Individual $C_t$ values ranged from 12.2 to 37.1, and 50% of the patients showed $C_t$ values up to 20.0. The initial pool size of 12 was successively reduced to 8, 4, and 3 to adjust it to the disease prevalence measured during the initial phase of the study. It should be noticed that at the moment when this study started, there was no information about the prevalence of the disease among the population of Belo Horizonte. The initial pool size of 12 was chosen assuming a prevalence figure under 1%. However, the results soon revealed a prevalence between 15% and 25% of COVID-19 in the patients with flu-like symptoms that attended that care unit. This led to the resizing of pools to keep the optimum size from an economic point of view. It is important to stress that pooling was random, and positive samples were not purposely included in pools.

3. Results

Among 23 positive pools, 11 pools presented a single positive sample. The pools with more than one detectable sample were not included in this study, as they do not follow Equations (2) and (3), since those equations assume the worst case in which one positive sample is diluted such that the viral RNA concentration is decreased by a factor of 1/N.

Table 1. $C_t$ values in pools with a single positive sample and recommended increase in cut-off value.

Pool Size	${\mathit{C}}_{\mathit{t}}$ Value of Individual Sample 1	${\mathit{C}}_{\mathit{t}}$ Values Obtained with Each Pool	Empirical Increase of ${\mathit{C}}_{\mathit{t}}$	Recommended Increase in Cut-Off Value: $\left(\frac{\mathbf{ln}\left(\mathit{N}\right)}{\mathbf{ln}\left(\overline{\mathit{\gamma }}\right)}\right)$
N	$C_t$	$C_t^{*}$	${∆}_e$
12	16.9	20.7	3.8	6.6
8	18.1	23.1	5.0	5.5
8	21.0	24.3	3.3	5.5
8	35.4	40.5	5.1	5.5
8	26.4	28.1	1.7	5.5
3	34.8	37.5	2.7	2.9
3	27.5	28.9	1.4	2.9
3	16.6	18.2	1.6	2.9
3	30.8	33.5	2.7	2.9
3	22.6	25.0	2.4	2.9
3	22.5	24.9	2.4	2.9

Note: ¹ a single sample was positive in each analyzed pool.

shows $C_t$ values of those individual samples as well as $C_t$ values obtained with each pool and the recommended increase in cut-off value (see also Appendix A). A recommended increase in cut-off values relative to the RT-qPCR original cut-off is calculated in order to compensate dilution caused by pooling. This increase in $C_t$ ranges from 6.6 in polls of 12 samples containing one positive sample, to 2.9 in polls with 3 samples containing one positive sample.

4. Discussion

All empirical increases in $C_t$ values, presented in Table 1, were smaller than the recommended increase in the cut-off. This outcome was expected since the adapted cut-off value was calculated from a lower bound estimate of the amplification factor $\gamma$. Not-withstanding, the excess of the recommended increase in relation to the empirical increase was relatively small in most cases. It should be noticed that, although it is expected that different experimental settings will not lead to recommended increases in cut-off values that are too different from the ones presented in Table 1, a prudent approach would be to carry out calibration experiments in each laboratory that would run pooled testing programs, at least until more data is available on the variation of the apparent amplification factor γ in different experimental contexts.

The proposed relaxation in cut-off is dependent on the pool size, allowing a relatively tight correction to avoid loss of detection of positive samples such as the ones presented in the 4th and 6th lines of Table 1. If the usual fixed cut-off ${\overline{C}}_t$ = 37 was employed in the pool processing, the pools containing those samples would have been considered negative. The proposed method also avoids excessively large corrections that could result in several false-positive pools.

The choice of a target cut-off ${\overline{C}}_t$ should be done based on an assessment of the risk of not detecting infective individuals as smaller values are adopted. A review article [9] asserts that an increasing body of evidence suggests that $C_t$ values are useful proxies for infectivity and discusses some studies that attempted to cultivate the virus from samples with different $C_t$ values. Those studies have found that viral culture was largely unsuccessful when $C_t > 33$. Reference [10] reports an experiment with more than 3000 samples in which less than 3% of the ones with $C_t>35$ presented virus that could be cultivated. Higher target values are more conservative: the value ${\overline{C}}_t=37$ has been used by CT-Vacinas laboratory in most of the individual tests that have been performed in the last months because it represents a good compromise between the objectives of detecting all infected individuals and avoiding false-positive results. In the case of pool testing, the choice of the target cut-off ${\overline{C}}_t$ causes a major impact in the cost-effectiveness outcomes, since different choices in the range 33–37 lead to maximum pool sizes that vary from 4 to 20, as shown in Equation (7) and Figure 1. A specific ${\overline{C}}_t$ value should be chosen according to the purpose of the testing procedure, which will determine the acceptable risk level.

As the cut-off value ${\overline{C}}_t^{*}$ to be adopted in a pool testing is increased in relation to the individual testing cut-off ${\overline{C}}_t$, it should be expected that the rate of false-positive results in pool testing also increases. In fact, among the pools that were processed in this study, there was one pool of size $N = 3$ which presented a positive result with $C_t^{*}= 38.7$, although the individual testing of all samples resulted negative. This event illustrates an important property of the proposed methodology: there is no increment in false-positive results in relation to individual testing, due to the final phase of individual testing of all samples whenever a pool is found to be positive.

It is worthy to comment that the optimal pool sizes, under the viewpoint of cost optimization, will depend on the prevalence of the disease in the population under study. As shown in Reference [11], the optimal pool sizes may vary from 12, for prevalence under 1%, to 3, for prevalence between 13% and 30%. For prevalence of 1%, the expected savings are nearly 80%, while in the case of prevalence of 13%, the expected resource savings are nearly 33%. For a prevalence of 30%, the expected savings are only about 1%, which means that the pool testing technique no longer presents an advantage. In the case of the study reported here, the weekly prevalence varied between 15% and 25%, leading to overall cost savings of approximately 15%.

In summary, the following procedure for the adaptation of $C_t$ cut-off value for SARS-CoV-2 detection in pools and for choosing the maximum admissible pool size is proposed here:

• Find an estimate of lower bound for the amplification factor value corresponding to the laboratory setting to be used, by a calibration assay that compares $C_t$ of individual and pooled samples, using Equation (3) and a lognormal parametric model of probability distribution.
• For each pool size, calculate the corresponding correction of cut-off value, according to Equation (4).
• Considering the equipment detection range and the desired target cut-off for individual testing, employ Equation (5) for finding the maximal admissible pool size.
• If the optimal pool size under the economic viewpoint, as stated in Reference [11], is greater than that maximal admissible pool size, adopt that maximal value; otherwise, employ the optimal size.

The proposed procedure enhances the consistency of RT-qPCR pool testing by enforcing that the scales of detectability in pool processing and in individual sample processing are compatible. This procedure may contribute to reduce false-negative results in RT-qPCR pool testing, enhancing its contribution to large-scale testing for COVID-19.