1. Introduction
In the USA, as of August 2023, the SARS-CoV-19 virus has infected more than 103 M people while causing more than 1.1 M deaths [
1]. Despite vaccination, these numbers continue to mount, particularly in areas outside of Western Europe and North America without access to vaccines and in parts of the USA with a higher percentage of vaccine-reluctant residents. Waning immunity and the emergence of novel variants also play a role in persistent infections. As the pandemic has persisted for 2+ years now, there is increasing demand to “reopen” cities, workspaces, and schools and university campuses in a bid to return to normal, despite the risks involved. Pandemic fatigue has become commonplace. To achieve a return to normalcy, it is necessary to be able to rapidly identify potentially large numbers of virus-infected individuals so that disease transmission can be stopped. Universities and academic hospitals can play a critical role in this process due to their experience in working with biohazardous specimens, familiarity with large-scale patient recruitment methods, and ability to deploy large-scale testing procedures.
Tracing efforts required to establish the identity of infected persons depend on the rapid identification of the primary infected person. The source of infection must be isolated and quarantined for observation. Testing strategies must return results to those responsible in as short a time as possible in order to limit viral spread [
2]. Typically, polymerase chain reaction (PCR) assays have been the gold standard for viral diagnostic testing [
3,
4], but other testing options are available. Unfortunately, the PCR test is not generally a point-of-care test and often takes 24–72 h to return results, is relatively expensive at scale, has been plagued by reagent scarcity, and may even be too sensitive at times [
2,
3,
4].
Two rapid viral antigen tests have recently become available in response to the above needs. A manual testing strategy is commercially available from Quidel, Inc. (San Diego, CA, USA) to analyze subjects for the presence of the SARS-CoV-2 viral antigen Ag [
5] using a fluorescence assay. The assay (Sofia SARS Antigen FIA) detects the presence of antigens produced by a viral infection rather than the presence of the viral genome, which may not be indicative of an active infection. The test reports the result within 15–30 min, can be scaled to perform thousands of tests per day, has US Food and Drug Administration (FDA) Emergency Use Authorization (EUA) for symptomatic subjects, is relatively inexpensive at USD 23/test, and can be performed using self-administered (anterior) nasal swabs. Beckman-Coulter has developed an automated high-throughput viral rapid antigen test using a luminescence assay to detect the presence of the SARS-CoV-2 viral antigen [
6]. The Beckman-Coulter test can also be read in 10–30 min after virus extraction, is capable of testing 200 or more samples per hour in an automated fashion, is extremely inexpensive at USD 10/sample, has applied for FDA EUA, and is amenable to the use of both NP (nasopharyngeal) and nasal swabs as biospecimen sources.
Both Quidel and Beckman-Coulter are companies with a history of producing FDA-approved clinical diagnostic tests, lending confidence to purported reliability claims. Both testing devices, the Sofia 2 and the Beckman-Coulter DxI, have the capability to both display results locally to the operator as well as report into cloud applications or include them directly in subject medical records. The sensitivity of both SARS-CoV-2 rapid Ag tests is reported to be 90%+ with near-100% specificity [
5] (FDA EUA Application and 5–6), but these results were not based on testing large numbers of “real-world” samples in a large asymptomatic population.
In the present study, we examined the sensitivity of the manual Quidel and the automated Beckman-Coulter rapid Ag tests in side-by-side use in screening asymptomatic individuals in the student and staff community at the University of Arizona from March to May 2021 (to coincide with Spring Break and semester graduation). After test validation [
7], a total of 176 validation sets and 378 asymptomatic subjects in 22 independent assessments were performed using both antigen tests side-by-side at the same time with specimens obtained from the same subject. All recruitment and testing procedures were performed by the University of Arizona Health Sciences (UAHS) Biorepository staff at the University of Arizona. In addition to the sensitivity of detection, we also compared the manual versus automated SARS-CoV-2 testing methods for sample throughput, time required for testing, and cost effectiveness (particularly personnel costs).
3. Results
The utility of the individual SARS-CoV-2 rapid viral antigen tests was assessed in two ways: using validation sets of known positive samples and using nasal swabs obtained from subjects that were probably exposed but asymptomatic. The results of testing using viral antigen “spiked” validation samples to assess tests’ sensitivity are shown in
Table 1. A total of 17 experiments were performed using the manual Quidel assay with 136 independent samples, while a total of 22 experiments were conducted using the automated Beckman-Coulter assay with 176 independent samples. In these experiments, a limiting dilution was performed using undiluted samples as well as twofold dilutions of the original sample up to a 1:128 dilution to determine the sensitivity to antigen concentration using the different tests. The results between the two testing methods were concordant and in positive agreement. The two tests were observed to be statistically identical up through a 1:64 dilution.
The potential for false positive results (common with rapid antigen tests; [
7]) was next examined. It was observed that, based on the nature of the automated Beckman-Coulter assay (a luminescence reaction in solution), false positive results were not observed, in contrast with the occasional false positive result with the manual Quidel fluorescence LFT assay. A false positive result was inferred when one testing method continued to produce a positive result at very high dilutions when the other test did not. It was observed that the Quidel assay produced positive results when UTM versus VTM [
9,
10] was used as a transport medium, which in fact were false positive results. These results were confirmed by the observation of “positive” results even when the transport medium without virus was tested (see
Table 2, Negative Controls). It should be noted that the Quidel Sofia protocol includes its own rehydration solution and that this solution was replaced with UTM media for these negative controls. No false positives were observed with the Quidel tests when the Quidel supplied rehydration solution was used.
We next examined the influence that the sample transportation buffer might have on the testing results in greater detail, as shown in
Table 2. Beckman-Coulter recommends the use of UTM for sample transport, while many facilities commonly use VTM for such activities, including sample collection for PCR assays. As shown in
Table 2, when using UTM, the Quidel assay produced positive results even at dilutions as high as 1:128 and even when virus was not present in the assay. The Beckman-Coulter test, however, produced comparable results using either UTM or VTM as the transport buffer, although the VTM buffer seemed to result in slightly higher S/CO values.
Both tests were then assessed in 378 asymptomatic subjects, as shown in
Table 3, where most, if not all results were expected to be negative in such an asymptomatic population, particularly as extensive vaccination efforts had been underway since January 2021. As shown in
Table 3, both tests were comparable in terms of detecting the low number of positive subjects in the asymptomatic population. A concordance of 100% was observed between the two tests in
Table 3.
Finally, the experiments shown in
Table 4 analyzed whether saliva specimens (obtained using the saline-gargle approach described in Methods) could be used as an alternate self-administered biospecimen source [
8] of nasal swabs. The results showed that both assays produced comparable results using their respective testing methods. A concordance rate of 96% was observed.
4. Discussion
Mitigation of a pandemic such as the SARS-CoV-2 pandemic depends on the rapid and reliable testing of infected (symptomatic) and exposed (asymptomatic) individuals. Although PCR testing has been utilized historically in similar circumstances, it does not lend itself to a rapid turnaround of results, hindering the real-time tracking and tracing efforts that are crucial to slowing the spread of a virus. Rapid antigen tests that identify contagious individuals have been developed that are amenable to a mitigation effort. Testing approaches developed by Quidel and Beckman-Coulter are rapid (30–60 min), can be self-collected using anterior nasal swabs, are relatively inexpensive ($10–23/test), and are capable of high-throughput testing (2000 tests or more per day). The Quidel manual test makes use of a fluorescence LFT approach while the automated Beckman-Coulter test uses a luminescence assay in solution. The Quidel approach uses a small, table-top reader and test cartridges into which the biosample is placed manually. This system was originally designed for doctors’ offices and clinics performing single tests at a time. Due to the volume of testing required, we were able to scale up the manual system to perform thousands of tests per day using multiple machines. The Beckman-Coulter approach uses a large free-standing machine that is automated to read sample large batches, also at the level of thousands of tests per day. Automated reporting is supported by both the Quidel and Beckman-Coulter systems. Both manufacturers have applied for FDA approval of their tests, and the reader devices for both companies can perform other types of clinical assessments (e.g., influenza). The Quidel manual approach to large-scale testing requires the utilization of 6–10 machines (at a cost of $1200 per machine) to achieve a daily testing rate of 2000+ tests per day, while the automated Beckman-Coulter approach requires the use of a single machine (at a cost of $125,000). The total machine costs would be $12,500 for the Quidel approach using 10 machines versus $125,000 for the Beckman-Coulter approach using a single machine (which does have a significantly larger physical footprint). The Beckman-Coulter approach requires a smaller technical staff to reach the same processing goals when considering only the operation of the machine. However, for both systems, a very large fraction of the technicians’ time is spent organizing and labeling samples, performing manual sample extractions from the sample collection swabs, then organizing and racking tubes for assay. At this point, whether the samples were read manually with Quidel or automated with Beckman-Coulter, the total technician time was comparable.
The Biorepository at the University of Arizona Health Sciences deployed a wide-scale biospecimen collection and testing effort during the pandemic in 2020–2021 to assess community infection rates and to implement mitigation efforts. Prior experience working with biohazardous specimens was instrumental in preventing viral infections in the workplace. In addition, familiarity with large-scale patient recruitment methods and attention to detail needed in the tracking of biospecimens and their donors was crucial to organizing mitigation efforts. Finally, the ability to deploy large-scale virus testing procedures needed to confirm the infectious disease status of the biospecimen donors was required to isolate and quarantine those shedding virus and inhibit further viral spread.
In the current study, we have validated and evaluated two methodologies (manual and automated) side-by-side using nasal samples collected from the same subjects at the same time. We validated the two methods using virus-antigen-positive nasal swabs and evaluated the sensitivity using nasal swabs collected from 378 asymptomatic individuals, a total of 22 independent experiments collected over a course of 4 months. This period of time was chosen as it spanned from Spring Break at the university to the end of the Spring semester, when infections were likely to be more prevalent, albeit undoubtedly influenced by the vaccination efforts underway since January 2021. Significantly, the two methodologies were greater than 98% concordant over the course of this evaluation. If one only examines the agreement of the two tests for neat, undiluted samples (i.e., disregards the limiting dilution samples shown), then the concordance rate was above 99%. The Beckman-Coulter automated test also did not seem to be prone to false positive results (probably due to the nature of the luminescence assay in solution versus a fluorescence methodology in a solid LFT assay) and was amenable to test samples collected in either VTM or UTM, while UTM media produced a high level of false positive results in the Quidel assay (as demonstrated when UTM was used alone for testing without added virus). Thus, testing of asymptomatic individuals should be performed with samples collected in VTM if a transport buffer is required and the exact nature of the downstream assay is uncertain. Significantly, the Beckman-Coulter test seemed to be applicable not only to anterior nasal samples but also to saliva (saline gargle) specimens, as was the Quidel test, which increases the applicability and utility of the two tests and may encourage more individuals to participate in the mitigation efforts. It should be noted that PCR testing was not performed on the samples used in the rapid antigen testing, and thus false positive and false negative results cannot be excluded, but test concordance between the two approaches makes it unlikely.
Overall, sample testing throughout the experiments was comparable between the manual and automated methods. Testing costs were lower for the manual method when tests and machines are considered. However, the automated method should require fewer laboratory personnel, which is a significant cost savings. When all factors are considered, the overall costs are fairly comparable and both methods are capable of performing tasks as needed.