1. Introduction
The outbreak of the new coronavirus (SARS-CoV-2) infection has become a global public health crisis [
1]. Research results show that the new coronavirus has a homology of 88 percent with the bat severe acute respiratory syndrome-like coronavirus (bat-SL-CoVZC45) [
2,
3]. Therefore, the Coronavirus Research Group of the International Commission on Classification of Viruses named the new pathogen “severe acute respiratory syndrome coronavirus 2” (SARS-CoV-2), and the disease was officially named as the corona virus disease 2019 (COVID-19) by the World Health Organization (WHO) [
4].
SARS-CoV-2 is a single-stranded RNA virus that can primarily cause upper and lower respiratory tract infections after invading the human body. The main symptoms are fever, fatigue, and dry cough. In severe cases, breathing difficulties may gradually occur. Some patients have mild or even no symptoms, whereas others may present with a constellation of symptoms originating from several organs in the body [
5]. Asymptomatic individuals do not present with any COVID-19-related symptoms, but as carriers of SARS-CoV-2, they may transmit the virus to other susceptible individuals. The condition of some critically ill patients may progress rapidly, and may lead to acute respiratory distress syndrome, septic shock, metabolic disturbances, hypercoagulation and multiorgan failure [
5,
6,
7]. According to current epidemiological studies, the incubation period of SARS-CoV-2 is 1 to 14 days, usually 3 to 7 days. COVID-19 may be asymptomatic; however, asymptomatic carriers of SARS-CoV-2 may transmit the virus to vulnerable individuals [
4].
Although the therapeutic approach for patients with COVID-19 has been significantly improved since the onset of the pandemic, prompt diagnosis remains the cornerstone of optimal therapeutic intervention and prevention of SARS-CoV-2 transmission [
8]. The diagnosis of SARS-CoV-2 infection may be challenging depending on the clinical setting. The gold standard is the reverse-transcriptase polymerase chain reaction (RT-PCR); however, it may take a relatively long time (up to 24 h) for the result depending on the setting; it requires specialized personnel, and must be performed in laboratories of at least a P2 level, whereas its specificity is not 100% [
9,
10,
11]. MALDI-TOF mass spectrometry may be superior to RT-PCR for SARS-CoV-2 detection, but it is not widely available [
12]. Although most general hospitals have been now well equipped for performing PCR with the appropriate microbiological precautions, hospitals in rural areas or in low-resource settings and several clinics may lack the equipment and the personnel for safe and rapid PCR testing. Furthermore, the increased need of mass testing may cause important delays [
11].
As an auxiliary diagnostic method, the antibody detection kit can simultaneously detect IgM and IgG antibodies [
13]. The procedure is simple and convenient. It provides indirect evidence of the existence of the virus. It is mainly used for supplementary testing or joint testing of suspected patients with negative nucleic acid test results [
14]. However, the production of antibodies requires a certain incubation period, which cannot be detected in patients in the window period. It is more suitable for patients two weeks or more after the onset of symptoms [
15]. Those infected during the incubation period cannot be detected promptly, and may be able to transmit the virus. In addition to the above, the antibodies will last in the human body for a long time even after the patient is cured, antibodies may become evident following previous vaccination with anti-SARS-CoV-2 vaccines, and antibodies may show variations among individuals depending on the disease severity, age, sex and underlying medical conditions [
16,
17,
18,
19,
20,
21,
22]. All these make difficult discrimination among current infection, past infection and prior vaccination, especially in cases of asymptomatic COVID-19 [
23].
Furthermore, antigen detection kits aim to detect the pathogen itself, and, therefore, they provide direct evidence of viral infection even at the early stages of the disease [
11,
24]. The specimens of the antigen detection kit could be nasopharyngeal swabs, nasal swabs, oropharyngeal swabs and saliva samples. Nasal swabs, oropharyngeal swabs and saliva specimen can be relatively widely used in the clinical screening, and can be extended even for home self-testing.
In this context, herein we evaluated the COVID-19 Antigen Detection Kit (Colloidal Gold) produced by Zhuhai Lituo Biotechnology Co., Ltd., which aims to detect the N protein of the SARS-CoV-2.
2. Materials and Methods
2.1. Patient Selection
Two retrospective studies were designed to assess the COVID-19 antigen detection kit (colloidal gold) manufactured by Zhuhai Lituo Biotechnology Co., Ltd., Changsha, Hunan, China. The institutional review board of the General Hospital Jesenice approved the study (Number 2020/02). Participants provided oral informed consent for their samples to be evaluated for SARS-CoV-2 testing. We compared the results from the COVID-19 antigen detection kit (colloidal gold) with the RT-PCR results of samples from clinical cases with suspected COVID-19 in order to determine the sensitivity, specificity, and accuracy of the new method. As we aimed to assess the role of the COVID-19 antigen detection kit (colloidal gold) in the early detection of COVID-19, only patients reporting symptoms for a maximum of 7 days were included. Two kinds of specimens were used for evaluation: nasal swabs and nasopharyngeal swabs. Nasopharyngeal swabs are the most widely used clinical diagnostic specimens for the antigen test due to their high sensitivity and specificity [
25]. However, nasopharyngeal swabs require an invasive procedure for collection performed, ideally, by a healthcare professional. In this context, nasal swabs are also accepted, especially for population screening, traveling controls and self-testing.
2.2. Sample Collection
All samples were collected from patients suspected of having COVID-19 at a single center in Slovenia. Nasal and nasopharyngeal swabs were simultaneously collected from the same patient. The swab specimens were collected as follows: The sampling swab was inserted into the nasopharyngeal/nasal cavity with the most secretions. The swabs were rotated gently, pressed on the nasopharyngeal/nasal wall three times, then inserted into the other nostril and the above steps were repeated. The swab head was then removed from the nasopharyngeal/nasal cavity quickly and was immersed in the sample treatment solution. After collection, the specimens were processed with the extraction buffer provided by the COVID-19 antigen detection kit (colloidal gold). The test was completed within 10 min. For negative specimens, the reagents actually can give out results in around 3 min. However, we still recommend to read the results between 10 to 15 min. Results after 15 min should not be considered valid. Left over fluids with appropriate swabs were transferred in two UTM mini transport media (UTM mini, Copan, Italy). Two specimens were used for extraction of RNA. Two specimens of each patient were collected simultaneously; one for the experiment kit and one for RT-PCR reagent detection. The CE marked two gene reverse transcriptase polymerase chain reaction (RT-PCR) (Seegene, South Korea) was used as comparator throughout the clinical evaluation. The handling of all samples was performed in accordance with the institutional rules of microbiologic safety.
2.3. Principles of Antigen Detection with the COVID-19 Antigen Detection Kit (Colloidal Gold)
The new coronavirus particles are wrapped with a lipid bilayer membrane. There are three glycoproteins on the membrane surface: Spike Protein (S, Spike Protein, which is the cell receptor binding site, the fusion site with the cell membrane); Membrane glycoprotein (E, Envelope Protein, a small protein that binds to the envelope); Membrane Protein (M, Membrane Protein, responsible for the transportation of nutrients across the membrane, the budding release of new viruses, and the formation of the virus envelope) [
26]. One of the most important proteins in the viral nucleocapsid is the nucleocapsid protein (N, Nucleocapsid Protein), which is mainly responsible for the replication function of RNA. N protein is abundant in coronaviruses and is a highly immunogenic protein that participates in genome replication and cell signaling pathway regulation [
26]. Therefore, Nucleocapsid protein (N protein) is often used as a diagnostic test tool for coronavirus, and is the core material of rapid immunological diagnostic reagents.
This kit adopts the double antibody sandwich method to qualitatively detect novel coronavirus antigens in human nasopharyngeal, oropharyngeal, and nasal swab specimens, and saliva specimens. The kit uses colloidal gold to label anti-human coronavirus monoclonal antibody 1. Then, coronavirus monoclonal antibody 2 and polyclonal antibody goat anti-mouse IgG is coated in nitrocellulose membrane.
When the tested sample is positive, the antigen in the specimen binds to the antibody 1 labeled with colloidal gold. Then, the antigen is chromatographed to the detection area and binds to the pre-coated anti-human coronavirus monoclonal antibody 2 to form a double antibody sandwich complex, which generates red color. Both antibody 1 and antibody 2 form specific binding with the N-protein on the SARS-CoV-2. The remaining colloidal gold-labeled antibody is combined with the polyclonal antibody at the quality control line to generate red color. Negative samples generate red color only at the quality control line.
The Cycle Threshold (Ct) value cut-offs of 25 and 33 are based on the normal requirements in most of the tenders in the EU. The Ct value of 40 can distinguish between the presence or absence of coronavirus infections in RT-PCR reagents; therefore, we set this cut-off as the highest value.
2.4. Statistical Analysis
The evaluation indicators for this clinical evaluation include sensitivity, specificity, and accuracy. The sensitivity of a test is its ability to determine the patient cases correctly. To estimate the sensitivity of the COVID-19 antigen detection kit (colloidal gold), we calculated the proportion of true positives in patient cases (PCR positive). The specificity of a test is its ability to determine the healthy cases correctly (PCR negative and no symptoms of COVID-19). To estimate the specificity of the COVID-19 antigen detection kit (colloidal gold), we calculated the proportion of true negative in healthy cases. The accuracy of a test is its ability to differentiate the patient and healthy cases correctly. To estimate the accuracy of the COVID-19 antigen detection kit (colloidal gold), we calculated the proportion of true positive and true negative in all evaluated cases. We used Cohen’s Kappa formula to analyze the raw data.
4. Discussion
In this study we present an independent clinical evaluation of the immunochromatographic COVID-19 antigen detection kit (colloidal gold) in nasopharyngeal and nasal swabs compared with RT-PCR, among patients suspected of having COVID-19. The new antigen test had excellent diagnostic accuracy (above 95%), regardless the type of specimen and the Ct value of PCR positivity. The specificity of the antigen test was also above 99% in all settings. The sensitivity of the antigen test was really encouraging and ranged between 89% and 100%. When the Ct value of PCR positivity increased to 33 and above, a drop in sensitivity became evident in both the nasopharyngeal and nasal specimens. Higher Ct values of PCR positivity are associated with a decreased concentration of SARS-CoV-2 viral particles in the solution and, therefore, the detection rate of positive samples with the antigen test decreases. For this reason, we included only patients suspected of having COVID-19 and with suspected COVID-19-related symptoms with a duration of no more than 7 days, in order to assure an adequate concentration of the virus in the specimens.
Furthermore, the absolute sensitivity value was lower for the nasal specimens in Ct values of PCR positivity less than 33 but higher for the nasal specimens in Ct values of PCR positivity less than 40, as compared with the corresponding values for the nasopharyngeal specimens. These slight differences may be attributed to the number of patients with high PCR Ct values in each sub study; more specifically, 8 patients had PCR Ct values between 35 and 38 in the nasopharyngeal sub study, whereas in the nasal sub study only 2 patients had PCR Ct values in that range.
Our results are comparable to other studies. One study compared two analyzer-based tests to single gene SARS-CoV-2 RT-PCR. The chromatographic immunoassay BD Veritor System for Rapid Detection of SARS-CoV-2 nucleocapsid antigen (Veritor) showed a high positive percent agreement with the PCR results ranging from 81.8% to 87.5%, which is compatible to the high accuracy rates found in our study [
27]. Based on these encouraging results, the FDA provided an emergency use authorization for Veritor as a point-of-care test for patients suspected of having COVID-19 presenting in less than 7 days from symptom onset.
Another study compared several immunochromatographic antigen tests to RT-PCR. Sensitivity was between 16% to 85%. The wide sensitivity range was attributed to the use of non-validated samples [
28]. In one study the antigen test identified 70.6% of RT-PCR positive samples. A major limitation of this study was that the authors diluted the sample in transport media and performed the analysis off-site [
29]. In comparison, our study was performed on fresh samples as a point-of-care test. This approach may explain our superior results compared to other studies using stored frozen samples to validate their antigen test in an offsite setting.
Another study evaluated the Standard Q COVID-19 Ag test (SD Biosensor, Chuncheongbuk-do, Republic of Korea), which is a rapid chromatographic immunoassay, for the detection of SARS-CoV-2 nucleocapsid antigen in 454 respiratory specimens. Similar to our results, the sensitivity and specificity were 98.33% and 98.73%, respectively [
30].
FindDx platform has published several validations of immunochromatographic antigen SARS-CoV-2 assays. Our approach showed similar performance to the best SARS-CoV-2 antigen immunochromatographic tests in other comparisons (sensitivity between 76.6% and 90.8%, specificity between 99.3% to 100%) [
31,
32]. A study that used an analyzer to screen a student population resulted in inferior sensitivity compared to our study [
31]. Importantly, our results with the colloidal gold assay are in accordance with the manufacturer performance data, which demonstrated 96.9% sensitivity and 99.5% specificity [
31]. The drop in sensitivity of our evaluation can be explained by the low viral load in patients that did not present early during the course of COVID-19.
In a Cochrane meta-analysis, the sensitivity of antigen tests varied considerably from 0% to 94% across five studies on 943 samples [
33]. In an updated version of the meta-analysis, 48 studies encompassing 58 evaluations of antigen tests were included. Interestingly, the average sensitivity was higher in the first week after symptoms onset (78.3%, 95% CI: 71.1–84.1%) compared with the second week of symptoms (51.0%, 95% CI: 40.8–61.0%). Sensitivity was high among those with PCR Ct values less than 25 (94.5%, 95% CI: 91.0–96.7%) compared with those with Ct values more than 25 (40.7%, 95% CI: 31.8–50.3%) [
24]. These results are in accordance with our study, which shows superior sensitivity of the new antigen test in the early period from the onset of COVID-19 related symptoms, and among positive samples with a lower threshold of PCR positivity.
Our study has some limitations. It was performed in a single centre and, thus, the level of expertise of the personnel performing the sampling may be different from other hospital settings. Therefore, the education of all health care workers is very important in order to safely and accurately perform this procedure. Furthermore, we included only symptomatic patients presenting with symptoms with a duration of less than 7 days; thus, generalization of our results for asymptomatic carriers and patients with long lasting symptoms should be made with caution. Another potential limitation lies in the fact that our study focuses on RT-PCR for the diagnosis of COVID-19. Although the approved RT-PCR detection kits have high sensitivity and specificity rates, the results should be interpreted in association with clinical and radiological findings in a tailored manner.
In conclusion, our results show that the COVID-19 Antigen Detection Kit (Colloidal Gold) fulfils the WHO criteria for point-of-care COVID-19 tests of 80% sensitivity and 97% specificity in health care settings. Nasopharyngeal and nasal swabs have comparable high sensitivity, specificity, and accuracy in symptomatic patients with COVID-19 symptoms experienced for less than 7 days. Nasal swabs may be more convenient to collect in the clinical practice, especially in high patient volume settings. In the case of a negative antigen test despite high COVID-19 suspicion, a confirmatory PCR is necessary as the gold standard method.