Introduction
Laboratory diagnosis is important in promptly initiating appropriate management for individuals with coronavirus disease 2019 (COVID-19). Test methods for diagnosing COVID-19 should be reliable, affordable, accessible, and provide rapid results [1,2]. The standard reference method for laboratory diagnosis of COVID-19 is molecular testing, which is used to detect a specific gene of the causative pathogen—severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2)—for which real-time RT-PCR is most commonly used. Although this method yields the highest sensitivity and specificity among laboratory diagnostic tests for COVID-19, it has the disadvantages of requiring dedicated equipment, reagents, and skilled professionals. Moreover, it takes several hours to obtain results, making it difficult to initiate management promptly [1,3–5].
Antigen testing is a method that detects antigens composed of viral components, such as proteins. Antigen tests have been developed for the detection of SARS-CoV-2 and corresponding COVID-19 management. In particular, the rapid antigen detection test (RADT) can confirm results within 15–30 minutes in most cases, and the test method is simpler and easier than molecular tests; therefore, RADT can compensate for some obstacles encountered with molecular testing. The cost is also lower than that of molecular testing [1,4–6]. However, unlike genetic components, antigen components cannot be amplified; therefore, COVID-19 antigen testing requires at least 1,000 times more virus in a specimen than genetic testing; as such, sensitivity is low [1]. According to previous reports, most RADTs have reported a sensitivity of ≥80%–90% when the cutoff cycle threshold (Ct) was set as <25 and demonstrated low sensitivity for the subjects with the Ct was ≥25. In particular, it has been reported that a sensitivity <10% is observed when the subjects show Ct >30 [5–8].
The Panbio COVID-19 Ag Rapid Test Device (Panbio COVID-19 Ag, Abbott Rapid Diagnostics, Jena, Germany) is a diagnostic kit approved for use as an RADT for COVID-19. Panbio COVID-19 Ag is a lateral flow immunochromatographic assay targeting the SARS-CoV-2 nucleoprotein in nasopharyngeal specimens for the diagnosis of COVID-19. It reduces the time to read completion to 15 minutes compared to 30 minutes for other RADT previously used in our clinical laboratory, thus enabling faster reporting [9].
The present study aimed to verify the performance of the Panbio COVID-19 Ag for implementation in a clinical laboratory.
Methods
Among the remnant nasopharyngeal swab specimens dipped in viral transport medium, 30 specimens each were confirmed positive and negative according to real-time RT-PCR testing. For positive cases, specimens collected within 1 month of evaluation and stored frozen (−70°C) were thawed immediately before testing; for negative cases, specimens collected and refrigerated the day before evaluation were used. Real-time RT-PCR testing was performed using the Allplex SARS-CoV-2 assay (Seegene, Seoul, Korea), and by applying the Ct of the RdRp/S gene as a standard, a similar number of specimens were selected for each Ct range. When collecting positive specimens, more than 30% of the positive specimens had a Ct value of 30 or more. This study was approved by the Institutional Review Board of Ewha Womans University Seoul Hospital.
For the Panbio COVID-19 Ag test, 300 μL of specimen in transport medium was mixed with 300 μL of buffer, and five drops of the mixed solution was applied to the specimen well and reacted for 15 minutes. Among the samples in which the control line was positive, if both operators interpreted the result as positive, it was reported as positive and, if both operators interpreted the results as negative, it was reconfirmed 5 minutes later. If the operators’ opinions did not agree, the Panbio COVID-19 Ag test was retested with 300 μL of specimen without buffer and reported as the final result.
Results
Each quality control swab was tested 10 times, and the positive and negative controls were positive and negative, respectively, and all results were consistent.
The Panbio COVID-19 Ag results revealed that 15 of the 30 positive specimens were positive, and 15 were negative (Table 1). The sensitivity was 88.2% (95% CI 63.5%−98.2%), based on the detection limit Ct of 26.67 provided by the manufacturer (Table 2). Two specimens with a Ct ≤26.67, but negative for Panbio COVID-19 Ag, had Ct values of 25.89 and 26.62, respectively. The sensitivity was 100.0% (95% CI 78.0%−100.0%). for the positive criterion of Ct values <25. Specificity was 100%. Seven specimens were retested due to disagreement between the operators (Table 3).
| Testing method | Real-time RT-PCR | Sensitivity (95% CI) | Specificity (95% CI) | ||
|---|---|---|---|---|---|
| P | N | ||||
| Panbio COVID-19 Ag | P | 15 | 0 | 50.0% (31.3−68.7) | 100.0% (88.3−100.0) |
| N | 15 | 30 | |||
| Ct | Positive results (n) | Sensitivity (95% CI) | |
|---|---|---|---|
| Real-time RT-PCR | Panbio COVID-19 | ||
| ≤25 | 15 | 15 | 100.0% (78.0−100.0) |
| >25 | 15 | 0 | 0.0% (0.0−22.0) |
| ≤26.67* | 17 | 15 | 88.2% (63.5−98.2) |
| >26.67 | 13 | 0 | 0.0% (0.0−24.9) |
Discussion
In verifying the performance of the Panbio COVID-19 Ag, the sensitivity was low (50.0%) for all evaluated specimens compared with real-time RT-PCR. However, based on the detection limit Ct provided by the manufacturer, the sensitivity was high (88.2%) for positive specimens with a Ct value lower than the detection limit Ct. In particular, all specimens with a Ct <25 were considered to be positive (sensitivity, 100.0%).
Previous studies that most COVID-19 RADTs show a sensitivity of ≥80%−90% with cutoff Ct values <25, and a low sensitivity for Ct values ≥25 [5–8], which is consistent with the findings of the present study. In a meta-analysis of published studies, the pooled sensitivity of Panbio COVID-19 Ag was 71.8% (95% CI 65.4%−77.5%). According to the Ct value, the pooled sensitivity was 95.8% (95% CI 92.3%−97.8%) for Ct values <25 and 61.2% (95% CI 38.8%−79.7%) for Ct values >25 [7].
The sensitivity of the Panbio COVID-19 Ag has been reported in a very diverse range, and the target test group’s diversity is believed to be the most significant factor. A total of 39 studies evaluating Panbio COVID-19 Ag were included in the meta-analysis, and the sensitivity varied from 23.1% to 95.0% [7]. Among them, a multicenter study evaluating 958 patients (RT-PCR positivity rate, 37.5%) reported a sensitivity of 90.5% (95% CI 87.5%−93.6%); however, this study only included specimens collected from individuals 7 days from the onset of symptoms or from exposure to a confirmed case of COVID-19 [10]. Another study evaluated 293 symptomatic (55.1%) and 239 asymptomatic (44.9%) patients in the emergency room of a university hospital, with a sensitivity of 41.2% [11].
A meta-analysis revealed that the average sensitivity was 73.0% (95% CI 69.3%−76.4%) for symptomatic participants, which was higher than the average sensitivity of 54.7% for asymptomatic participants [8]. The performance evaluation results provided by the manufacturers were superior to those reported in published studies [9]. In the evaluation of specimens (140 positive, 445 negative) from symptomatic individuals, the sensitivity was 91.4% (95% CI 85.5%−95.5%). According to the Ct value, the sensitivity was 97.6% (95% CI, 93.2%−99.5%) for specimens with a Ct ≤30 and 94.1% (95% CI, 88.7%−97.4%) for those with a Ct ≤33. The results evaluated using specimens from asymptomatic individuals were less sensitive than those evaluated using specimens from symptomatic individuals. Of 483 specimens, 50 were RT-PCR positive, and the sensitivity for all positive specimens was 66.0% (95% CI 51.2%−78.8%). According to Ct value, the sensitivity was 93.8% (95% CI, 79.2%−99.2%) for specimens with a Ct ≤30 (n=32) and 80.0% (95% CI, 64.4%−90.9%) for those with a Ct ≤33 (n=40) [9]. Because clinical information was not collected in the present study, it was not possible to determine sensitivity according to whether patients were symptomatic or asymptomatic.
The lateral flow immunochromatographic assay interprets the result with the presence/absence of test and control lines. However, in some cases, it is difficult to read the lines by the naked eye when the intensity is low [12], and there may be differences among operators. Using a reader is known to improve these limitations [13], but Panbio COVID-19 Ag testing results are interpreted by naked eyes without a reader. Previous studies evaluating Panbio COVID-19 Ag have reported no differences between operators [14,15]. However, in this study, retests were performed for seven specimens due to discrepancies in the initial results interpreted by two operators. All specimens to be retested had Ct values in the range of 20−30 and, in particular, all specimens in the range of 25−30 (n=5) were retested. Because the detection limit Ct of Panbio COVID-19 Ag was 26.67, most of the retested specimens must have had virus concentrations near the detection limit. The cut-off, which is the criterion for distinguishing positive from negative in a qualitative test, can be both positive and negative through repeated tests. However, from a practical perspective, it is a disadvantage that a result is unclear or a retest is required. Although the COVID-19 RADT is simpler and easier to perform than molecular testing and does not require skilled operators, visual reads and interpretation may require some training and experience.
The World Health Organization (WHO) recommends that SARS-CoV-2 Ag RADTs that meet the minimum performance requirements of ≥80% sensitivity and ≥97% specificity compared with a nucleic acid amplification test reference assay can be used to diagnose COVID-19 in suspected cases of SARS-CoV-2 infection. According to the WHO guidelines, RADTs are less sensitive than nucleic acid amplification tests, particularly in asymptomatic populations; however, careful selection of cohorts for testing can mitigate this limitation. The WHO suggests that RADTs perform best in individuals with high viral loads and early in the course of infection, and will be most reliable in settings where the regional prevalence of SARS-CoV-2 infection is ≥5% [2]. The European Centre for Disease Prevention and Control (ECDC) agrees with the WHO minimum performance criteria of ≥80% sensitivity and ≥97% specificity but also advocates the use of higher performance tests (≥90% sensitivity and >98% specificity) [16]. In this study, Panbio COVID-19 Ag met the performance requirements of WHO and ECDC in specimens with Ct values <25.
Rather than evaluating the performance of Panbio COVID-19 Ag, this study aimed to verify its performance for implementation in a clinical laboratory; as such, there is a limitation in that the number of specimens was not sufficient. In addition, because the clinical information of individuals was not collected, the presence or absence of symptoms could not be determined.
In this study, the performance of the Panbio COVID-19 Ag was verified. Considering that Panbio COVID-19 Ag yielded high sensitivity at Ct values <25 and yielded results within 15 minutes, we believe that Panbio COVID-19 Ag can be useful in clinical laboratories. However, for RADTs to demonstrate proper performance, it is desirable to define a target group to be tested and to use it in limited situations, while considering the regional prevalence of COVID-19 and the accessibility of molecular tests and methods.
