**Contents**


Reprinted from: *Diagnostics* **2020**, *10*, 960, doi:10.3390/diagnostics10110960 ............ **129**

## **Andreas Hahn, Hagen Frickmann and Ulrike Loderst¨adt**


Reprinted from: *Diagnostics* **2022**, *12*, 716, doi:10.3390/diagnostics12030716 ............. **185**

## **About the Editor**

## **Tivani P. Mashamba-Thompson**

Tivani P. Mashamba-Thompson is a Medical Scientist (registered by the Health Professions Council of South Africa) and a Professor of Diagnostics Research. She was appointed as a Full Professor and a Deputy Dean of Research and Postgraduate Studies for the Faculty of Health Sciences, University of Pretoria, with effect from 1 January 2021. She also serves as a nonexecutive board member of the National Health Laboratory Services (NHLS) as a Deputy Chairperson of the NHLS Research and Innovation Committee. She is also a member of the South African Department of Health Office of Health Standards Compliance technical task team. She has previously served as a diagnostics expert advisor for Abbott International.

## *Editorial* **Implementation Science: Bridging the Gap between Point-of-Care Diagnostics Research and Practice**

**Tivani P. Mashamba-Thompson**

Faculty of Health Sciences, University of Pretoria, Pretoria 0002, South Africa; tivani.mashamba-thompson@up.ac.za

The advent of the novel Coronavirus 2019 (COVID-19) pandemic has fuelled technological innovation and led to the increased research on development and deployment of new diagnostics for use at point-of-care (POC). The rapid uptake of the newly developed diagnostics requires a systematic approach to bridge the research-to-practice gap. Implementation science (IS) involves the use of evidence-based practices (EBPs) that are characterised by both quality improvement and dissemination methods aiming to promote the scaling up of health interventions such as POC diagnostics to enhance quality and outcomes [1]. This research approach employs transdisciplinary quantitative and qualitative designs with solid grounding in theory. Implementation science studies are designed to enable identification of factors that impact uptake of health interventions across multiple levels, including the patient, provider, clinic, facility, organisation, and often the broader community and policy environment. In this Special Issue, we present a summary of twelve studies that employed implementation science approaches demonstrating research aimed at optimising implementation various kinds of point-of-care (POC) diagnostics among different population groups and different healthcare settings globally.

Nucleic acid amplification tests (NAATs), such as the reverse-transcriptase polymerase chain reaction (RT-PCR) tests, were the first to be developed and widely deployed at the beginning of the COVID-19 pandemic. These tests were designed to detect viral RNA. A RT-PCR-positive result is highly specific for the presence of viral nucleic acid. A study on portable, easy-to-use SARS-CoV-2 RT-PCR POC diagnostic device showed that the POC test's performance was comparable to that of the conventional RT-qPCR tests [2]. It also showed that the POC RT-PCT displayed the characteristics of a POC test, with a short turnaround time for quick patient isolation. The implementation of the first official approved self-administered rapid antigen tests (RATs), CoviSelf, was assessed as part of a community-level COVID-19 pandemic response in rural India [3]. Results of the study show that self-administered RATs have potential in rural settings as they are cheap, quick and reasonably reliable. However, the tests kits were found to not be user-friendly and required equitable distribution to minimise the spread of COVID-19. One study conducted at a hospital setting in Germany evaluated the lung ultrasound (LUS) in 101 symptomatic patients with suspected COVID-19 infection at hospital admission [4]. Results of this evaluation demonstrate that early LUS examination as part of in-patient admission provides a diagnostic gain and is valuable for the clarification of SARS-CoV-2-suspected patients at hospital admission. An assessment of the correlation between hospital-based lung ultrasound (LUS) and chest X-ray (CXR) findings in 247 COVID-19 patients in Spain showed positive results [5]. A significant correlation between LUS findings and CXR in patients with suspected or confirmed COVID-19 infection and supported use of the LUS exam as the initial POC diagnostic imaging test was demonstrated.

Interventions such as mobile Health (mHealth) have been shown to help enhance health service delivery and access to disease diagnosis. Increased availability and use of mHealth to help improve access to diagnostics is recommended for resource-limited settings. Osei et al., 2021 conducted a study to examine the availability and use of mHealth

**Citation:** Mashamba-Thompson, T.P. Implementation Science: Bridging the Gap between Point-of-Care Diagnostics Research and Practice. *Diagnostics* **2022**, *12*, 1648. https://doi.org/10.3390/ diagnostics12071648

Received: 5 July 2022 Accepted: 5 July 2022 Published: 6 July 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

for disease diagnosis and treatment support by healthcare professionals in the Ashanti Region of Ghana [6]. The findings in this study show minimal use of mHealth for disease diagnosis and treatment support by healthcare professionals at rural clinics, which in turn affects the accessibility of medical care in such resource constrained settings. One of the contributing factors for equitable accessibility, distribution and availability of POC tests is the agility of supply chain management systems. Maluleke et al., 2021, conducted a scoping review to systematically map evidence on supply chain management systems for POC diagnostics services with a focus on optimising the SARS-CoV-2 testing capacity in resourcelimited settings [7]. The results of the review showed has showed that there is limited research evidence on POC diagnostics supply chain management systems globally. In addition to ensuring accessibility of new POC diagnostics to those who need it, acceptance by stakeholders is key to enabling uptake and appropriate usage of available diagnostics. Thirty-one stakeholders involved in adoption of POC ultrasound (POCUS) implementation in a US academic medical centre were interviewed to determine their perspective on this diagnostic intervention [8]. The Practical Robust Implementation and Sustainability Model (PRISM) was employed to guide framework analysis for the data collected from interviews with stakeholders. Stakeholders considered the following overarching themes to be important for the adoption and fidelity of POCUS by clinicians and health systems: clinical impact; efficiency; cost; development of credentialing policies; and robust quality assurance processes.

The need to optimise and monitor POC diagnostics quality management systems has been emphasised in other studies [9–11]. A systematic review and meta-analysis of global evidence showed moderate accuracy of mobile-linked POC diagnostics in detecting infections and recommended the development and deployment of more highly accurate mHealth-linked POC diagnostics [11]. Evaluation of the prognostic capacity of ΔLA (delta lactate) (correlation between prehospital lactate (pLA) and hospital lactate (hLA)) with respect to in-hospital two-day mortality among emergency department patients was also performed. Results of the evaluation demonstrate that lactate clearance in the initial moments of ED care appears to be a more reliable prognostic index than a baseline lactate value taken alone [9]. Hahn et al., 2021, employed a theorical model to demonstrate sensitivity-optimised screening as a "diagnostics as prevention" strategy for managing infectious diseases using HIV infections as a prototype [10]. The model was designed to increase case definitions for diagnostic test sensitivity of by compensating for the limited sensitivity of a test in the early stage of a disease. The model also enabled inclusion of known symptoms of the respective disease stage and RDT-based exposition prevention in a pandemic. This concept was widely used for the management of COVID-19 through applying rapid diagnostic tests with imperfect diagnostic accuracy.

Despite the technological advancements presented by the COVID-19 pandemic, disruption to services directed at management of existing infectious and deadly pandemics, such as TB was reported during the early stages of the pandemic. Dlangalala et al., 2021 systematically mapped available evidence on TB services at the primary healthcare (PHC) level during the COVID-19 period using as scoping review study [12]. The study revealed that pandemic mitigation strategies, as well as the fear and stigma experienced at the beginning of the pandemic, may have limited uptake of TB diagnostic services at PHC level. The presented poor TB service up-take may also be a result of poor health literacy, which has been shown to be generally low among vulnerable populations. In this context, health literacy would be defined in line with access to technology enabling disease screening, diagnosis and linkage to care, i.e., diagnostics literacy. The COVID-19 pandemic has revealed diagnostics literacy as one of the unmet needs among vulnerable populations that continue to experience short- and longer-term socio-economic consequences. To address this unmet need a multi-level diagnostics literacy advocacy model was proposed to help improve diagnostic uptake among vulnerable populations [13]. Sustainable implementation of the proposed model will require involvement of all key stakeholders.

**Funding:** This research received no external funding.

**Conflicts of Interest:** The author declares no conflict of interest.

## **References**


## *Brief Report* **Analytical Evaluation of Visby Medical RT-PCR Portable Device for Rapid Detection of SARS-CoV-2**

**Adriana Renzoni 1,\*, Francisco Perez 2, Marie Thérèse Ngo Nsoga 2,3, Sabine Yerly 1, Erik Boehm 1, Angèle Gayet-Ageron 4, Laurent Kaiser 1,2,3 and Manuel Schibler 1,3**


**Abstract:** Extended community testing constitutes one of the main strategic pillars in controlling the COVID-19 pandemic. Reverse transcription PCR (RT-PCR) targeting the SARS-CoV-2 genome on nasopharyngeal swab samples is currently the reference test. While displaying excellent analytical sensitivity and specificity, this test is costly, often requires a substantial turnaround time, and, more importantly, is subject to reagent and other material shortages. To complement this technology, rapid antigen tests have been developed and made available worldwide, allowing cheap, quick, and decentralized SARS-CoV-2 testing. The main drawback of these tests is the reduced sensitivity when RT-PCR is the gold standard. In this study, we evaluate Visby an innovative, portable, easy-to-use RT-PCR point-of-care (POC) diagnostic device. Our retrospective analysis shows that overall, compared to the Cobas 6800 RT-qPCR assay (Roche), this RT-PCR POC technology detects SARS-CoV-2 RNA with 95% sensitivity (95%CI = 86.3–99%) and 100% specificity (95% CI = 80.5–100%). For samples with cycle-threshold values below 31, we observed 100% sensitivity (95% CI = 66.4–100%). While showing an analytical sensitivity slightly below that of a standard RT-qPCR system, the evaluated Visby RT-PCR POC device may prove to be an interesting diagnostic alternative in the COVID-19 pandemic, potentially combining the practical advantages of rapid antigen tests and the robust analytical performances of nucleic acid detection systems.

**Keywords:** SARS-CoV-2; rapid diagnostic techniques; POCT techniques

## **1. Introduction**

Complementary to prevention measures, such as social distancing, mask wearing, and hand hygiene, extensive population screening with isolation of positive cases is one of the key means of controlling the COVID-19 pandemic caused by severe SARS-CoV-2 [1]. SARS-CoV-2 RNA detection by RT-qPCR) in nasopharyngeal respiratory samples is currently considered to represent the gold standard for COVID-19 diagnosis. This technology, known for its high analytical sensitivity and specificity, has been applied to clinical virology diagnostics since the 1990s. On the other hand, it is costly, requires complex laboratory infrastructure, often has a high turnaround time, and even more importantly, RT-qPCR reagents and plastic materials are subject to shortages in the current crisis [2].

Rapid antigen tests are increasingly widely used, allowing for user-friendly, low-cost, quick, and decentralized testing. The main limitation of these tests is the reduced sensitivity in comparison to RT-qPCR, around 90% in the best-case scenario. Innovative nucleic acid

**Citation:** Renzoni, A.; Perez, F.; Ngo Nsoga, M.T.; Yerly, S.; Boehm, E.; Gayet-Ageron, A.; Kaiser, L.; Schibler, M. Analytical Evaluation of Visby Medical RT-PCR Portable Device for Rapid Detection of SARS-CoV-2. *Diagnostics* **2021**, *11*, 813. https:// doi.org/10.3390/diagnostics11050813

Academic Editor: Tivani P. Mashamba-Thompson

Received: 8 March 2021 Accepted: 28 April 2021 Published: 29 April 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

detection point-of-care (POC) diagnostic technologies are now being developed, potentially combining the excellent analytical performance of RT-qPCR and the practical advantages of rapid antigen tests [3,4].

An alternative to the detection of antigen is the amplification of nucleic acid by reverse transcription in combination with isothermal methods such as: loop-mediated isothermal amplification (RT-LAMP) [5–7], recombinase polymerase amplification (RT-RPA) [8–10], and nicking-endonuclease amplification reactions (RT-NEAR) [11–14]; which do not need a thermal cycler, potentially enabling their use at POC facilities. The reference standard of RT-PCR also detects viral nucleic acid, and some devices capable of RT-PCR have been developed that are suitable for POC use, such as the DNAnudge Covid-19 test.

Recently, Visby Medical Inc. (Visby) has developed and begun manufacturing small palm-sized RT-PCR devices capable of detecting SARS-CoV-2. This device contains all necessary reagents, and is operated simply by loading a sample and sequentially pushing buttons; thus representing a self-contained easy-to-use RT-PCR POC diagnostic device. The device makes use of a continuous flow serpentine PCR channel combined with an enzyme linked detection chemistry to produce a colorimetric signal that is easily observable by the user. The device, originally developed for detection of bacterial targets in a sexually transmitted infection panel [15] has been adapted for rapid RT-PCR identification of the SARS-CoV-2 N gene and has received an FDA EUA for use in high and moderate complexity settings. More recently, on 8 February 2021, the Visby test received EUA for use in CLIAwaived settings (https://www.fda.gov/media/145917/download, accessed on 28 April 2021). Of note, the device may only be used once before requiring disposal or refurbishment by the manufacturer. Visby is intended for testing individuals suspected of COVID-19 by their health care provider using nasopharygeal, anterior nasal or mid-turbinate swabs collected into transport media,

In this study, we tested the new RT-PCR POC Device developed by Visby to detect the presence of SARS-CoV-2 RNA in a retrospective study using frozen SARS-CoV-2 positive nasopharyngeal samples using Cobas 6800 RT-qPCR assay (Roche) as the gold standard.

## **2. Material and Methods**

## *2.1. Patient Samples and Assay Controls*

According to the Swiss Ethics Committee on research involving leftover clinical samples, this study occurred in the context of a method quality validation in an emergency setting and therefore did not require any authorization from our ethics committee. The vast majority of positive samples come from outpatient symptomatic patients enrolled in a COVID-19 screening center at the Geneva University Hospitals. Sixty-one confirmed positive SARS-CoV-2 nasopharyngeal swab (NPS) samples were tested. Sample positivity, that is samples with a cycle-threshold (Ct) value below 35, was previously determined by using the Cobas 6800 SARS-CoV-2 RT-PCR assay (Roche, Switzerland) and stored in Copan UTMTM or an in-house DMEM-based media that has been validated in our institution [16]. Samples were collected around 6 weeks or less prior to device testing and frozen at −80 ◦C. Patient specific data include demographics, symptom severity, and date of symptoms onset. Seventeen SARS-CoV-2 PCR negative NPS samples were tested as negative controls. Two external positive and negative controls (NATSARS (COV2)-ERC and NATSARS (COV2)- NEG, respectively) were obtained from ZeptoMetrix (Distributed by Helvetica Health Care, Geneva, Switzerland). External controls were performed when different operators were running the experiments. In total, 17 samples were considered as controls in this study.

## *2.2. Visby Medical POC Device Testing*

Analysis of samples and external controls was performed following the manufacturer protocol (https://www.fda.gov/media/142228/download, accessed on 28 April 2021) with some modifications, as we performed a retrospective study using frozen SARS-CoV-2 samples. Briefly, specimens were mixed by inversion 5 times, and 650 μL was inserted into provided dilution buffer using the special Pasteur pipette provided with the device

(Visby Medical) (Figure 1A). The diluted sample was mixed five times by inversion, 650 μL was taken with a second Pasteur pipette and dispensed immediately into the device (Button No. 1). After loading the sample; sample extraction, reverse transcription, PCR amplification, and result visualization were started by sequentially pushing down buttons 1, 2 and 3 and plugging the device in. A successful device start is indicated by a white light. After 30 min, a green or red check mark appears confirming a valid or invalid device performance, respectively (Figure 1B). Samples run in invalid devices were retested using a new device.

**Figure 1.** (**A**) Schematic steps for Visby Medical kit procedure. (1) Kit contents. (2) Patient sample preparation, with dilution buffer and cassette. (3–5) The sample is put into the dilution buffer and mixed by inversion. (6–7) 650 uL of the diluted sample is inserted into the device using the pastette. (8) Buttons 1, 2, and 3 are pushed, and the device is connected to electrical power. (9) Results displayed. A white checkmark appears, indicating that the test is in progress (see description below). (**B**) The left panel shows the white or green or red check marks denoting correct power and processing of the device, a valid test result ready to be read, or an invalid result due to an electrical error, respectively. The right panel shows an example of a valid (green light) and positive (visible upper and lower purple spots) or negative (only the upper purple spot is visible).

Results of valid devices were displayed in the test result window as visible purple marks (Figure 1B). A purple mark in the upper position (indicated by a "√" symbol) indicates a valid sample extraction and amplification reaction, which was determined by detecting the human 18S ribosomal structural RNA present in nasopharyngeal samples. A purple mark in the lower position (indicated by a "+" symbol) indicates a positive amplification of SARS-CoV-2 nucleic acid. Visualization of both upper and lower position purple marks denotes a positive SARS-CoV-2 sample, while visualization of a single upper position purple mark denotes a negative SARS-CoV-2 sample (Figure 1B). The external positive and negative controls were used to ensure that test reagents are working correctly. Each control was run when a new operator was running the tests. Results interpretation was performed in a blinded manner by two laboratory staff members.

## *2.3. Comparison RT-qPCR Assay*

The validated Roche Cobas 6800 SARS-CoV-2 RNA extraction and RT-qPCR assay were used as a reference assay (Cobas SARS-CoV-2 Ref 09175431190; Cobas SARS-CoV-2 Control kit Ref: 09175440190; Cobas 6800/8800 Buffer Negative Control kit Ref 07002238190; Roche, Switzerland). In case of discrepant results between the Cobas and Visby assays, viral RNA genome detection was also performed using GeneXpert (Ref: XPRSARS-CoV2-10 from Cepheid). Cobas targets ORF1a/b and a pan-Sarbecovirus conserved region of the E gene, while GeneXpert targets N and E genes.

## *2.4. Evaluation of Analytical Performances*

The sensitivity and specificity of the device were determined by comparison to the Cobas assay, according to sensitivity = TP/(TP + FN); Specificity = TN/(TN + FP); where TP = true-positives, FN = false-negatives, TN = true-negatives, and FP = falsepositives. Confidence intervals of 95% (95% CI) were calculated using the Clopper–Pearson method [17]. We reported overall sensitivity and specificity, as well as sensitivity by delay from symptoms onset and by categories of Ct values.

## *2.5. Limit of Detection (LoD)*

As recommended by the manufacturer, a SARS-CoV-2 negative nasopharyngeal cell matrix was prepared to perform LoD analyses. First, nasopharyngeal samples, which were previously verified to be SARS-CoV-2 negative, were pooled to obtain a total of 12 mL. The pooled nasopharyngeal sample was retested by Cobas RT-qPCR to ensure that the sample pool was negative and immediately frozen at −80 ◦C. SARS-CoV-2 viral stock was produced as follows: SARS-CoV-2 isolates were collected from ex vivo infections of airway epithelia cultured in an air-liquid interface as previously described [18]. Briefly, after 3 h of apical virus inoculation, in vitro differentiated respiratory tissues (MucilAirTM, Epithelix SARL, Geneva, Switzerland) were washed three times with PBS (Phosphate Buffered Saline, Sigma, St. Louis, MO, USA) and incubated in MucilAirTM medium at 33 ◦C and 5% CO2. Four days post infection, respiratory tissues culture supernatant is removed and 200μL of MucilAirTM medium was added apically and viral culture supernatant was collected after 20 min of incubation at 33 ◦C at 5% CO2. Viral load in the culture supernatant was determined by RNA quantification. Briefly, RNA was extracted with NucliSens easyMAG (bioMérieux, Marcy-l'Étoile, France), and quantified by RT-qPCR using SuperScript™ III Platinum™ One-Step qRT-PCR Kit (Invitrogen, Carlsbad, CA, USA) in a CFX96 Thermal Cycler (Bio-Rad, Hercules, CA, USA). RT-qPCR was performed using a specific set of primers and probes targeting the SARS-CoV-2 E gene (forward primer: 5 -ACAGGTACGTTAATAGTTAATAGCGT-3 , reverse primer: 5 -ATATTGCAGCAGTACG CACACA-3 , and the probe: 5 -6-FAM- ACACTAGCCATCCTTACTGCGCTTCG-BBQ-3 ) [19]. In vitro transcribed SARS-CoV-2 E gene RNA (EVAg, Essen, Germany) was used as a reference standard to convert Ct values into RNA copies/mL. Data were analyzed using Bio-Rad CFX maestro software (Bio-Rad, Essen, Germany). Quantified culture supernatant was finally diluted in PBS to a concentration of 2.1 × <sup>10</sup><sup>6</sup> copies/mL.

## **3. Results**

## *3.1. Visby POC Device and Cobas SARS-CoV-2 Nucleic Acid Detection Concordance*

Frozen samples were thawed, dispensed immediately into dilution buffer provided with the device, and further processed to avoid nucleic acid damage as much as possible. After 30 min of sample processing, purple mark recording was performed immediately in devices displaying the green light. An invalid run occurred for a single sample, but after retesting, yielded a valid result.

The clinical accuracy of the device compared to Cobas RT-qPCR was calculated. From the 61 Cobas-positive samples (Ct values from 15.5 to 34), the device detected 58 positive samples with 3 false-negative samples. All 17 Cobas-negative samples were also found to be negative by the device (Table 1). We observed a sensitivity of 95.1% (CI 95% = 86.3–99%) and a specificity of 100% (CI 95% = 80.5–100%). An analysis by Ct values showed that the device achieved 100% sensitivity in samples with Ct values below 35 (Table 2). As shown in Table 3, the device achieves > 95% sensitivity with tests performed from 2 days after onset of symptoms, and 100% (73.5–100%) from 4 days post symptoms onset. We further evaluated Visby results in relation to viral loads and days post onset of symptoms, and observed that the few false-negative results were observed with three samples with low viral loads (Figure 2).

Since discrepant results might be explained by the effects of sample freezing on RNA stability, we retested the discordant samples with the Cobas assay. Nearly identical positive Cobas Ct values were found, showing that RNA degradation upon thawing samples could not explain the negative device results (Table S1). Discrepant sample results might be explained by detection of different genes used by the device (N-gene) and the Cobas (E and Orf1 genes) assay. We therefore analyzed discrepant samples with the SARS-CoV-2 GeneXpert assay, targeting N and E genes, however, both genes were detected (Table S1).

**Table 1.** Sensitivity and specificity of the Visby device.


Sensitivity: 95% (95% CI 86–99)/specificity: 100% (95% CI 80.5–100).


**Table 2.** Sensitivity of Visby device depending on RT-qPCR CT values.

**Table 3.** Sensitivity of Visby device by the delay since symptom onset.


**Figure 2.** Detection of SARS-CoV-2 by Visby device based on viral loads and the time of symptom onset. SARS-CoV-2 viral loads by the time of symptom onset for the symptomatic and RT-qPCR positive individuals. Black and red dots represent positive or negative Visby results, respectively. Dotted line: 6 log10 SARS-CoV-2 RNA copies/mL shows the VL threshold limit determining the number of culturable viruses. Note: Samples classified at day 0 include 4 samples with unknown dates of symptoms.

## *3.2. Analytical Limit of Detection (LoD)*

A clinical SARS-CoV-2 isolate was cultured in an air-liquid interface respiratory epithelium system (mucilAir). Viral supernatant concentration was quantified using Egene standards from the European virus archive and diluted to a final concentration of 2.1 × <sup>10</sup><sup>6</sup> copies/mL. Live virus from cell culture supernatant were serially diluted (105, 104, 103, 102, 101 copies/mL) in SARS-CoV-2 negative nasopharyngeal cell matrix to evaluate the LoD. We tested a single replicate of each dilution and found them all positive, showing a Visby positive detection with low viral loads (102 copies/mL). Our data are in agreement with the manufacturer LoD determination, which showed a 95% detection rate for 1112 genomic copies/mL and a detection rate of 45% for 125 genomic copies/mL (Visby medical package insert).

## **4. Discussion and Conclusions**

The aim of this study was to evaluate the Visby POC device to detect the presence of SARS-CoV-2 RNA in previously collected frozen clinical nasopharyngeal samples, in comparison to the Cobas 6800 RT-qPCR assay (Roche). The devices detected SARS-CoV-2 RNA with 95% sensitivity and 100% specificity.

Few discrepant results were observed that might be explained by potential N-gene mutations that are undetectable by the device, similarly to previously published observations showing that failure to detect SARS-CoV-2 by the Cobas 6800 assay was linked to E-gene mutations [20]. This is a possible scenario, but an unlikely one, as extensive in silico analysis of variants were detected by Visby primers and probes (https: //www.fda.gov/media/142228/download, accessed on 28 April 2021). Having excluded potential issues related to different viral genes targeted with both assays, the reduced sensitivity observed with the devices probably results from the different technologies applied, which differ in technical details such as buffers and sample processing methods. Visby's test is not directly compatible with most viral transport media (manufacturer package insert). While dilution of samples collected in transport media mitigates the inhibitory effects of the transport media, it reduces the test sensitivity, which could account for the 3 Cobas-positive samples that the device missed. However, this dilution step did not prevent detection of other samples with similar viral loads. We cannot exclude the possibility of PCR inhibiting substances within these nasal samples that prevent gene amplification exclusively with the device methodology [21].

Dilution of the sample collected into transport media to mitigate the inhibitory effects of the transport media may reduce the sensitivity of the test. While a remnant sample evaluation was performed in this study, it is reasonable to assume that direct swab elution with no dilution step would yield an improvement in sensitivity. A prospective clinical study would be important to assess the clinical sensitivity of the direct swab method.

In the present analytical study, the POC Device performs less well at Ct values ≥35; however, the reduction in sensitivity is relatively unimportant since high Ct values probably indicate a low transmission risk [22]. Furthermore, the sensitivity displayed by the device is still substantially higher than that of the 85% to 90% of the best antigen tests [23–27].

This device operates by detecting viral nucleic acids such as the reference standard (RT-PCR) and isothermal amplification tests. RT-LAMP amplification unfortunately has issues such as: non-specific amplification that can be a problem in later steps as a result of "carryover contamination", "product cross contamination" [28–30], or primer hybridization [31].

In contrast, the device evaluated here uses the same robust and well established principle used by the gold-standard tests, and thus is expected to be reliable and accurate. Other RT-PCR devices suitable for POC diagnostics are on the market (i.e., NudgeBox or LIAT) but are less portable than the Visby POC device evaluated here, and may be unavailable or have a sample processing time of up to 90 min that may be prohibitive for some envisioned POC uses [4].

It is important to note the limitations of our study. The selection of samples tested is not representative of an epidemiological trend because only frozen samples were used, and no prospective recruitment of patients was performed with both methods concomitantly. No estimation of positive or negative predictive values can be performed as samples were not consecutively collected and prevalence is not interpretable. Future clinical evaluations, including head-to-head comparisons with antigen tests and isothermal tests, testing of SARS-CoV-2 asymptomatic-contacts, and screening healthcare workers will help to better position SARS-CoV-2 nucleic acid detection among the POC diagnostics.

In summary, the Visby device appears to be an attractive alternative POC test, performing nearly as well as conventional RT-qPCR tests, while displaying the appealing characteristics of a POC test. It is available in a portable format with minimal requirement of technical skills. After sample acquisition, less than 1 min is needed for treatment and loading samples onto de VISBY device. Viral extraction, amplification and easy-readout are obtained after exactly 30 min. This short turnaround time allows for rapid patient isolation or orientation decisions, e.g., in nursing homes or even hospital settings lacking more complex laboratory equipment. A potential disadvantage includes the generation of waste that remains an issue to be addressed. The company is exploring recycling components to reduce generated waste.

**Supplementary Materials:** The following are available online at https://www.mdpi.com/article/ 10.3390/diagnostics11050813/s1. Table S1: Positive and negative RT-qPCR Cobas samples tested on RT-PCR Visby diagnostic device. C = corncordant result; NC = Non-concordant results. INV = invalid result.

**Author Contributions:** A.R., conception, experimental manipulation and analysis, writing the manuscript. F.P., experimental manipulation and critical reading of the manuscript. M.T.N.N. experimental manipulation and critical reading of the manuscript. S.Y., conception and critical reading of the manuscript. E.B., conception and critical reading of the manuscript. A.G.-A., statistical analysis and critical reading of the manuscript. L.K., conception and critical reading of the manuscript. M.S., conception, experimental analysis, writing and critical reading of the manuscript. All authors have read and agreed to the published version of the manuscript.

**Funding:** Marie Thérèse Ngo Nsoga is a beneficiary of the excellence grant from the Swiss Confederation and the grant from the humanitarian commission of the University Hospital of Geneva.

**Institutional Review Board Statement:** According to the Swiss Ethics Committee on research involving leftover clinical samples, this study occurred in the context of a method quality validation in an emergency setting and therefore did not require any authorization from our ethics committee.

**Acknowledgments:** We thanks Pascal Cherpillod for the pictures.

**Conflicts of Interest:** The authors declare that there are no conflicts of interest.

## **References**


## *Article* **Can Self-Administered Rapid Antigen Tests (RATs) Help Rural India? An Evaluation of the CoviSelf Kit as a Response to the 2019–2022 COVID-19 Pandemic**

**Marika Vicziany 1,\* and Jaideep Hardikar <sup>2</sup>**


**Abstract:** This paper evaluates India's first officially approved self-administered rapid antigen test kit against COVID-19, a device called CoviSelf. The context is rural India. Rapid antigen tests (RATs) are currently popular in situations where vaccination rates are low, where sections of the community remain unvaccinated, where the COVID-19 pandemic continues to grow and where easy or timely access to RTPCR (reverse transcription-polymerase chain reaction) testing is not an option. Given that rural residents make up 66% of the Indian population, our evaluation focuses on the question of whether this self-administered RAT could help protect villagers and contain the Indian pandemic. CoviSelf has two components: the test and IT (information technology) parts. Using discourse analysis, a qualitative methodology, we evaluate the practicality of the kit on the basis of data in its instructional leaflet, reports about India's 'digital divide' and our published research on the constraints of daily life in Indian villages. This paper does not provide a scientific assessment of the effectiveness of CoviSelf in detecting infection. As social scientists, our contribution sits within the field of qualitative studies of medical and health problems. Self-administered RATs are cheap, quick and reasonably reliable. Hence, point-of-care testing at the doorsteps of villagers has much potential, but realising the benefits of innovative, diagnostic medical technologies requires a realistic understanding of the conditions in Indian villages and designing devices that work in rural situations. This paper forms part of a larger project regarding the COVID-19 pandemic in rural India. A follow-up study based on fieldwork is planned for 2022–2023.

**Keywords:** COVID-19; Indian pandemic; rapid antigen tests (RATs); CoviSelf; CoWIN; digital divide; rural India; Indian villagers; poverty; discourse analysis; qualitative medical/health research strategies

## **1. Introduction**

As the Indian COVID-19 pandemic enters a new phase in 2022 with the rising dominance of the highly infectious Omicron variant, the use of self-administered RATs is increasing in India and other countries as a way of relieving the pressure on stressed clinical and hospital settings that administer and analyse reverse transcription polymerase chain reaction (hereafter RTPCR) tests [1]. As a third pandemic wave emerged in India, on 10 February 2022, the Indian Council of Medical Research (hereafter ICMR) announced that a positive result from a self-administered RAT was the equivalent of a positive result from an RTPCR procedure and that repeat testing using the latter was no longer necessary [2]. In this paper, we evaluate the first self-administered RAT to receive official approval by ICMR some nine months earlier, on 10 May 2021 [3]—CoviSelf. ICMR is the 'apex body in India for the formulation, coordination and promotion of biomedical research' [4] and sits within the Department of Health Research (Ministry of Health and Family Welfare), Government of India. As the first Indian made, self-administered RAT to receive official recognition,

**Citation:** Vicziany, M.; Hardikar, J. Can Self-Administered Rapid Antigen Tests (RATs) Help Rural India? An Evaluation of the CoviSelf Kit as a Response to the 2019–2022 COVID-19 Pandemic. *Diagnostics* **2022**, *12*, 644. https://doi.org/ 10.3390/diagnostics12030644

Academic Editor: Tivani P. Mashamba-Thompson

Received: 22 November 2021 Accepted: 21 February 2022 Published: 6 March 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

CoviSelf deserves our scrutiny. This diagnostic device for home use was developed by the private Indian company Mylab Discovery Solutions Pvt Ltd, based in Pune in the state of Maharashtra [5]. The tone of the company's press release of 10 May 2021 reflects the excitement surrounding the development of this innovative testing kit during increasingly desperate times [3]:

With CoviSelfTM, Mylab aims to make testing reach *the doorstep of every Indian* to help them fight the second and any subsequent waves of [*sic*] pandemic. *Now, any citizen can test for Covid-19 themselves*, isolate and seek treatment quickly. Early detection can help save thousands of lives and significantly reduce [*sic*] burden on hospitals. The test can be purchased without a prescription from local pharmacies and online channel partners [italics added by authors of this article]

The claims about delivering a diagnostic tool for containing COVID-19 to 'every Indian ... any citizen' are ambitious and admirable but need to be tested. Our analysis is not based on a scientific or laboratory assessment of the accuracy of this testing device. Few reliable studies of RAT evaluations have been published; perhaps the best-known exception is the UK's Cochrane Report, published in March 2021 [6]. Until detailed and systematic quantitative data are published regarding Indian RAT devices, the official list of ICMR-approved RATs is our only source of information about the four criteria used to validate and approve the RATs distributed in India, including those used in home testing [7]. Moreover, official criteria are not an exact guide to how RATs function in medical, health and home settings. In mid-January 2022, ICMR published a list of eight self-test kits that it had approved, in addition to another list of five that it did not approve [7]. At the top of the approved list appears CoviSelf by Mylab.

As the first government sanctioned, self-administered RAT for COVID-19, our analysis of CoviSelf also provides a benchmark for evaluating the others that have become available in India since 10 May 2021. However, a scientific appraisal of the sensitivity and specificity of the RATs in India is not our aim in this paper [8]. Rather, our evaluation is based on the extent to which the CoviSelf kit is compatible with the practical needs and concerns of Indian villagers. We do this by considering the characteristics of CoviSelf in the context of the socioeconomic circumstances in Indian villages. Some 66% of Indians live in rural areas [9], so the applicability of CoviSelf has widespread implications for addressing the country's pandemic while India waits for additional medical technologies to become available—for example, anti-COVID medication [10]. Yet, there is nothing inevitable about the arrival of such drugs or the trajectory that the pandemic could take. Given this, lessons can be learnt about the longer-term contribution that self-administered RATs could make to villagers' wellbeing by analysing the practicality of the CoviSelf kit. Mylab's aim for CoviSelf to reach 'every Indian and any citizen' is best tested against the socioeconomic parameters that define villages, where the majority of Indians live.

Our paper provides a qualitative evaluation of CoviSelf within a broad socioeconomic context. In 2008, Sir Michael Rawlins stressed the importance of recognising the value of qualitative research for health interventions. He questioned the hierarchy of evidence that gave greater importance to randomised clinical trials as the basis for health decisions. 'Observational evidence' or qualitative data, including 'expert opinion', formed an important component of therapeutic recommendations, he said [11]. Rawlins' advice has been taken seriously and new health-medical literature has emerged in response [12,13], including an appreciation of the value of analysing documents of both a textual [14] and non-textual kind. This article belongs to this growing school of thought regarding the significance of qualitative research and its role in supporting better health and medical initiatives. Our analysis sits within the framework of discourse analysis, using textual sources to analyse CoviSelf. The words and language of these texts constitute our data. Our interest in the socioeconomic context of new diagnostic medical devices was prompted by one of our engineering colleagues, who asked: 'what is the point of us inventing diagnostic tools unless we know how these fit into the daily life and needs of ordinary people?' [15].

## **2. RATs in Clinical Settings in India**

We are not aware of any published research assessing the self-administered CoviSelf kit, so we turn to what is known about RATs in clinical settings during India's COVID-19 pandemic. 'Clinical settings' are those in which trained professionals administer the test in either an institutional or private environment, including clinics, hospitals, schools, offices, prisons, military facilities etc. This experience speaks to the value of making self-administered RATs accessible to all Indians.

By April 2021, 49% of all the tests for COVID-19 in India used RATs, 'principally the SD Biosensor test (SD Biosensor Standard Q COVID 19 Antigen test)' [16]. This was in response to the second wave of COVID-19 in the early months of 2021, a wave defined by the Delta strain and the shift of the Indian pandemic from towns to villages. By contrast, during the first wave of COVID-19 between December 2019 and October 2020 [17], only about a third of the infections occurred in rural India, but that increased in 2021 to over 50 percent according to the State Bank of India [18]. By mid-May 2021, 69% of India's 748 districts (the lowest administrative level in the country) reported rates of infection above 10% [19]. The spread of COVID-19 generated interest in RATs as a way of saving the time taken for testing by RTPCR [20] and keeping track of the pandemic [21]. Although many extra RTPCR laboratories were set up throughout India during the first wave, the establishment and running costs of molecular biological testing facilities can be a limitation during times of a raging pandemic. In Bihar and Uttar Pradesh, states where rural healthcare facilities are especially limited, RATs played an even larger role by accounting for 87% and 59% of all the tests, respectively [16]. The cost of a RAT test is almost half or less than the cost of an RTPCR, an added incentive in the poorer states of northern India.

One review of RATs in India stated that they greatly 'helped us ... in detecting and diagnosing COVID-19 at its early stage and also by large scale screening of communities residing in hot-spot areas with high incidence of disease' [22]. The Cochrane Report confirmed the value of all the above factors when using RATs for widespread diagnostic purposes [23]. Another study spoke of the benefits of RATs involving 'triage and emergencies' needing priority diagnoses [24]. RATs have long been used for diagnosing other illnesses, such as pharyngitis caused by Streptococcus bacteria [25] and Bancroftian filariasis [26]. Despite this, before the pandemic, there was a tendency amongst some Indian medical professionals to see RATs as an unreliable diagnostic tool [27]. That view has changed.

Compared with the 'gold standard' of RTPCRs, RATs are less accurate, although they are highly valuable during a rapidly escalating health crisis. One recent study based on 2168 outpatient samples reported a false negative rate from RATs of 4.3% in the first wave of the pandemic and 8.1% in the second wave [17]. It is for this reason that negative test results from RATs still need to be confirmed by RTCPR tests. However, in desperate situations where the status of a person with a possible COVID-19 infection needs to be quickly established, the small margin of inaccuracy in RATs is not a prohibitive factor to their use. Detailed reports about ICMR-approved RATs are only available to state governments, but we found one published study of the first RAT named in the ICMR list of 6 January 2022 [28]. Gupta et al. evaluated the Standard Q rapid antigen detection test (produced by SD Biosensor, Inc., Gurugram) and showed that the test accuracy was 95.4% and the 'overall sensitivity and specificity of the test were 81.8 and 99.6 per cent' [23]. The authors concluded that RATs with these kinds of properties could help alleviate the pressure on emergency departments in hospitals by allowing 'patients showing positive results [to be]... immediately triaged', thereby reducing the risks of COVID-19 infection spreading to other patients and staff in waiting rooms. ICMR then issued an advisory to Indian hospitals saying that the test should be 'adopted in [their] diagnostic algorithms'. Unfortunately, the use of RATs in clinical settings is seriously constrained in rural India, where medical and health professionals are in short supply [29]. In that scenario, the question is whether a self-administered COVID-19 RAT might make the benefits of new diagnostic technologies more accessible to villagers. The need for rapid diagnostic assessments of

COVID-19 infections in rural areas is evident from the situation that emerged from the start of pandemic in India.

## *The Urgent Need for Self-Administered RATs in Villages*

The negative effect of the pandemic has arrived on top of a long-term agrarian crisis in India. Agriculture has not been profitable for decades, and many farmers have been driven to suicide [30]. Poor, small, semi-literate farming families (including those who lease or share crop land, marginal farmers and labourers) have chosen to migrate to the cities for casual employment. When the Indian government declared an immediate national lockdown on 24 March 2020 due to the pandemic, millions of migrant workers found themselves walking home over hundreds of kilometres along India's railway lines, or catching buses, trains or any other possible conveyance. Of the estimated 139 million migrant workers, perhaps a fifth formed a wave of reverse migration as they returned to their natal village homes in search of sustenance and emotional security. Plans to assist them during the pandemic were constrained because the central government had no idea of the scale of the problem due to a lack of statistics. What we do know is that about 6.3 million migrant workers were taken to state transport hubs closest to their villages by special trains between May and August 2020 [31]. There, they were left to walk to their homes, which were often hundreds of kilometres distant. Perhaps three to four times that number travelled home by other means. Back in their villages, they often returned to subsistence farming.

The official data regarding the mortality and morbidity rates of COVID-19 in India are no better than the data on migrant labour, and government estimates have been criticised for their lack of realism. Alternative estimates suggest that the total cumulative deaths in India have been between 2.5 and 7 times higher than the reported number of less than half a million [32–36]. Estimates of the total number of rural deaths *from* COVID-19 are significantly lower than the real figures because India's percentage of medically certified causes of death (MCCD)—based on the signing of the 'Medical Certificate of Cause of Death' by a medical professional—is about 20% [37] of all the deaths at the national level. This low level reflects the fact that the 'Medical Certificate of Cause of Death' is typically signed in urban hospitals and clinics where the majority of Indian doctors are based. What people die of in rural areas is largely unknown unless special surveys are conducted. Thus, we can assume that the rural deaths from COVID-19 accounted for much more than the reported 55% of the total deaths between 2020 and 2022.

The impact of COVID-19 s devastation was revealed on 5 May 2021, showing that, in the preceding week, COVID-19 had become India's leading cause of death, outstripping ischemic heart disease (the previous leading cause of mortality) by more than 2.5 times [38]. In summary, Indian villages remain in desperate need of medical attention during the ongoing COVID-19 pandemic. Given the inadequacies of rural health provision (see Section 4.3.2), self-administered RATs hold out the promise of giving villagers timely and useful information about how to protect their health. Our earlier study of the positive attitudes of villagers to point-of-care blood testing at their doorsteps suggests that what holds back the health of rural dwellers is not their opposition to innovative allopathic technologies but their poverty and lack of access to adequate healthcare (see Sections 4.3.2 and 4.3.3).

## **3. Materials and Methods**

This article employs qualitative data and methods to evaluate the self-administered Indian rapid antigen test CoviSelf.

## *3.1. Qualitative Framework*

Content analysis and discourse analysis are two of many qualitative research methods that can be applied. The former assesses the frequency with which certain words, phrases or metaphors appear in a particular documentary text. This approach would not suit our purposes because we wish to understand the structure of the main document rather than focus on the frequency of particular words. Hence, we employ discourse analysis, where discourse is defined as 'contextually sensitive written and spoken language produced as part of the interaction between speakers and hearers and writers and readers' [39]. In our analysis, the speaker or writer is the producer of CoviSelf and the hearers or readers are the users of it.

## *3.2. Our Main Data Source*

The main document we have analysed is the text of the printed 'patient information leaflet' that is part of the six components of the CoviSelf kit (Figure 1). Throughout this article, we refer to this document as the 'instruction leaflet' or the 'leaflet' [40]. Mylab calls this leaflet the 'user manual'. We downloaded the English language version of the leaflet from the Mylab website [41]. We treat the leaflet as an agent reflecting the urgent diagnostic needs thrown up by the COVID-19 pandemic in India. As the first officially approved, self-administered, Indian made rapid antigen test for COVID-19, the text of this document has its own 'potency as well as capacity' [42]. We argue that the information embedded in the leaflet does not constitute 'inert data' but rather that the data are directed to particular social actors [42] who, in turn, interact with the data when they use the kit. That interaction involves the user of the kit reading the data (in the form of instructions) in the leaflet, interpreting it, interpreting the results of the COVID-19 test and interacting with any external agents who may be involved in the testing process—for example, any person helping the user to administer the test or the Government of India database that seeks to capture the user's test results.

**Figure 1.** The components of the self-administered CoviSelf RAT kit. (Showing in clockwise order from the bottom left-hand side: the plastic bag for disposing of used items, the CoviSelf package, the instruction leaflet and, in the centre, the sterile nasal swab, the test card and the pre-filled extraction tube.) Source: Ms Anusha Kesarkar-Gavankar, photograph of 18 November 2021.

Our discourse analysis involved a close reading of the language of the instruction leaflet by checking for the three things required to effectively communicate the instructions to the consumer: first, we interrogated the text to see whether the assumptions it made about the user's handling of the testing device were reasonable; second, we asked whether the language of the leaflet was clear, consistent and non-technical; and, third, we evaluated those sections of the leaflet that instruct the user about the IT components of the kit. From the start of our analysis, we were struck by the two main characteristics of the leaflet: it addresses both the test component and a special IT feature.

The leaflet was defined by us as the main document because it ends up in the hands of those who purchase the CoviSelf RAT kit. For this reason, we read the text multiple times to cross-check our findings, test out alternative interpretations of the data and resolve any inconsistencies that emerged. In reporting the 'Results' of our analysis below, we arranged our findings according to the sequence involved in using the self-administered CoviSelf RAT in order to place ourselves in the situation faced by a user. In evaluating the leaflet, we engaged in the 'core activity' of discourse analysis by thinking and categorising the '...actions, intentions, characters, events....' [40] as revealed by this document.

## *3.3. Contextualising Our Main Data Source*

An important part of our research method was to acknowledge that the meaning of the leaflet is not self-evident. The reader of the leaflet is not a passive, predictable sponge that merely absorbs the prefigured information provided in the printed text. Rather, particular readers construct the meaning of the instruction leaflet within specific cultural and socioeconomic contexts that vary from user to user. In other words, there is a process of negotiating taking place between the text and context, and it is this that enables the reader to construct meaning. Early in our evaluation, discourse analysis compelled us to ask where the CoviSelf diagnostic device sat within the broader framework of India's IT revolution. For instance, which Indian consumers had smart phones capable of downloading the necessary app referred to in the leaflet? Mylab's press release of 10 May 2021 stressed the benefits to Indian citizens of the kit's features because they could now buy the diagnostic device 'without a prescription from local pharmacies and online channel partners', and, in using the kit, the consumer would promote the traceability of the pandemic through the company's 'AI-powered mobile App' [3]. Our 'Results' below question the assumption that lies behind Mylab's statement. We ask, does 'every Indian and any citizen' have access to the information technology that Mylab has taken for granted?

## *3.4. Our Second and Third Data Sources*

These questions led us to the second data source—a group of documents produced by the Indian media, professional associations and the Supreme Court of India about India's digital divide. The digital divide stressed the disadvantages suffered by rural residents in accessing electronic information.

Following the logic of discourse analysis, we then widened the context for understanding the CoviSelf leaflet even further by using a third data source, namely our own 'expert opinion', as documented in peer-reviewed, published research about living conditions in Indian villages. Fieldwork in central India before the pandemic showed us that villagers were receptive to point-of-care blood testing at their doorsteps [43]. We assumed, therefore, that there might be positive responses to the idea of using the self-administered CoviSelf RAT kit because the demand for modern medical interventions is driven by the desire to prolong longevity. This may well be a universal motivation, but our findings remind us that the specific cultural context, in Prior's words, renders the 'unremarkable, remarkable' [42]. With this third data source, we explored the interplay between the digital divide in rural India and the multiple constraints faced by villagers in accessing timely health and medical care.

## *3.5. The Limitations of This Study*

With the ongoing COVID-19 pandemic, it has not been possible to undertake fieldwork because the desperate circumstances facing rural residents prevents them from participating in time-consuming, in-depth interviews. This is the main limitation of our present study. On the other hand, a discourse analysis of the three data sources (specified above) allowed us to refine our research questions and hypotheses for future research on village responses to the pandemic. In-depth interviews in rural areas during the next two years will constitute

the fourth and final data source for an expanded study of the current paper. The locus of this extended study will build on our existing work in Wardha District (Maharashtra), the site of our research on point-of-care blood testing technologies. That earlier period of fieldwork was informed by our prior analysis of classical and modern Hindu texts that suggested that there were unlikely to be any cultural objections to modern diagnostic blood technologies. Our hypothesis for further fieldwork (in 2022–2023) on self-administered RATs in rural India questions the value of CoviSelf in villages. As a result of the present paper, we have come to the view that self-administered RATs for detecting COVID-19 infection would be more suitable to villagers if they did not include an IT component and if they were free. This hypothesis is not based on any questions about cultural resistance to the innovative CoviSelf device but rather doubts about its current practicality and cost. The urgent need for self-administered RATs in rural India suggests that investing in the redesign of this diagnostic device is well worthwhile.

## **4. Results**

Our results are discussed under three main headings. Firstly, we report the findings from our discourse analysis of the main document, namely the kit's printed instruction leaflet. In the second part, we explain the findings from our analysis of the data regarding India's digital divide and its impact on rural India. In Section 3, we use our own data from previous studies of Indian villages. All three parts locate the CoviSelf kit within the wider socioeconomic context of Indian villages in conformity with discourse analysis, which seeks to understand how particular readers construct the meaning of different kinds of documents.

## *4.1. The CoviSelf Instruction Leaflet*

Our analysis of the language of the leaflet focuses on four issues affecting the practicality of this self-administered RAT, but we begin our discourse analysis by considering the structure of that document. After that, the next two findings deal with the processes by which a test result is obtained, and the last two findings report on different aspects of the IT components of the kit. The data take the form of the leaflet's instructions to the reader. Where appropriate, short comments are made about the practicality of the text in terms of its limitations and benefits.

## 4.1.1. The Structure of the Instruction Leaflet

The layout of the leaflet is an important consideration in our discourse analysis because it tells us how the information provided to the user has been structured by Mylab, i.e., the order of priority given to various points. The leaflet takes the form of a folded, double-sided sheet of printed paper measuring 21 cm × 57 cm. One side is in English and the reverse side in Hindi. The leaflet is also available in eight other Indian languages representing different linguistic regions. The online version of each leaflet is identical in information and size to the copy included in the CoviSelf kit. The text is accompanied by diagrams illustrating all the instructions and how to read the positive, negative and invalid results. The only exceptions are the text in column one (both top and bottom panels, see below) and the bottom part of columns seven and eight, stating the FAQs (frequently asked questions), the limitations of the procedures, Mylab's contact details and a QR code that can be scanned to access a video demonstration about how to use the kit.

Both sides of the leaflet are divided into eight columns, each with two panels (top and bottom). When folded, the leaflet measures about 10 cm × 7 cm. Column one, top panel, is taken up by the front cover of the leaflet, and the panel beneath it explains the kit's intended use, kit storage and stability and the principles of the test; this last section explains some of the science behind the test device. Column two, top panel, is headed 'Kit Contents' and names the six components that make up the kit. The panel beneath that explains the 'test preparation', which includes washing and thoroughly drying the hands. The explanatory video that can be downloaded by some users says that a table top should

be sanitised so that the contents of the kit can be laid out [44]. No such instruction appears in the leaflet.

Between the top and bottom panels is a central box that states: 'Download the Mylab CoviSelf App from Play Store/App Store'. Column three, top panel, instructs the user to open the CoviSelf App and activate it; the bottom panel has instructions for preparing the pre-filled extraction tube. Column four is named 'Step 1' and instructs the user how to collect the nasal sample. Column five, designated 'Step 2', has instructions for placing the swab into the pre-filled extraction tube. 'Step 3' appears at the top of column six and tells the user how and where to place the sample on the test card. 'Step 4', beneath 'Step 3', asks the user to take a photo of the test card and '*wait for the App to analyse and display your Covid-19 test results*' (authors' italics). Under 'Step 4' appear instructions for disposing of the waste once the test has been completed.

Columns seven and eight, the leaflet's last two columns, have two sections: the top explains the appearance of the test card when positive, negative or invalid test results appear. The bottom section provides additional information to the user: three FAQs are listed, five limitations to the procedure are stated and there is a QR code that can be scanned if the user wants to watch a video demonstration of how to use the kit. The bottom strip across both columns gives the name of the manufacturer and contact details.

Our analysis of the structure of the leaflet suggests that the order in which the instructions are arranged encourages the reader to use the IT component and to depend on it for 'analysing' the test results. Instructions about using the IT features are given at the start and end of the leaflet. The first instruction appears prominently in the middle of column two and asks the user to 'Download the Mylab CoviSelf App'. This is followed by instructions in column three saying '*Open the Mylab CoviSelf App and fill in the credentials*'. After that, the user is told to scan the QR code on the test card. More significantly, 'Step 4' is presented to the user as the last step in the sequence of actions needed to complete the testing procedure. It asks the user to take a photograph of the test card and '*wait for the App to analyse and display your Covid-19 test results*'. However, the test card has already completed both of these functions: it has analysed the nasal sample thanks to the bioactive ingredients used in the test process and it has displayed the test result. In the instruction leaflet, there is a section that explains how the test card tells users whether they have a positive, negative or invalid test result. However, these explanations appear at the end of the leaflet, just before the FAQs section in columns seven and eight. A more logical order would have been for information about the test results to be labelled 'Step 4'. The function of uploading a photograph of the test card to the app could then have been labelled as an optional 'Step 5'. This new 'Step 5' would also need to give the user an explanation for why the photograph of the test result should be uploaded to the app. In other words, the user should be given the choice of taking or not taking a photograph of the test result that has appeared on the test card. Instead, the instructions and current layout of the leaflet create the misleading impression that the IT functions of the kit are an integral part of the testing process. The true purposes of the IT functions are analysed below (Section 4.1.5). We also address the risks that these pose.

4.1.2. The Ease of Using the Kit (the Relevant Headings in the Instruction Leaflet Are Steps 1, 2 and 3)

After scanning the QR code on the test card, the user is told to remove the pre-filled extraction tube from its packaging, hold it erect and tap the tube so that the 'extraction buffer settles at the bottom of the tube'. The user is warned to be careful not to spill the contents of the pre-filled extraction tube—but no reason is given for this, so the user has no way of knowing the critical nature of this instruction. The fluid in the tube contains the reagent needed for testing the nasal sample. 'Step 1' then follows: it involves removing the swab from its packet by holding the tail end opposite the swab head. The swab head goes into the nostrils and so should not be touched because that could risk contamination. It is rolled five times in each nostril to collect cell samples.

Removing the swab from its package with one hand may be difficult if the user is already holding the pre-filled extraction tube in the other hand. Thus, there is a risk of spillage and contamination if the tube or swab are put down on a non-sterile surface. A second person could help by holding the pre-filled extraction tube, but, if that person has not washed and dried their hands thoroughly, there is another risk of contamination. In any case, instructions about this alternative approach are not given in the leaflet; it is assumed that the user is nimble.

'Step 2' comes next. On removing the swab from the nostrils, the swab head is inserted into the pre-filled extraction tube. The bottom of the tube needs to be pinched while the swab stick is swirled ten times to ensure that the nasal sample mixes properly with the pre-filled liquid. After that, the swab stick needs to be broken 'at its breakpoint', but this point is hard to see. The lower half of the swab, with the swab head, remains inside the tube. The top broken off part is placed into the disposable bag to be thrown away at the end of the testing process.

Juggling these different tasks might not be so hard for an experienced user, but firsttime users risk spilling the tube's contents, contaminating the swab head or not collecting sufficient material from the nostrils. Nevertheless, supposing all has gone smoothly, the swab head is now inside the pre-filled extraction tube, which is then sealed with the attached cap. The test can now move to 'Step 3'. The user is instructed to press the bottom part of the tube and place two full drops of the liquid into the sample well at the far end of the test card near the letter T (in Figure 2, the well is located on the far right of the image). Within 10 to 15 min, a test result should appear in the test window.

**Figure 2.** The CoviSelf test card. (From the far left: the QR code, the control point C, the test point T and the sample well for receiving the nasal mixture). Source: Ms Anusha Kesarkar-Gavankar, photograph of 18 November 2021.

While waiting for the test results, the user can place all the discarded items into the disposable bag and 'disinfect all surfaces that the specimen may have touched' in addition to washing their hands after throwing the bag into household waste (bottom of column six). Soap and water are the best disinfectants against the coronavirus, and soap is readily available in Indian villages. However, what villagers might not know is that effective cleaning requires the user to scrub their hands 'for at least 20 s' [45].

4.1.3. Reading the Test Result and the Clarity of the Leaflet's Language (the Relevant Headings in the Instruction Leaflet Are Positive Result, Negative Result and Invalid)

The user can read the results on the test card, which has two letters in the centre beneath the test window: C and T. C is the 'control' point and T is the 'detection' point. If the test result is *positive*, two lines appear in the test window located in the centre of the card—one line next to the letter C and the second line next to T. The T line can be any shade of pink or purple, but all shades, no matter how light, indicate a positive test result. If only one line appears next to the letter C and there is no line next to the letter T, it means that the test result is *negative* for the virus. If no line appears in the test window next to the letter C, the result is invalid. The text should, however, clearly state that the test has failed.

The language in which the above information is expressed in the leaflet is complex and technical. For example, in the case of a positive test result, the leaflet reads: 'If both the quality control line 'C' and the detection line appear, novel coronavirus antigen has been detected and the result is positive for antigen'. Antigen is not a well understood term in most countries, and neither is the notion of 'detection line' clear. However, the labels next to the diagrams illustrating the appearance of a positive and negative result are better: the 'detection line' is described as the 'T test line'.

If a negative test result appears, the 'symptomatic' user is advised to immediately proceed with an RTPCR test because 'RAT[s] are likely to miss [a] few positive cases presenting with a low viral load'. We have inserted the word '[a]' into this quotation because, without it, the actual sentence says the opposite of what is intended. This problem in the English language version of the leaflet does not, however, occur in the Hindi version.

The leaflet also uses inconsistent terminology that can confuse users. For example, the test card is also called the test device or cassette, the pre-filled extraction tube is also called the pre-filled buffer tube and the liquid in the tube is called an extraction buffer and the antigen buffer.

4.1.4. How to Respond to the Test Results (the Relevant Headings in the Instruction Leaflet Are Positive Result and Negative Result)

From the above account, it is obvious that the CoviSelf kit is a stand-alone device that has the capacity to tell the user whether they are or are not infected with the COVID-19 virus. However, the kit contains another component—an IT factor, which is problematic beyond our analysis of the misleading importance given to the app in the instruction leaflet (see Section 4.1.1).

The leaflet states that all positive results are 'true positives' and the user is advised to follow 'home isolation and care as per the ICMR and Ministry of Health and Family Welfare protocol'. A website address with further instructions is provided for the user. All users lacking access to the internet will be excluded from further information—and we are speaking of millions of Indian citizens.

The advice is clearer in the case of a negative result. Here, the leaflet tells a user who has symptoms but receives a negative result to isolate at home and get an RTPCR test as soon as possible. Despite the misleading English language text explaining why more testing is required (see Section 4.1.3), users are familiar with the acronym RTPCR because, during the pandemic, such tests have become a routine procedure for hospital admissions. Nevertheless, users may not see the point of having an RTPCR test if their CoviSelf test results are negative. There is no compelling explanation in the leaflet to justify the expense and inconvenience of visiting a clinic or hospital to get an RTPCR test. The leaflet only states that a low viral load could mean that some positive cases are missed, but the words 'low viral load' are not automatically meaningful to most readers, who will not know about the science of virus testing.

More importantly, the user is not encouraged to take *another self-administered test* if the first result is negative. If the viral load is low at the time of the first test, a second test could pick up a true infection if the viral load has increased. There has now been considerable research and experience to show that, in the words of Richard Hatchett, 'the antigen tests are less sensitive if you give just one. But if you can do it in a sequential way, they become cumulatively as sensitive as a PCR' [46].

If the user is a villager, the advice to have a follow-up RTPCR test involves a timeconsuming and expensive trip to an urban medical institution (see Section 4.3.2 below). 'Isolating at home' in the case of symptomatic individuals is also advice that is virtually impossible to follow as, in most villages in India, a whole family lives in a single room. In such cases, isolation at home actually increases the risk of all family members becoming infected. More fundamentally, individuals interpret the word 'symptomatic' very differently depending on their tolerance for risk. In India, recognizing the symptoms for COVID-19 is also difficult because the virus mimics the signs for many other illnesses—for example, other respiratory conditions accounted for a total of about 15% of Indian deaths between 2010 and 2013 [47] and much higher levels of morbidity.

4.1.5. The Test Results and the Objectives of the ICMR (the Relevant Section in the Instruction Leaflet Is Step 4)

Instructions about the IT component of the kit are stated at the beginning and end of the leaflet. Prior to conducting the test, the user is asked to download the Mylab CoviSelf app from either Google Play or the App Store; a mobile phone message is then received asking the user to enter their personal details and scan the unique QR code printed on one end of the test card (on the left hand side of the image in Figure 2). The personal details are: name, age, gender, address, pin code, mobile and Indian ID number (or *Aadhar* card number). Except for the Indian ID number, which is an 'optional' detail, all the information is stated as being 'mandatory as per ICMR guidelines'. Once the user has completed these tasks, they are ready to conduct the test. The result should appear within 15 min. The user is then asked to take a photograph of the test result and '*Wait for the App to analyse and display your COVID-19 test results*'.

There are two problems with these instructions. First, making the personal details 'mandatory' sounds ominous even if ICMR has no way of enforcing this. In Indian villages, such statements are taken seriously (see Section 5 'Discussion' about '*sarkar*'). Secondly, it is misleading to state that the test result for COVID-19 infection is being analysed by the Mylab app. As already explained above, the result of the test appears within 15 min on the test card. What the leaflet should be saying is that, once the test result has been photographed, it is uploaded to Mylab records and then forwarded to the ICMR database, which forms part of the records of the Government of India. Equally important, there is no provision in the leaflet for the user to give their consent to sharing their personal details or test results with the government.

The role of ICMR is only explained on the Mylab CoviSelf website, where there is a section headed FAQs. From this, we can learn about the destination of the uploaded test result in response to the following question: 'What are the benefits of reporting the result to ICMR?' The following answer is given [48]:

ICMR is the regulatory body in India which is responsible for the curb of COVID-19 pandemic along with other important regulatory bodies. Reporting the results helps the body and authorities curb the spread of the disease. It thus becomes our moral obligation to help ICMR by reporting our test result data.

It is significant that the above FAQ and answer do not appear in the user leaflet, although the FAQ section in that leaflet takes up a total of 20 lines, suggesting that there was enough space to include the five lines about the role of the ICMR database. Helping the government to collect better records of COVID-19 infection is a worthy objective, but it could end up being counterproductive to do so without the prior agreement of the user (see Section 5 'Discussion'). Moreover, it is just as important to protect the right of Indian citizens to keep their personal details private and their medical records confidential [49].

From the user's perspective, is there any benefit to activating the IT component of CoviSelf? The analytical methods of discourse analysis tell us that the answer to this question depends on who the user is.

After the user sends a photograph of the test result to the Mylab CoviSelf app (and ultimately to the ICMR) via their mobile phone, they receive a notification of their COVID-19 status. This can be beneficial for several reasons. First, it might reassure those users who are uncertain about their reading of the results on their test card. The response from the Mylab app gives them a statement about their test result, and this validation enables them to act accordingly. Second, many middle-class Indians have used this statement as evidence of their COVID-19 negative status when boarding domestic flights and entering hotels. Such certification has enabled some professionals and holiday seekers to be mobile inside India [50]. One hopes that such travellers have been cautious and only travelled when they were asymptomatic. However, as already noted, everyone's tolerance for, and interpretation of, 'symptoms' differs.

These benefits are counterbalanced by the multiple problems that arise from trying to use the IT components of CoviSelf. Many urban users have complained about the difficulties (see Section 6 'Conclusions'). It is regrettable that the instruction leaflet conflates the two components of CoviSelf, i.e., the test and the IT functions, because this gives users the impression that the kit will not work without activating the IT features. In rural India, the inadequate provision of healthcare services suggests that the potential value of a self-administered COVID-19 RAT is high, but the IT component compromises the device's usefulness for the reasons we explain in the next two sections.

## *4.2. India's Digital Divide*

India's digital divide is part of a global problem and influences the value of the CoviSelf kit because it contains an IT feature. In 2002, the Secretary General of the UN spoke of the digital divide as the widening gap between the 'haves' and 'have nots' in accessing the new information and communications technologies and the dangers this posed by excluding the world's poor from the 'emerging knowledge-based global economy' [51]. The key technologies for such access were the internet and mobile phones. Two years earlier, UN members agreed that one of the Millennial Goals was to 'ensure that the benefits of new technologies, especially information and communication technologies, are available to all'. The data we have used to provide the general context for understanding the implications of this digital divide for CoviSelf come from the following sources: the Indian press, a 2021 report by the Internet and Mobile Indian Association (hereafter IAMAI) and a court case by the Software Freedom Law Centre before the Supreme Court of India in 2021.

## 4.2.1. The Indian Press

The phrase 'Indian press reports about India's digital divide' revealed over 46 million hits on the Google search engine. A report in 2019 from the *Economic Times* stated that India had an estimated 450 million smart phone users and about 550 million users of 'feature phones' [52]. A feature phone is defined by Kantar IMRB/MMA as 'A mobile phone that incorporates a fixed set of functions beyond voice calling and text messaging such as limited web browsing and e-mail, ability to play music *but generally cannot download apps from an online market place*' [authors' emphasis] [53]. Whilst most mobile feature phones allow users to access the internet for entertainment, they typically lack the capacity to support complex internet apps, such as the one used by CoviSelf. This places the innovative CoviSelf device beyond the majority of Indians despite the hopes that Mylab has of reaching all citizens.

## 4.2.2. The Internet and Mobile Association of India (IAMAI) Report of 2021

A more fundamental problem is the unreliability of telecommunication signals in many parts of India, especially rural areas [54]. The 2021 IAMAI report on internet usage in urban and rural India shows unsurprisingly that states with relatively good telecommunications infrastructure have a higher percentage of active internet users (hereafter AIUs) amongst their population (e.g., 61% in Maharashtra) than states lacking such facilities (e.g., 24% in Bihar) (Table 1). This reflects the level of urbanisation: 45% of Maharashtra's population live in towns and cities compared with 11% in Bihar [55]. For the whole of India, AIUs represent about 31% of the rural population compared with 67% percent of urban residents (Table 1). The percentage of people who have what is often called 'connectivity' in India may be less than this as many AIUs might have more than one phone and more than one internet account.

**Table 1.** Internet usage in urban and rural India in 2020 (based on active internet users (AIUs)).


Source: Collated from Kantar, *Internet Adoption in India: ICUBE 2020*, Internet and Mobile Association of India/Kantar: Delhi, India, June 2021, (https://images.assettype.com/afaqs/2021-06/b9a3220f-ae2f-43db-a0b4-3 6a372b243c4/KANTAR\_ICUBE\_2020\_Report\_C1.pdf, accessed on 20 February 2022).

## 4.2.3. Civil Society and the Supreme Court of India

The exclusion of the rural poor from internet access is well known within Indian elite circles, yet, despite this, the Government of India created the CoWIN app in early 2021 to make it easier for Indian citizens to get appointments for vaccinations against COVID-19. The idea behind this innovative app was to stop citizens standing in long queues for vaccinations—thus saving time, reducing the risks of cross-infection and taking the pressure off clinics, hospitals and vaccination centres. Unfortunately, when the new system of appointments was announced on 18 April 2021, it also became mandatory for all Indians aged from 18 to 44 to make online appointments via the CoWIN app. The Software Freedom Law Centre (hereafter SFLC) in New Delhi wrote to the Indian Ministry of Health objecting to the mandate and, after being ignored, raised a Sou Moto case in the Supreme Court of India against the ministry [56]. The SFLC argued that the CoWIN app [57] failed to address the 'digital exclusion' of Indians who lacked appropriate mobile phones and internet access and it also failed to protect the privacy of citizens [58]. Justice Chandrachud (Supreme Court) described the government's assumptions behind the CoWIN portal as 'far-fetched' and 'exclusionary of the rural areas' [59].

The objections against the CoWIN app (before the mandatory requirements were removed) apply equally to the IT component of CoviSelf. However, there has been no public outcry against CoviSelf because buying the device in India is a matter of personal choice rather than something mandated by the state. Nevertheless, this should not blind us to the fact that the digital divide deprives millions of citizens from the benefits of this self-administered RAT. The government's failure to address the needs of villagers in the provision of self-administered RATs is inconsistent with its strong campaign to improve rural hygiene since 2014 through the Swachh Bharat Mission to end, for example, open defecation and the risk of faecal infection [47]. This contradiction in the government's rural development programs cannot be addressed in this paper because the topic is too large and complex, but, in the following section, we consider the practicality of CoviSelf for Indian villagers.

## *4.3. The Conditions of Life in Rural India*

In extending the context in which to understand the benefits and limitations of CoviSelf, we have drawn on our own research published in peer-reviewed international journals and books. Vicziany has published on the poverty and marginalisation of *Dalits* (former 'untouchables') [60], the family planning program [61,62], food security [63–65], Koli villages in Mumbai [66] and rural health [29,43,47], including work on the potential of point-of-care blood testing in villages [29,43]. Hardikar has reported extensively on questions of rural poverty, co-authored work with Vicziany on poor farmers, agriculture and health [29,43] and has published two books on India's farming crisis [30,67].

## 4.3.1. Village Poverty and Employment

The bulk of our research has dealt with poor, semi-literate people in marginalised and socio-economically disadvantaged households. Our most recent research in Wardha District (Maharashtra) involved 36 in-depth interviews with farming groups defined by various criteria: a high proportion of families living below the poverty line, households with very little irrigated land, poor farmers, landless villagers and tribal or *Dalit* families [43]. Our findings showed that poor rural households are preoccupied by three major concerns. Their first priority is to secure a source of income through any kind of employment- even if they farm; supplementary sources of income are important, in particular through shortor long-term migration to urban centres. Those who remain in villages seek to diversify their income locally—for example, by serving as government agents for the public food distribution system, working as linesmen for electricity companies or providing local taxi services.

Our understanding of the drivers of poverty in rural India and the constraints it imposes on the capacity of villagers to avail themselves of the benefits of modern technologies was confirmed by the Supreme Court's judgement in the CoWIN case (see Section 4.2.3). The judges noted that the minimum internet tariff plan would equal '4–5% of the month's income' of urban and rural people living below the poverty line [68]. Accessing internet data is, therefore, something that they cannot afford given that the poor are also heavily indebted. The burden of poverty extends into all aspects of life, including the lack of consumer power to buy the CoviSelf kit that retails for about Rs. 250. Given that the daily wage for a farm labourer at the top end of the range of agricultural jobs (i.e., ploughing and tilling) is about Rs. 365 [69], the cost of CoviSelf is equal to 68% of the daily wage that sustains a whole family. For the unemployed and those living below the poverty line of Rs 1316 and Rs 896 *a month* in urban and rural India, respectively [68], the expense of the CoviSelf kit is unthinkable. Moreover, the benefits of all self-administered RATs are greater if used more than once over a number of days to check the onset and end of infection. Such costs are beyond the means of India's poor whether they are marginal farmers, labourers, the under-employed or the unemployed.

Our research on rural poverty is supported by other quantitative data showing that 27.9% of India's population is suffering from multidimensional poverty [70], defined as multiple and overlapping conditions of deprivation. The MPI (multidimensional poverty index) goes beyond previous attempts to measure poverty using simple headcounts and monetary estimates of the poverty line. As a result, the MPI replaced the Human Poverty Index (HPI) of the United Nations Development Programme in 2010. The data show that interstate variations in poverty in India are stark and support our previous reference to the large regional disparities between, for example, Maharashtra and Bihar. The proportion of the population in these two states that is multidimensionally poor is 15% and 52%, respectively [71]. Poverty weighs especially heavily on disadvantaged social groups throughout India (such as the *Dalits* and tribal people), with five out of six Indians belonging to these communities suffering from multidimensional poverty [72]. Rural economic insecurity is the overall context in which we need to evaluate the practicality of CoviSelf, the affordability of mobile phones and internet access and the future of self-administered RATs for villagers.

## 4.3.2. Village Health and Diagnostic Medical Interventions

The second priority of Indian villagers is their health. India's medical system is a plural one, ranging from trained allopaths (i.e., doctors trained in the 'western' medical system) and Ayurdevic-Unani practitioners to herbalists, soothsayers, spirit men, midwives, gurus, spiritual mediums and 'quacks'. The Maharashtra Medical Association, for example, has been engaged in a campaign to marginalise all but allopaths, yet the plurality remains because there are insufficient allopaths willing to serve in rural areas. Even in states that have seemingly adequate rural health infrastructure (such in Maharashtra), the human resource constraints are serious. The result is that villagers bypass government rural medical institutions and travel considerable distances to clinics and hospitals in or near towns [43]. While this is costly, rural residents have the satisfaction of getting a more timely and reliable diagnosis of their condition, a prescribed treatment regime and admission to hospital in urgent cases.

The healthcare sector is now dominated by private services, with many hospitals being part of larger systems of education involving private medical colleges. Providing medical training and degrees has become a lucrative business in India and attracted many entrepreneurs. The appearance of the private health sector has resulted in villagers experiencing a medical revolution, with entrepreneurs working to make their hospitals more accessible and affordable to the poor by offering 'membership cards' and using their political links to help residents in need of attention. In return, these entrepreneurs and politicians are rewarded when their beneficiaries vote for them in elections [43].

Yet, despite this, the shortage of medical and health professionals in rural areas has not been alleviated. The private entrepreneurs that have developed western based medical institutions are urban based. In the government's rural health infrastructure there are large human resource shortfalls and vacancies relative to requirements [73]. At the village level in India, health care is left to what can best be described as a pre-modern private sector. One rare study reported that the village level private sector accounted for 68% of local healthcare and that 75% of villages had at least one health care provider. However, of the 3373 providers surveyed, only 8% were allopaths with an MBBS qualification and 24% were AYUSH providers with degrees in alternative medicine, namely in Ayurveda, Yoga, Unani, Siddhi and Homeopathy. The remaining 68% lacked any formal training [74]. In other words, the size of the private sector reflects the poor relationship that exists between villagers and government funded institutions. It does not mean that the private sector provides better health and medical services, even if these are more accessible.

For all the above reasons, villagers with life-threatening health problems go straight to the nearest hospitals, even though these are expensive. 'Free' treatment in government institutions, subsidised health care in the private sector and the support of an emerging insurance system have failed to reduce the out-of-pocket (hereafter OoP) expenses for inpatients and outpatients. Our study of the potential for point-of-care blood testing at the village level showed that there are many potential rural innovators because OoP costs are too high. Of the 36 villagers we interviewed between 2017 and 2018, 61% were receptive to the idea of introducing simple blood tests into their villages [43]—using bioactive paper, such tests would require only a few drops of blood and allow villagers to avoid the many costs incurred by travelling to town-based institutions. More generally, we discovered that medical innovations do not come as surprises to Indian villagers; their receptivity to blood tests at their doorsteps reflected their experience with modern medical practices, ranging from complex procedures, such as liver transplants and transfusions for accidents through to regular blood testing and treatment for genetic blood diseases [43].

Rural residents place a high value on their health, and, because of this, they are willing to spend large amounts of money on medical diagnoses and treatments to address their illnesses and increase their longevity. Many rural households go into debt to cover medical expenses. There is no evidence in the available literature or our fieldwork to suggest hesitancy arising from their distrust of modern medicine; rather, they may be distrustful of the government institutions charged with the delivery of rural healthcare.

## 4.3.3. Village Religion and Modern Medicine

The third preoccupation of villagers is to ensure that their family and local religious traditions are respected, including the worship of goddesses. The COVID-19 pandemic has seen the proliferation of Corona goddesses. This could easily be misunderstood by non-Indian observers thinking that here is a contemporary example of Indian fatalism and attitudes opposed to allopathic medicine. However, our research showed that goddess worship, including blood sacrifices to appease village and household deities, sat comfortably alongside the demand for modern, scientific health interventions. One of the villagers that we met explained that her husband had died of alcoholism despite the fact that a goat had been sacrificed to the family deity on his birth. We asked her why the mother goddess, Firasti-Aai, had not provided her husband with lifelong protection against ill fortune. She exonerated the deity and blamed her husband:

He had good health but he brought it upon himself by drinking too much. How is our deity responsible for that? [43].

Powerful religious beliefs, in other words, do not cancel out personal responsibility. We found no evidence of any clash between local religious traditions and innovative medical practices.

The prudent course of action for believers wishing to prolong their lives is to propitiate the goddesses and simultaneously address their ill-health by resorting to appropriate medical solutions. Since independence in 1947, Indian governments have also promoted widespread immunisation for infectious diseases, and this has contributed to a decline in death rates. However, despite great demand in rural India, the present pandemic has been characterised by shortages of vaccines, oxygen, hospital beds, masks and drugs, such as remdesivir. There has not been any cultural resistance to these measures. For most of 2020 and 2021, vaccinations against COVID-19 in rural India lagged seriously behind those in urban areas, demonstrating the 'urban bias' that Indian development policies suffer from. By mid-May 2021, only about 15% of rural residents had received at least one dose of vaccine [75], partly because urban–rural disparities cause India's cold chain to be unevenly distributed [76]. By early 2022, however, the vaccination gap had closed thanks to a new focus on villages, although urban–rural differences remain, with 79% of eligible urban residents fully vaccinated compared with 69% of rural dwellers in late January 2022 [77].

## **5. Discussion**

The unique status of the India-made, self-administered CoviSelf RAT kit is based on official approval by ICMR. In the context of a pandemic, that places a special responsibility on the government to ensure that this diagnostic device can benefit all Indian citizens. The IT component of the kit fails this basic test because millions of people in India do not have access to the internet and thus the app embedded in the CoviSelf kit is out of reach for them. India's digital divide has prevented universal internet access because there are too many poor households that lack the financial resources to buy sufficiently sophisticated mobile phones or pay for online information. There is also the matter of the privacy of the user for, as explained above, personal details are being entered into a Government of India database without prior consent from the individual users of CoviSelf. These realities fly in the face of Mylab's objective of bringing CoviSelf to the 'doorstep of every Indian ... any citizen', as stated in the company's press release of 10 May 2021 [3].

Might the official approval of CoviSelf be defended by the argument that the privacy of the poorest Indian citizens cannot be compromised if they are unable to download the Mylab app or upload their test results?

Such a defence makes no sense for a number of reasons. First, villagers come and go between villages and towns, and they might well purchase a CoviSelf kit thinking that it will help to protect their families from infection. The poor are attracted to innovations that hold out the promise of a better life, and unsuitable decisions can be made as a result. Second, even if they cannot read the CoviSelf leaflet, villagers can go to a local 'fixer', who, for a small fee, will read the leaflet and try to handle the IT component of CoviSelf on their smart phone. This 'solution' has been reported for rural residents accessing other government apps [54]. Should this happen, the villager becomes caught up in all the limitations of the kit. When the fixer follows the leaflet, the first thing they will read is the instructions for downloading the Mylab app and the last thing they will read will be about sending the image on the test card to Mylab. The fixer might not realise that the test result is before them on the test card and that nothing further needs to be done to find out whether the user has tested positive or negative for the virus.

The self-administered CoviSelf RAT kit, therefore, fails two tests of importance in the world's largest democracy: the right of all Indians to access new technologies approved by the Indian government and the right to privacy. The IT component of CoviSelf carries an additional risk, namely that it could create suspicion about the real purpose of this self-test diagnostic device and also about other government-approved medical technologies. If the government hopes to contain the spread of COVID-19 and harness self-administered RATs to that end, it should consider using local hospitals with good reputations in rural areas. For example, in Wardha District, we discovered that the Kasturba and Datta Meghe Hospitals are highly regarded by villagers for their excellent services, including rural outreach work [43]. It would not be difficult to involve the professionals from such institutions in the promotion and use of CoviSelf. Such professional backing could also help to disseminate more accurate information about the nature of COVID-19 and how to avoid infection. Trusted health professionals could explain to villagers the health risks posed by *symptomless* individuals who are nevertheless carrying COVID-19. Normally, villagers do not seek diagnosis or treatment for symptomless family members, as we know from the failures of the government's anti-TB program [78].

The prerequisites for such outreach are twofold: make the kit free to all citizens and remove the IT component. Given the multidimensional nature and extent of Indian poverty, the first point is significant. The second point also matters if CoviSelf is to be trusted by villagers. The present design of CoviSelf is likely to raise the suspicions of villagers: if medical professionals are seen entering a villager's personal details into a smart phone, it would soon become widely known that the test result and private information were being sent by Mylab to the Government of India. The Hindi word for government is '*sarkar*', and, amongst India's poor, it inspires fear rather than confidence. For complex historical reasons, '*sarkar*' has never been a trusted institution in India. The fear would be not only about the possible uses of such data but also anxiety regarding some kind of government or police action in the event that they test positive.

## **6. Conclusions**

Our evaluation of India's first officially approved self-administered COVID-19 RAT kit uses the framework of discourse analysis to ask whether CoviSelf is of any practical value in rural India. Drawing on data from a wide range of qualitative, textual sources, we have identified a number of significant problems with the language of the instruction leaflet and the assumptions it makes about the living conditions in rural India; specifically, its promotion of the IT features of this diagnostic device betrays a lack of insight into why India's 'have-nots' have been excluded from the information technology revolution.

The key lesson emerging from our evaluation is that, if CoviSelf is to be made more practical and accessible to Indian villagers, the IT functions should be removed and the kit made cost-free to users. Such changes would also automatically protect the privacy of the user and their test result. The ambitions of Mylab could well be realised by such adjustments, namely the production of a self-administered, Indian-made COVID-19 diagnostic device for every Indian. In removing the embedded IT features of CoviSelf, the instruction leaflet would need to be rewritten, and that would provide an opportunity to address the other language problems that we found in the text.

Such changes would also help frustrated urban middle-class users whose struggles with the kit are recorded on the Google Store website. Many users wrote that the kit was a waste of money, some suggesting that it was a scam, others that the instructions were wrong and that the test card was not working properly. One frustrated customer (Ramana G) wrote on 1 November 2021 [79]:

I had to take three tests and the result is still inconclusive, what is the problem. The 1st test ... I couldn't squiz [*sic*] the liquid onto sample level and I broke the well bottom and the lines so I exited the App. But then I tried the second time and did everything right and found that results were inconclusive as I scanned later than 20 min... I tried third time and exactly waited for 15mins and it's the same problem.

Some three months later, another customer (Arjita Mukherjee) on the same website commented:

I followed all instructions yet I didn't receive my results. Just said Invalid Casette, low server problem. Second, I understand that there might be server issue, then what's the point of keeping another 15 min time interval when the whole kit gets invalid after 20 min. Just keep the test result appearance that's it. Really disappointed. The manual rtpcrs are far safe and better I suppose.

This kind of feedback allows us to predict a villager's likely experience with CoviSelf in its present form. The online customer reviews confirm our evaluation of the instructional leaflet—namely that there are serious communication problems with the language in the leaflet, the instructions to users and the IT features.

The potential for self-administered RATs in Indian villages, especially in the face of the highly infectious Omicron strain, remains undiminished. The majority of Indians live in rural areas where access to timely medical diagnosis, assistance and drugs is seriously constrained. Modifying the CoviSelf kit to remove the limitations that we have identified in this paper could assist with its equitable distribution as an effective, risk-minimising device against galloping infections in villages. Richard Hatchett has given a prescient warning at the height of the Omicron wave: 'unpredictably, the virus appears to have the capacity to become, essentially, a pandemic at any time ... access to diagnostics—and updated diagnostics, are absolutely critical to managing an infectious disease crisis' [46]. Given this, the distribution of self-administered RATs such as CoviSelf should accompany vaccines and anti-viral drugs as an essential part of India's pandemic planning.

**Author Contributions:** M.V.: lead author; conceptualisation; methodology; formal analysis; resources and data curation; original draft preparation; writing, review and editing; supervision; project administration; 40 years of fieldwork experience in India. J.H.: validation; data curation; review and edition; 20 years of fieldwork experience in India. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Informed Consent Statement:** Not applicable.

**Acknowledgments:** We are very grateful to Leon Piterman (General Practice, Monash Medical Faculty, Melbourne), Naiyana Tikky Wattanapenpaiboon (Monash Asia Initiative), Ashok Vivek Patel (International Association of Rural Health and Medicine, Pune, Maharashtra) and the three reviewers for their invaluable comments on an earlier draft of this article. We also thank Vivien Seyler (*South Asia: J. South Asian Stud.*) for her editorial support and Anusha Kesarkar-Gavankar (IITB-Monash Research Academy, Mumbai, India) for her work as our research assistant. Our special appreciation also goes to Ashok Gulati (Agriculture, ICRIER, New Delhi, India) for his advice on some data issues.

**Conflicts of Interest:** Neither author has any connections with the Indian government, Covi-Self or any other stakeholder discussed in this paper. The paper did not involve any human or animal experiments.

## **References**


## *Article* **Significance of Lung Ultrasound in Patients with Suspected COVID-19 Infection at Hospital Admission**

**Holger Gutsche 1, Thomas G. Lesser 1, Frank Wolfram 1,\* and Torsten Doenst <sup>2</sup>**


**Abstract:** With a lung ultrasound (LUS) the typical findings are interstitial pneumonia. COVID-19 pneumonia is often manifested in sub-pleural areas, which is preferably detected by sonography. An RT-PCR test cannot always ensure a safe differentiation of COVID-19- and non-diseased cases. Clinically challenging is that a reliable and time efficient decision regarding COVID-19 suspects requiring isolation. Therefore, this study was aimed at evaluating the significance of LUS in symptomatic patients with COVID-19 suspicion at hospital admission. A total of 101 patients admitted to a suspect ward with COVID-19-typical symptoms were assessed. All patients received prospectively a standardized LUS at admission. Patients were classified as LUS-positive and -negative cases based on a specific LUS score. The RT-PCR test in combination with the clinical findings served as a reference. Correctly classified were 14/15 COVID-19 diseased suspects as LUS-positive (sensitivity: 93.3%). Twenty-seven out of 61 non-positive cases were classified as false positive with LUS (specificity: 55.7%). In 34/35 patients who were assessed as LUS negative, no COVID-19 disease was detected during the hospitalization. The PPV and NPV of the LUS were 34.1% and 97.1%. LUS is a valuable tool in symptomatic patients for the assessment of COVID-19-disease. The high negative predictive value of LUS is helpful to rule out the disease.

**Keywords:** lung ultrasound; COVID-19; POCUS

## **1. Introduction**

The COVID-19 pandemic poses a major challenge to the health care system. Among other things, the timely and safe differentiation between symptomatic patients with COVID-19 disease and those not infected by Sars-COV-2, is a challenge particularly for hospitalized patients. Due to the very non-specific symptoms such as fever, cough, cold, or unclear infection parameters, a high number of patients without COVID-19 disease are admitted to isolated COVID-wards. Especially elderly, often multi-morbid, patients are thereby exposed to an additional risk of infection, and treatment of their underlying disease and comorbidities might be delayed. To this end, medical resources are tied up in such cases, and this increases the strain upon the health care system in a pandemic situation.

With currently available testing methods, notably the RT-PCR test, the absence of Sars-Cov2 infection can only identified correctly and expeditiously to a limited extent. The false negative rate of the RT-PCR test is being reported up to 38% [1], which can arise as a result of poor sampling quality, improper collection, or unfavorable transport conditions [2]. Furthermore, due to limited test capacities, results can be delayed for several days, or positive test results might be present only after the second or third sample [3]. Therefore, a single negative RT-PCR test result on respiratory specimens is insufficient to rule out COVID-19 disease [4–6].

Symptomatic COVID-19 disease is dominantly associated with pneumonia. For imaging and monitoring of those, chest X-Ray (CXR) is of minor importance compared

**Citation:** Gutsche, H.; Lesser, T.G.; Wolfram, F.; Doenst, T. Significance of Lung Ultrasound in Patients with Suspected COVID-19 Infection at Hospital Admission. *Diagnostics* **2021**, *11*, 921. https://doi.org/10.3390/ diagnostics11060921

Academic Editor: Tivani P. Mashamba-Thompson

Received: 29 March 2021 Accepted: 17 May 2021 Published: 21 May 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

with computed tomography (CT) because of missing specificity [7]. Lung CT of COVID-19 pneumonia shows typical findings of interstitial pneumonia with a sensitivity of 94% [8]. It is recommended in case of urgent COVID-19 suspicion and a negative RT-PCR. However, the examination is not practicable as an incoming screening examination due to high patient volume in a pandemic situation, transport problems, hygiene guidelines, and associated radiation exposure.

The lung ultrasound is becoming a more and more important point-of-care imaging modality in the differential diagnostics of pulmonary and pleural diseases [9]. In the case of interstitial pneumonia, typical ultrasound findings such as an irregular or fragmented pleural line, B-lines, white lung syndrome (WLS), or consolidations can be collected [10]. Since COVID-19 pneumonia manifests mainly peripheral/subpleural, an ideal prerequisite for the use of LUS is available [11]. Early stages of pneumonia can be detected with ultrasound, and the findings correlate well with those by computed tomography [12–14]. Due to its portability, LUS allows bedside imaging in absence of radiation and could be a valuable diagnostic screening tool for the early detection or exclusion of patients with Sars-COV-2 infections.

Therefore, this study is aimed to investigate the significance of LUS in symptomatic patients with COVID-19 suspicion at the time of hospitalization. For this, the diagnostic value of a lung ultrasound with reference to the RT-PCR test with the clinical findings was determined.

## **2. Materials and Methods**

## *2.1. Patients*

In the period from April to June 2020, patients with clinical suspicion of COVID-19 disease underwent prospective and standardized LUS after admission to the COVID-19 suspect ward of the SRH Wald-Klinikum Gera. The symptoms for inclusion were cough, dyspnea, unclear fever (>38 ◦C), as well as an increase in laboratory infection parameters (CrP > 10 mg/L; PCT > 0.5 mg/L). Patient characteristics were separately documented from the digitally stored patient data after anonymization. These included ages, gender, symptoms, computed tomography of the chest (if done), and laboratory findings, as well as secondary diseases.

Inclusion criteria were an age over 18 years, no pregnancy, and LUS examination within 48 h after hospital admission. Exclusion was performed in need of immediate ICU transfer and an uncooperative condition.

## *2.2. Lung Ultrasound Examination*

All examinations were carried out by one experienced LUS investigator with a background of six years in thoracic and vascular surgery with daily practice of general and lung sonography. The examiner received prior to the study a general LUS training according to EFSUMB and AIUM recommendations and COVID-19 specific online training.

The systematic LUS examination was performed bedside on six thoracic areas of the right (R) and left (L) hemi-thorax in an insulating room at the COVID-suspect ward.

The investigation was conducted on the basis of international recommendations for COVID-19 lung ultrasound examination [15,16]. Herein the probe was placed onto the intercostal place to ensure imaging without rib shadowing. The investigator scanned for suspicious or pathological LUS feature within each areal.

The following chest areas were investigated:


The examinations were performed with a clinical sonography scanner (LOGIQ/VENUE, GE HealthCare, Solingen, Germany) using a linear probe (9L) or in obese patients with a convex probe (C1-5). The scanner specific lung ultrasound pre-set was used with a median MI of 1.1 (0.8–1.2) and TIS of 0.1 (0.0–0.2). The operator adjusted the placement of the single focus onto the pleural line. LUS frames of five seconds were initially recorded from each areal to ensure the monitoring of at least one breath cycle. The video sequences were evaluated blindly by two LUS experienced investigators (FW, HG) independent of each other, and without knowledge of patients' condition including the RT-PCR status. In the case of different assessments, a third clinician (TL) was consulted for consensus formation. The definitive classification of all 12 lung areas was carried out by means of a lung score of 0–3 based on the international COVID-19 LUS scoring system according to Soldati et al. [16]:


Patients were classified as "LUS-COVID-positive" if a score of 2 or 3 was present in one area out of 12. A classification as "LUS-COVID-negative" was made when a score of 0 or 1 was found throughout all areas. The patient was excluded from the study if more than one area was missing or could not be assessed.

## *2.3. RT-PCR Reference*

The patients suspected for COVID-19 disease already received a RT-PCR test in the emergency department (ED) at admission. In the case of negative test results and continued symptomatic course, a repeated RT-PCR test was carried out by sampling an irritant sputum or throat rinse water. As soon as a positive single test was present during the course of the disease, the patient was classified as "COVID-19-diseased". All RT-PCR tests were carried out by SYNLAB Medical Care Center Gera (SYNLAB MVZ, Weiden, Germany). A "first RT-PCR" test was defined if taken and successfully analyzed at the ED in the moment of admission. Out of 76 included cases a first RT-PCR test was successfully performed on 68 cases. Four RT-PCR samples could not be evaluated due to material defects and four patients were admitted to the COVID-19 ward due to a positive ambulatory COVID-19 test without detailed documentation.

## *2.4. Statistical Analysis*

The statistical analysis was performed using MedCalc (Vers. 19.1.7 Medcalc LTD, Ostend, Belgium). Quantitative variables are presented as median and Inter Quartile Range (25th percentile, 75th percentile), and categorical as counts and proportion. Statistical evaluation was performed on the categorical variables with the Fisher's Exact–Boschloo test, and on continuous data with Mann–Whitney U tests. The tests' power was estimated based on the resulting counts [17]. In addition, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and diagnostic accuracy were evaluated for the LUS score, first RT-PCR Test, and in a combination of both. The positive count for combination of LUS and RT-PCR test was made when an LUS-COVID-positive or positive first RT-PCR test result was present. The estimates are reported with 95% confidence interval (CI). A two-sided *p*-value < 0.05 was defined as significant.

## **3. Results**

## *3.1. Patient Characteristics*

A total of 101 patients with clinical suspicion of COVID-19 disease were examined using LUS. After excluding 25 cases, 76 patients were included for the study. A summary of studies patient selection is shown in Figure 1.

**Figure 1.** Flow chart of the patient selection and classification according to the lung ultrasound and reference standard.

Baseline characteristics, secondary diseases and clinical symptoms of the study patients are presented in Table 1. The median age was 75.5 years. Both genders were almost equally distributed (male 48.7% and female 51.3%). The median length of stay on the COVID-19 suspect ward was 4.7 days for all patients, 2.4 days for non-COVID-19-diseased and 13.9 days for COVID-19-diseased patients. In 15 (19.7%) of the patients, COVID-19 disease was finally diagnosed, in 61 patients (80.3%) COVID-19 disease had been excluded after the end of the diagnostics. With regard to the incidence of fever, cough, or dyspnea as well as blood gases pO2 and pCO2, no differences were found between COVID-19 and non-COVID-19-diseased patients (*p* > 0.05). The loss of taste and/or sense of smell were found significantly more commonly found among patients with confirmed COVID-19 disease (*p* = 0.047). No significant difference with regard to the secondary diseases was present between non- and COVID-19-diseased cases.


**Table 1.** Baseline characteristics of the study patients with secondary diseases and clinical symptoms.

A *p*-value < 0.05 was defined as significant.

## *3.2. Diagnostic Value of Lung Ultrasound and in Combination with RT-PCR Test*

The LUS examination was performed at the COVID-suspect ward within a median time delay of 18 h 5 min (9 h 48 min–23 h 30 min) after admission to the hospital's ED. Fourteen out of the fifteen COVID-19-diseased patients were classified LUS-COVIDpositive (sensitivity: 93.3%; CI: 68.1% to 99.8%). Of 61 non-COVID-19-diseased patients, 27 LUS-COVID-positive cases were false-positive classified with LUS (specificity: 55.7%; CI: 42.5% to 68.5%). In 34 patients who were assessed as negative by LUS (LUS-COVIDnegative), no COVID-19 disease was detected in the further course of the hospitalization. One patient was classified LUS-COVID-negative in the presence of COVID-19 disease and therefore judged to be false-negative. This patient was admitted with symptomatic cough and an increased infection parameter (CrP = 47 mg/L) due to a fracture from an external, not COVID-19 treating, hospital. No fever or dyspnea and lung infiltrations on CXR were present.

The positive and negative predictive value (PPV, NPV) of the LUS was 34.2% (CI: 27.5% to 41.5%) and 97.1% (CI: 83.5% to 99.6%), respectively. The first RT-PCR test, carried out at hospital admission such as in the ED (68 of 72 RT-PCR tests), showed a sensitivity of 90.9% (CI: 58.7% to 99.8%), a specificity of 100% (CI: 93.7% to 100%), resulting in a PPV of 100% and an NPV of 98.3% (CI: 89.8% to 99.7%). Until a reliable result was available, approximately 2.1 (CI: 1.9 to 2.3) RT-PCR tests were performed, therefore non-COVID-19 diseased patients were discharged from the COVID-19 ward after 2.4 days (CI: 2.0 to 2.9). The result of the first RT-PCR test combined with the LUS-COVID classification, revealed a sensitivity of 100% (CI: 71.5% to 100%) and thus a negative predictive value of 100% (Table 2).


**Table 2.** Diagnostic significance of LUS score classification, first RT-PCR test, and in combination of both in COVID-19 suspected patients.

LUS, lung ultrasound; PPV, positive predictive value; NPV, negative predictive value.

## *3.3. Pathological Lung Ultrasound Findings*

More than two B-lines, white lung syndrome, pleural fragmentation, and consolidations were found significantly more frequently in COVID-19-diseased than in non-COVID-19-diseased patients. When COVID-19 was present, more than two B-lines appear in all patients, WLS in 66.7%, subpleural consolidation in 53.3%, and pleural fragmentation in 93.3%. COVID-19 patients were significantly more often classified as LUS-COVID-positive than non-COVID-19-diseased cases (93.3% vs. 44.3%, *p* < 0.01). For COVID-19-diseased patients an LUS score of 3 was found in 53.3%, a score of 2 in 40%, whereas only one patient (1.6%) showed an LUS score 1, characterized by a thickened pleural line in three areas and increased B-lines in two areas. Pleural effusion was found more frequently in non-COVID-19-diseased (23%) than in COVID-19-diseased (6.7%) patients which revealed no statistical significance. A summary of LUS score and pathological findings are presented in Table 3.

**Table 3.** Classification results and lung ultrasound characteristics of study patients.


A *p*-value < 0.05 was defined as significant, \* indicate Fischer's exact test with zero counts, see limitation.

The most common LUS findings in non-COVID-19-diseased patients were thickened pleura (98.4%), increased B-lines (75.4%), and a pleural fragmentation (42.6%). Subpleural consolidation as a typical finding in interstitial pneumonia was shown in 26.2%. LUS scores of 3 and 2 were present in 26.2% and 18.0%, respectively. An LUS score 1 was significantly more common in non-COVID-19-diseased than in COVID-19-diseased patients (54.1% vs. 6.7%).

Non-pathological, aerated lung in all areas (score 0) was not present in all COVID-19 diseased, and in only one non-COVID-19-diseased (1.6%) patient, which was not significant.

## **4. Discussion**

We were able to show that the early lung ultrasound examination of inpatient admission provides a diagnostic gain and is valuable in the clarification of Sars-COV-2 suspected patients at hospital admission. With LUS, 93.3% of the COVID-19-positive cases could be classified correctly. In 97.1% of the LUS-negative cases, no COVID-19 disease was present. The NPV increases to 100% when the LUS findings are combined with the first RT-PCR test, carried out at hospital admission.

In contrast, symptomatic screening (fever, cough, dyspnea, blood gas) is insufficient, as our baseline characteristic shows. COVID-19 specific symptoms are slightly more frequently found on diseased patients, which was not significant. Although the loss of taste and smell is a significant symptom it is not applicable for screening due to the low incidence (6.8%).

The duration on a COVID suspect ward (2.4 days) for non-COVID-19-diseased cases is signally inappropriate, causing a delay of therapy, risk of infection, and an unnecessary consumption of resources. The RT-PCR test shows a valuable diagnostic accuracy for the study specific cohort of symptomatic suspects at hospital admission. However, the LUS provides a higher sensitivity than the first RT-PCR test, but with a much lower specificity. Using LUS in combination with an RT-PCR test could help reducing the isolation time on a suspect ward, and therefore reduce the strain on the health care system in a pandemic situation.

The specificity of the LUS examination is low (55.7%), as the sonographic findings are not only COVID-19 specific. Such pathological LUS signs can occur in other diseasecausing interstitial pneumonia, or are present due to underlying chronical lung disease, as frequently found in patients requiring hospitalization. Therefore, LUS cannot provide an etiological diagnosis. Comparable results were found in a recent study by Sorlini et al. [18], reporting a sensitivity and specificity of 92.0% and 64.9%, respectively. The higher specificity can be explained by the higher prevalence of the COVID-19 disease (74.7%) in the cohort studied.

In addition, Volpicelli et al. [19] reported an LUS sensitivity of 90.2% and specificity of 50.5% investigating early LUS under various inclusion criteria. Narinx et al. [20] found an LUS sensitivity of 93.3% and specificity of 21.3% for patients at ED admission. Highly interesting, is the work of Pivetta et al. [21], with a high LUS sensitivity (94.4%) and specificity (95%). In contrast to our work, all the above studies classified the appearance of increased B Lines and WLS as a COVID-19 related LUS sign, which would represent a score of 1 in our study. This choice, different inclusion criteria, and the clinical situation of the LUS examiner being aware of the patients' condition (not blinded), might explain the high specificity found by Pivetta et al. However, these studies in addition to our results, demonstrate the potential of LUS, but also the need for a deeper definition of COVID-19 specific LUS features and for an appropriate definition of inclusion criteria.

The LUS score classification has a decisive influence on the test results of the LUS. A lower cut off, that includes score 1 to be LUS-COVID-positive, leads to a high sensitivity and high false positive rate. LUS as an early diagnostic method of non-COVID-19 diseased cases should be focused on the exclusion of the disease to avoid hospital admission. Our chosen cut off provides a justifiable NPV, which is to our knowledge the highest one reported so far. Therefore, we recommend expressing suspicion of COVID-19 disease only with an LUS score of ≥2 (Figure 2) for patients in early hospital admission. In our study, a high prevalence of cardiac disease (82%) and hypertension (80%) were present in the non-COVID-19-disease group, which are often associated with increased lung water. These underlying pathologies frequently manifest pathological LUS artefacts such as B-lines and WLS (score 1) [9].

**Figure 2.** Lung ultrasonography image of a patient with COVID-19 disease: (**a**) pronounced fragmentation of the pleural line and small subpleural consolidations (according to LUS score 2) and (**b**) lung ultrasonography image of a patient with COVID-19 disease. Thickened pleural line with irregularity and a large consolidation with air bronchogram (arrow; according to LUS score 3). Corresponding video frames are available online (Supplementary Videos S1 and S2).

> When assessing LUS findings, the presentation of clinical symptoms as well as the stage of the disease should be considered. There is likely to be a weak correlation between the severity of symptoms and the specificity of LUS signs. As found in studies with non-critical and mild symptomatic COVID-19 patients, the characteristic sonographic manifestations show increased numbers of B-lines as well as sub-pleural consolidation [22]. In our experience with hospitalized symptomatic patients, sub-pleural consolidations and pleural fragmentation are more typical LUS findings in the presence of COVID-19, even if detected in only one lung areal.

> Furthermore, the timing of the ultrasound examination with regard to the disease stage, impacts the availability of the LUS features. If applied at the recovery stage, pathological LUS findings may be diminished. Thereby, the false negative case in our study could be explained. The patient was admitted from the local hospital due to a fracture and cough. The COVID-19 disease might already have been in remission, which was further indicated by the negative CXR that showed an absence of lung infiltration.

> During COVID-19 disease, the histopathological features may vary and coexist, dependent upon the time-point in disease evolution, and the severity of disease [23]. Different phases of the alveolar damage were described, such as pre-exudative, exudative, organizing, and fibrotic phases. We believe that the LUS features can correspond approximately to the histopathological phases. Sonographic appearance of thickened pleura and increased B-line counts are signs at the onset of the disease, whilst pleural fragmentation and consolidations occur more frequently in the later stages.

> Herein it was noticed that linear probes visualize pleural irregularities more clearly than curved probes, and therefore represent our first choice.

> One unspecific LUS finding, is the pleural effusion which was initially considered as a pathological COVID-19 finding [24]. However, as found in this study, pleural effusion was more frequently present in non-COVID-19-diseased patients and appeared only in one COVID-19-positive case, which confirms the findings of other studies [14,25].

> The limitations of our study are primarily the low number of cases. The results should be verified on a larger patient sample size. However, a sufficient power (>0.8) was estimated for exact testing, based on the proportions found in the study. The significance

found for pathological B-Lines should be considered with caution, due to instability of Fischer's exact test when zero counts are present [26].

In order to optimize the triage of incoming symptomatic patients in need for hospitalization, early LUS should be recommended for use in the ED. Despite the relatively long inclusion criteria of 48h after admission, most of the LUS exams were performed within the first day (median delay 18 h). A corresponding prospective study is recommended. The evaluation of the LUS findings and the classification into a score system are subjective and examiner dependent. Using a retrospective assessment by three experienced LUS clinicians, we have tried to minimize this subjective influence. The study included patients with clinical symptoms or infection parameters that led to hospitalization. Whether patients in the outpatient setting with low symptoms also have the described LUS findings, cannot be answered with the present study.

## **5. Conclusions**

The manifestation of COVID-19 pneumonia in the peripheral/subpleural lung areas, is an ideal condition for the application of LUS. Point-of-care ultrasound of the lung in patients with suspected COVID-19 disease should play a key role at the initial examination. The high negative predictive value of LUS is helpful for the exclusion of the infection and is further improved in combination with a simultaneous negative RT-PCR test. LUS can be helpful to minimize the risk of COVID-19 infection due to unfounded admission to COVID-19 wards.

**Supplementary Materials:** The following are available online at https://www.mdpi.com/article/10 .3390/diagnostics11060921/s1, Video S1: Video frame corresponding to Figure 2a, Video S2: Video frame corresponding to Figure 2b.

**Author Contributions:** Conceptualization, H.G., T.G.L., F.W., and T.D.; LUS acquisition was performed by H.G. Analysis of LUS records performed by H.G., F.W., and T.G.L. Statistics were determined by F.W. and H.G. Writing and revision the original draft T.G.L., H.G., F.W., and T.D. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding. Costs were covered by internal budgets of the SRH Wald-Klinikum Gera.

**Institutional Review Board Statement:** The study was conducted according to the guidelines of the Declaration of Helsinki. The study data were acquired during general clinical procedures and analyzed retrospectively. Therefore, no Ethics approval was required. All patients gave consent verbally or in writing to be examined by LUS.

**Informed Consent Statement:** Informed consent was obtained from all subjects involved in the study, either verbally or in writing.

**Data Availability Statement:** The LUS datasets (recorded frames) of Figure 2 are available online (see Supplementary Materials) and for further request from the corresponding author.

**Acknowledgments:** The authors would like to thank the staff of the SRH Wald-Klinikum COVID station, to make this work feasible, in particular Sylke Schneider.

**Conflicts of Interest:** The authors declare no conflict of interest.

## **References**


## *Article* **Comparison of Lung Ultrasound versus Chest X-ray for Detection of Pulmonary Infiltrates in COVID-19**

**María Mateos González 1,2, Gonzalo García de Casasola Sánchez 1,2, Francisco Javier Teigell Muñoz 1,2, Kevin Proud 3,4, Davide Lourdo 1,2, Julia-Verena Sander 5, Gabriel E. Ortiz Jaimes 3,4, Michael Mader 6, Jesús Canora Lebrato 7, Marcos I. Restrepo 3,4 and Nilam J. Soni 3,4,8,\***


**Abstract:** Point-of-care lung ultrasound (LUS) is an attractive alternative to chest X-ray (CXR), but its diagnostic accuracy compared to CXR has not been well studied in coronavirus disease 2019 (COVID-19) patients. We conducted a prospective observational study to assess the correlation between LUS and CXR findings in COVID-19 patients. Ninety-six patients with a clinical diagnosis of COVID-19 underwent an LUS exam and CXR upon presentation. Physicians blinded to the CXR findings performed all LUS exams. Detection of pulmonary infiltrates by CXR versus LUS was compared between patients categorized as suspected or confirmed COVID-19 based on reverse transcriptasepolymerase chain reaction. Sensitivities and correlation by Kappa statistic were calculated between LUS and CXR. LUS detected pulmonary infiltrates more often than CXR in both suspected and confirmed COVID-19 subjects. The most common LUS abnormalities were discrete B-lines, confluent B-lines, and small subpleural consolidations. Most important, LUS detected unilateral or bilateral pulmonary infiltrates in 55% of subjects with a normal CXR. Substantial agreement was demonstrated between LUS and CXR for normal, unilateral or bilateral findings (K = 0.48 (95% CI 0.34 to 0.63)). In patients with suspected or confirmed COVID-19, LUS detected pulmonary infiltrates more often than CXR, including more than half of the patients with a normal CXR.

**Keywords:** ultrasound; imaging; X-ray; chest; diagnosis; SARS

## **1. Introduction**

Diagnosing coronavirus disease 2019 (COVID-19), the disease caused by the novel coronavirus SARS-CoV2, has been a major challenge as the pandemic has spread rapidly across the globe. Most patients present with nonspecific symptoms, including fever, cough, dyspnea, myalgias, and headache [1], that are indistinguishable from other respiratory infections. To confirm the disease in suspected patients, clinicians most often order reverse

**Citation:** Mateos González, M.; García de Casasola Sánchez, G.; Muñoz, F.J.T.; Proud, K.; Lourdo, D.; Sander, J.-V.; Jaimes, G.E.O.; Mader, M.; Canora Lebrato, J.; Restrepo, M.I.; et al. Comparison of Lung Ultrasound versus Chest X-ray for Detection of Pulmonary Infiltrates in COVID-19. *Diagnostics* **2021**, *11*, 373. https://doi.org/10.3390/ diagnostics11020373

Academic Editor: Byeong-Ho Jeong

Received: 1 February 2021 Accepted: 19 February 2021 Published: 22 February 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

transcriptase-polymerase chain reaction (PCR) testing, but PCR testing has limited availability, relatively high false negative rates early in the course of the disease, and a delay of a few hours to days for results to be obtained [2,3].

Diagnostic imaging is being used to support a diagnosis of COVID-19 by detection of pulmonary infiltrates in suspected patients. Chest computed tomography (CT) scans have demonstrated superior diagnostic sensitivity for detecting pulmonary infiltrates in COVID-19 compared to chest X-ray (CXR) with reported sensitivity of 97–98% after 6 days of symptoms [2–5]. Though sensitive for pulmonary infiltrates, obtaining chest CT scans in all suspected COVID-19 patients is impracticable due to limited access to CT scanners worldwide and infection control requirements for disinfecting CT scanners. The American College of Radiology has recommended against routine use of CT scans for evaluating patients with suspected COVID-19 [6]. For these reasons, CXR and lung ultrasound (LUS) have been the primary imaging modalities used in the diagnosis of COVID-19 worldwide. CXRs can be obtained rapidly with minimal radiation exposure to patients, but have low sensitivity (46–69%) for detecting pulmonary infiltrates in COVID-19 patients [7,8].

Lung ultrasound (LUS) is an attractive alternative to CXRs and CT scans in COVID-19. Point-of-care or bedside LUS has several unique advantages in COVID-19, including immediate availability of findings to guide clinical decision-making, availability of portable ultrasound devices in austere settings such as field hospitals, repeatability to monitor patients serially, and ease of machine decontamination. Studies in non-COVID-19 patients have shown LUS has superior sensitivity (95% (95% CI 92–96%) vs. 49% (40–58%)) and similar specificity (94% (CI 90–97%) vs. 92% (CI 86–95%)) compared to CXRs when using chest CT scan as the gold standard [9]. Several recent studies have described lung ultrasound patterns in COVID-19 [10–16], but few studies have compared the diagnostic accuracy of LUS versus CXR for identifying lung abnormalities [17,18]. The objective of this study was to assess the correlation of LUS and CXR for detecting pulmonary infiltrates in COVID-19 patients.

## **2. Materials and Methods**

## *2.1. The Study Design and Subjects*

A prospective observational study of consecutive patients presenting with a clinical diagnosis of COVID-19 during the first COVID-19 surge in Spain was conducted from March 18, 2020 to April 5, 2020. The setting was an emergency department of a 247 bed university-affiliated teaching hospital in Madrid, Spain. Subjects were eligible for enrolment if they were an adult (age >18 years) and had a clinical diagnosis of COVID-19 based on classic symptoms of COVID-19 (fever, chills, cough, shortness of breath, sore throat, headache, myalgias, anosmia, ageusia, or diarrhea), close contact with an individual with active COVID-19, and abnormal laboratory findings (lymphopenia, elevated c-reactive protein, lactate dehydrogenase, D-dimer, and liver transaminases).

During the first surge of the COVID-19 pandemic in Spain in March of 2020, SARS-CoV-2 PCR testing had limited availability, and test results were delayed by 24–72 h. PCR test results of study subjects were not known at the time of study enrolment. During data analysis, subjects were categorized as having "confirmed" COVID-19 defined by a positive PCR test result or "suspected" COVID-19 defined by either a negative PCR test result or nonperformance of PCR testing.

After informing subjects about the study objectives and minimal risks, verbal consent was obtained and documented in the electronic medical record. Written consent using paper was not feasible due to the risk of fomite transmission of SARS-CoV-2 to study personnel. This study complied with the Declaration of Helsinki and was approved by the local ethics committee and hospital research committee (PI 64/20).

## *2.2. Lung Ultrasound Exam*

A bedside LUS exam was performed on each subject by one of two physicians with expertise in point-of-care ultrasound (M.M.G., F.J.T.M.). Both physician sonographers

performed an LUS exam on all subjects who were clinically diagnosed with COVID-19 by an attending physician in the emergency department. The LUS exam was performed independent of the evaluation by the attending physician in the emergency department. Both physician sonographers were blinded to each patient's history, laboratory results, and radiographic images and were not directly involved in the patient's care. PCR test results were not available until 24–72 h after presentation and were not known at the time of the LUS exam.

Two portable ultrasound machines with curvilinear transducers (Mindray M9 (Shenzhen, China) and Esaote MyLab Omega (Genoa, Italy)) were used. The ultrasound machine and transducer were covered with plastic cling film during each exam. The physician sonographers wore N-95/FFP2 masks, impermeable gowns, and two pairs of gloves. Despite the use of personal protective equipment, the physician sonographers were required to stand behind the subjects when performing the LUS exam to avoid face-to-face contact and minimize the risk of viral transmission. The chest wall skin was cleaned with an alcohol-based antiseptic solution before each LUS exam.

The LUS protocol included 5 zones per hemithorax—three posterior zones (superior, middle, and inferior) and two lateral zones along the mid-axillary line (superior and inferior) (Figure 1). A total of 10 zones were scanned per patient. Pathological LUS findings have been previously described [10,11,16]. LUS findings were categorized as normal, discrete B-lines (3 or more B-lines per rib interspace), confluent B-lines, small subpleural consolidations (<3 cm), and lobar consolidations (Figure 2). LUS findings were recorded as video clips and written descriptions were entered into a database.

**Figure 1.** Lung Ultrasound Exam Points. (**A**) After identifying the diaphragm, the transducer was slide cephalad to image the inferior, middle, and superior zones of the posterior chest. (**B**) Along the mid-axillary line, the inferior and superior lung zones of the lateral chest were imaged.

**Figure 2.** Characteristic Lung Lesions in coronavirus disease 2019 (COVID-19). (**A**) Normal lung ultrasound is defined by visualization of pleural sliding and A-lines. (**B**) Discrete B-lines are individual hyperechoic, laser-like artifacts the emanate from the pleural line and are due to increased interstitial fluid in the acute setting. Discrete B-lines are typically the first sign of COVID-19. (**C**) Fused or confluent B-lines are seen when individual B-lines coalesce as interstitial fluid increases. (**D**) Subpleural consolidations are typically small (<3 cm) areas of consolidation that are seen just below the pleural line.

## *2.3. Chest Radiographs*

All CXRs were obtained by a radiology technician and interpreted by a board-certified radiologist. Two CXR views (posterior-anterior and lateral) were taken in the radiology department. The final CXR report was entered into a database for comparison with the LUS findings. A blinded third investigator with ultrasound expertise (G.G.C.) compared the LUS and CXR findings reported by the two physician sonographers and radiologists, respectively.

## *2.4. Statistical Analysis*

Subjects were categorized as having suspected or confirmed COVID-19 based on the PCR testing as stated above. CXR and LUS findings were classified into three ordinal categories for each diagnostic method—disease absent (normal lung), unilateral pulmonary infiltrates, and bilateral pulmonary infiltrates. Agreement between the two diagnostic methods was calculated using the weighted Kappa statistic using the ordinal classification system. The Kappa statistic was interpreted as follows—0.20 to 0.45 moderate agreement, 0.45 to 0.75 substantial agreement, and 0.75 to 1.0 perfect agreement [19]. Sensitivity of each method was calculated, and compared using the McNemar test. Statistical analyses were performed using the frequency (FREQ) procedure in SAS (v.9.4. Cary, NC, USA: SAS Institute Inc.; 2014).

## **3. Results**

One hundred and one subjects were enrolled in the study. Five subjects were excluded (three were pregnant and could not receive a CXR; two had alternative diagnoses found). Data were analyzed from a total 96 subjects with a clinical diagnosis of COVID-19.

Characteristics of the subjects are presented in Table 1. The median age of all subjects was 48 years and half were women. The most common comorbidities were hypertension, obesity, asthma, and diabetes mellitus. A majority of subjects presented with fever, cough, and dyspnea. A greater proportion of suspected COVID-19 subjects presented <7 days whereas more confirmed COVID-19 subjects presented ≥7 days. Compared to suspected COVID-19 patients, the confirmed COVID-19 subjects had a significantly lower oxygen saturation, elevated C-reactive protein, elevated lactate dehydrogenase, and lower lymphocyte count. Most confirmed COVID-19 subjects (81%) were hospitalized while most suspected COVID-19 subjects (94%) were discharged home with close monitoring.

LUS detected pulmonary infiltrates in more subjects than CXR (81% vs. 63%) with a greater difference among subjects with suspected COVID-19 (70% vs. 40%) versus confirmed COVID-19 (95% vs. 91%) (Figure 3). Among the subjects with a normal CXR but abnormal LUS exam, 20 subjects (55%) had pulmonary infiltrates detectable by LUS (Figure 4). Furthermore, most of these subjects (*n* = 12) had bilateral infiltrates that were seen on LUS but not on CXR (Figure S1). On the contrary, among the subjects with a normal LUS exam but abnormal CXR, only two had pulmonary infiltrates detected on CXR which were described as "doubtful" or "minimal" infiltrates in the medial or left basilar lung fields per the radiologist's official report (Figure 4).

**Figure 3.** Chest X-ray and Lung Ultrasound for Detection of Pulmonary Infiltrates. The number of suspected or confirmed COVID-19 subjects (*n*) with or without pulmonary infiltrates detected by chest X-ray or lung ultrasound is demonstrated. In both suspected and confirmed COVID-19 subjects, lung ultrasound was able to detect pulmonary infiltrates more often than chest radiography.


**Table 1.** Characteristics of Subjects with Suspected and Confirmed COVID-19.


**Table 1.** *Cont.*

IQR, interquartile range; LDH, lactate dehydrogenase; CRP, C-reactive protein.

**Figure 4.** Correlation of Chest X-ray and Lung Ultrasound in Detection of Pulmonary Infiltrates. The number of subjects (*n*) and agreement between chest X-ray and lung ultrasound is shown for (**A**) all cases, (**B**) suspected COVID-19 cases, and (**C**) confirmed COVID-19 cases. Lung ultrasound detected pulmonary infiltrates in 20 subjects with a normal chest X-ray, whereas chest X-ray detected pulmonary infiltrates in 2 subjects with a normal LUS exam.

The types of LUS and CXR findings are shown in Table 1. More suspected COVID-19 subjects had a normal LUS and CXR compared to those with confirmed disease. Among all 78 subjects with LUS abnormalities, all subjects had discrete B-lines with pleural line irregularities. Half of all subjects had confluent B-lines and 43% had small subpleural consolidations (<3 cm). In confirmed COVID-19 subjects, alveolar infiltrates on CXR and discrete or confluent B-lines on LUS were more often seen compared to those with suspected COVID-19.

The distribution of pulmonary infiltrates detected by LUS versus CXR in suspected and confirmed COVID-19 subjects is shown in Figure 5 (Tables S1 and S2). LUS detected pulmonary infiltrates compared to CXR in a greater proportion of subjects in both the right (77% vs. 57%) and left lungs (67% vs. 58%). Regarding specific lung lobes, LUS detected pulmonary infiltrates more often than CXR in all lung lobes with the greatest differences in the right middle lobe (62% vs. 32%), right lower lobe (65% vs. 46%), and left upper lobe (52% vs. 35%). In all lung lobes, pulmonary infiltrates were detected more frequently in confirmed versus suspected COVID-19 subjects by either LUS or CXR.

**Figure 5.** Distribution of Pulmonary Infiltrates Detected by Chest X-ray vs. Lung Ultrasound. The number of subjects (*n*) with confirmed or suspected COVID-19 who had pulmonary infiltrates detected in the upper, middle, or lower lobes of the right and left lung is demonstrated.

The correlation between LUS and CXR was assessed by weighted Kappa statistic (Figure S2). A substantial level of agreement was demonstrated between LUS and CXR for normal, unilateral or bilateral pulmonary infiltrates (K = 0.48 (95% CI 0.34 to 0.63)), as defined by Munoz et al. [19]. Comparing normal versus abnormal LUS and CXR, the Kappa statistic similarly showed substantial agreement (K = 0.46 (95% CI 0.28 to 0.63)). LUS was more sensitive than CXR for detecting pulmonary infiltrates (81% vs. 63%; *p* = 0.002) using the McNemar test.

## **4. Discussion**

We reported the findings of a large prospective study assessing the correlation of LUS and CXR for detection of pulmonary infiltrates in noncritically ill COVID-19 patients. A substantial level of agreement was demonstrated between LUS and CXR, and LUS detected pulmonary infiltrates more frequently compared to CXR in all subjects. Most importantly, among the subjects with a negative CXR, abnormalities were detected by LUS in more than half of these subjects.

Confirming a diagnosis of COVID-19 by laboratory testing or diagnostic imaging is challenging, especially early in the course of the disease. PCR testing is limited by availability, high false negative rate (sensitivity 65–83%), and delays in test positivity (mean 5.1 days) [2–5]. In one study, PCR test results turned positive in 21% of patients after two consecutive negative results [20]. In our study, PCR test results were not available until 24–72 h after presentation and were unknown when the LUS exam and CXR were performed. Among the diagnostic imaging modalities, chest CT scan has been reported to have the highest sensitivity (97–98%) [2–5]. However, obtaining chest CT scans on all patients with suspected COVID-19 is impracticable during a pandemic when resources are limited, and most of the world's population lacks access to CT imaging [21]. Thus, clinicians have had to rely primarily on CXRs and LUS to detect pulmonary infiltrates to support a clinical diagnosis of COVID-19.

The LUS findings in COVID-19 have been well described in several reports [10–16]. However, only two small case series have reported both CXR and LUS findings in COVID-19 patients, but neither study directly compared CXR and LUS findings nor assessed the correlation of the two imaging tests [17,18]. In our study, all patients underwent both LUS and CXR upon presentation that were interpreted by blinded experts. We demonstrated a substantial level of agreement between LUS and CXR, but LUS had a higher sensitivity for detecting pulmonary infiltrates compared to CXR (81% vs. 63%). Our findings are consistent with another study reporting the sensitivity of CXR (69%) in COVID-19 patients [8]. Similar sensitivity of LUS (85%) was reported in a meta-analysis of non-COVID pneumonia studies comparing LUS to CXR or chest CT scans [22].

A key finding of our study was the ability of LUS to detect pulmonary infiltrates in more than half of the subjects with a normal CXR. Furthermore, one-third of these subjects had bilateral findings on LUS that were not seen on CXR (Figure S1). On the contrary, only two subjects had lung infiltrates reported on CXR that were not seen on LUS; however, the radiologist's official report commented that these were "doubtful" or "minimal" infiltrates. Based on our findings, institutions with trained clinicians can develop protocols that include LUS as part of the initial bedside evaluation of suspected COVID-19 patients. Though not assessed in our study, bedside detection of pulmonary infiltrates by LUS has the potential to guide triage and treatment decisions as new therapies emerge.

In our study, disposition decisions about hospital admission versus close monitoring at home were determined using a hospital protocol independent of the LUS findings. However, a few points deserve mention from our post-hoc analysis of disposition (Tables S3 and S4). First, subjects with a normal CXR or LUS were more often discharged home. Second, though COVID-19 PCR test results were not known at the time of presentation, more confirmed versus suspected COVID-19 subjects were admitted to the hospital versus discharged home (81% vs. 15%). Most importantly, LUS detected more unilateral (25% vs. 17%) or bilateral pulmonary infiltrates (42% vs. 19%) compared to CXR

in suspected COVID-19 subjects that were safely discharged home. Whether LUS is overly sensitive for detecting pulmonary infiltrates that could lead to unnecessary admission of individuals that could be safely monitored at home is an important question to address in future studies.

We recognize that our study has limitations. First, PCR testing could only be performed on approximately half of subjects in our study because laboratory testing supplies were extremely limited during the initial surge of the COVID-19 pandemic in Madrid. However, given the high false negative rates of early PCR test kits and the 24–72 h delay in obtaining PCR test results, a clinical diagnosis of COVID-19 was typically made based on close contact and supportive laboratory findings. Second, due to concerns of healthcare workers contracting COVID-19, a rapid and focused LUS exam was performed with the physician sonographer standing behind the patient and interrogating the posterior and lateral chest walls. Recent publications have recommended standardization of LUS protocols in COVID-19 to foster pooling of data from multiple institutions in future studies [14,23]. Third, chest CT scans could not be obtained in all patients with suspected COVID-19 due to limited hospital resources, and only three subjects underwent a chest CT scan.

## **5. Conclusions**

In summary, LUS findings correlated well with those of CXR in patients with suspected or confirmed COVID-19. Lung ultrasound was able to detect pulmonary infiltrates in more than half of patients with a normal CXR. Thus, a LUS exam may be performed at the bedside as the initial diagnostic imaging test in patients with COVID-19. Future studies are needed to evaluate the use of a standardized LUS protocol on triage decisions and health services of patients with suspected COVID-19.

**Supplementary Materials:** The following are available online at https://www.mdpi.com/2075-4 418/11/2/373/s1, Figure S1: Comparison of Normal, Unilateral and Bilateral Infiltrates on Chest X-ray and Lung Ultrasound. The agreement be-tween chest X-ray and lung ultrasound is shown for (A) all cases, (B) suspected COVID-19 cases, and (C) confirmed COVID-19 cases. Lung ultrasound detected bilateral pulmonary infiltrates in a substantial proportion of subjects with either a normal or unilateral infiltrates on chest X-ray, Figure S2: Correlation of Lung Ultrasound and Chest X-ray in COVID-19. (A) This plot shows the correlation of findings for normal, unilateral, or bilateral disease. (B) This plot shows the correlations between normal and any disease. The darkest areas indicate exact agreement between LUS and CXR, lightest areas indicate partial or no agreement between LUS and CXR. The 45-degree line above the intersection of the middle rectangles indicates that LUS (plotted vertically) detects more disease than does CXR (plotted horizontally), Table S1: Number of lobes with pulmonary infiltrates detected by chest radiograph stratified by the suspected and confirmed groups of patients with COVID-19, Table S2: Number of lobes with pulmonary infiltrates detected by lung ultrasound stratified by the suspected and confirmed groups of patients with COVID-19, Table S3: Detection of pulmonary infiltrates (none, unilateral, or bilateral) by lung ultrasound versus disposition to discharge home in all, suspected, and confirmed patients with COVID-19, Table S4: Detection of pulmonary infiltrates (none, unilateral, or bilateral) by chest X-ray versus disposition to discharge home in all, suspected, and confirmed patients with COVID-19.

**Author Contributions:** Conceptualization, M.M.G., G.G.d.C.S., K.P., F.J.T.M., D.L., J.-V.S., M.M., J.C.L., M.I.R., and N.J.S.; methodology, M.M.G., G.G.d.C.S., K.P., F.J.T.M., D.L., J.-V.S., M.M., J.C.L., M.I.R., and N.J.S.; software, X.X.; validation, M.M.G., G.G.d.C.S., K.P., F.J.T.M., D.L., J.-V.S., M.M., J.C.L., M.I.R., and N.J.S.; formal analysis, M.M.G., G.G.d.C.S., K.P., F.J.T.M., D.L., J.-V.S., G.E.O.J., M.M., J.C.L., M.I.R., and N.J.S.; investigation, M.M.G., G.G.d.C.S., K.P., F.J.T.M., D.L., J.-V.S., G.E.O.J., M.M., J.C.L., M.I.R., and N.J.S.; resources, M.M.G., G.G.d.C.S., K.P., F.J.T.M., D.L., J.-V.S., M.M., J.C.L., M.I.R., and N.J.S.; data curation, M.M.G., G.G.d.C.S., K.P., F.J.T.M., D.L., J.-V.S., G.E.O.J., M.M., J.C.L., M.I.R., and N.J.S.; writing—original draft preparation, M.M.G., G.G.d.C.S., K.P., F.J.T.M., D.L., J.-V.S., G.E.O.J., M.M., J.C.L., M.I.R., and N.J.S.; writing—M.M.G., G.G.d.C.S., K.P., F.J.T.M., D.L., J.-V.S., G.E.O.J., M.M., J.C.L., M.I.R., and N.J.S.; visualization, M.M.G., G.G.d.C.S., K.P., F.J.T.M., D.L., J.-V.S., G.E.O.J., M.M., J.C.L., M.I.R., and N.J.S.; supervision, M.M.G., G.G.d.C.S., K.P., F.J.T.M., D.L., J.-V.S., M.M., J.C.L., M.I.R., and N.J.S.; project administration, M.M.G., G.G.d.C.S., K.P., F.J.T.M., D.L., J.-V.S., M.M., J.C.L., M.I.R., and N.J.S.; funding acquisition, none. G.G.d.C.S. has full access to all of the data

in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. All authors have read and agreed to the published version of the manuscript.

**Funding:** Nilam J. Soni receives funding from the U.S. Department of Veterans Affairs, Quality Enhancement Research Initiative (QUERI) Partnered Evaluation Initiative Grant, I50 HX002263-01A1. This material is the result of work supported with resources and the use of facilities at the South Texas Veterans Health Care System in San Antonio, Texas. The contents do not represent the views of the U.S. Department of Veterans Affairs or the United States Government.

**Institutional Review Board Statement:** The study was conducted according to the guidelines of the Declaration of Helsinki, and was approved by the ethics and research committee of Hospital Universitario Puerta de Hierro-Majadahonda in March 2020 (PI 64/20).

**Informed Consent Statement:** Verbal consent was obtained and documented in the electronic medical record. Written consent using paper was not feasible due to the risk of fomite transmission of SARS-CoV-2 to study personnel.

**Data Availability Statement:** All the data used in this study will be made publicly upon publication of the study.

**Conflicts of Interest:** The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

## **References**


## *Article* **Availability and Use of Mobile Health Technology for Disease Diagnosis and Treatment Support by Health Workers in the Ashanti Region of Ghana: A Cross-Sectional Survey**

**Ernest Osei 1,\*, Kwasi Agyei 2, Boikhutso Tlou <sup>1</sup> and Tivani P. Mashamba-Thompson 1,3**


**Abstract:** Mobile health (mHealth) technologies have been identified as promising strategies for improving access to healthcare delivery and patient outcomes. However, the extent of availability and use of mHealth among healthcare professionals in Ghana is not known. The study's main objective was to examine the availability and use of mHealth for disease diagnosis and treatment support by healthcare professionals in the Ashanti Region of Ghana. A cross-sectional survey was carried out among 285 healthcare professionals across 100 primary healthcare clinics in the Ashanti Region with an adopted survey tool. We obtained data on the participants' background, available health infrastructure, healthcare workforce competency, ownership of a mobile wireless device, usefulness of mHealth, ease of use of mHealth, user satisfaction, and behavioural intention to use mHealth. Descriptive statistics were conducted to characterise healthcare professionals' demographics and clinical features. Multivariate logistic regression analysis was performed to explore the influence of the demographic factors on the availability and use of mHealth for disease diagnosis and treatment support. STATA version 15 was used to complete all the statistical analyses. Out of the 285 healthcare professionals, 64.91% indicated that mHealth is available to them, while 35.08% have no access to mHealth. Of the 185 healthcare professionals who have access to mHealth, 98.4% are currently using mHealth to support healthcare delivery. Logistic regression model analysis significantly (*p* < 0.05) identified that factors such as the availability of mobile wireless devices, phone calls, text messages, and mobile apps are associated with HIV, TB, medication adherence, clinic appointments, and others. There is a significant association between the availability of mobile wireless devices, text messages, phone calls, mobile apps, and their use for disease diagnosis and treatment compliance from the chi-square test analysis. The findings demonstrate a low level of mHealth use for disease diagnosis and treatment support by healthcare professionals at rural clinics. We encourage policymakers to promote the implementation of mHealth in rural clinics.

**Keywords:** mHealth applications; disease diagnosis; treatment support; sub-Saharan Africa

## **1. Introduction**

Sub-Saharan African (SSA) countries, including Ghana, are confronted with a double burden of communicable and non-communicable diseases [1,2]. They also have weak healthcare systems, which has been exacerbated by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) pandemic [1,3–5]. In addition, poor access to healthcare due to insufficient healthcare infrastructure, poor road networks, long-distance travel to health facilities, inadequate health education, lack of financial resources, insufficiently trained health professionals, and many others also further weaken healthcare systems [6,7]. The government of Ghana (GoG) has committed to improving the digitisation of healthcare systems, training and posting many skilled health professionals to rural communities, and expanding mobile networks to rural Ghana [8].

**Citation:** Osei, E.; Agyei, K.; Tlou, B.; Mashamba-Thompson, T.P. Availability and Use of Mobile Health Technology for Disease Diagnosis and Treatment Support by Health Workers in the Ashanti Region of Ghana: A Cross-Sectional Survey. *Diagnostics* **2021**, *11*, 1233. https:// doi.org/10.3390/diagnostics11071233

Academic Editor: Chao-Min Cheng

Received: 17 May 2021 Accepted: 7 June 2021 Published: 9 July 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

Digitisation of healthcare systems such as mobile health (mHealth) technologies and applications have been identified as promising strategies for improving access to healthcare delivery and patient outcomes [9,10]. Mobile health technology is defined as mobile devices, their various components, and other related technologies in healthcare delivery [11,12]. These applications have been shown to provide a cost-effective, convenient, and broadly accessible modality to implement population-level health interventions [13]. In Ghana, mobile phones' availability and utilisation as of 2018 was reported to be about 52% and is expected to increase steadily [14]. The high rate of mobile phone penetration and its innovativeness could become a promising tool to enhance healthcare provision and bridge the inequalities of healthcare accessibility [15–17]. Mobile phone adoption and acceptability are disproportionately high in resource-limited settings [18]. Thus, mHealth applications can address healthcare disparities among hard-to-reach populations to help achieve universal health for all [19].

Studies in some low- and middle-income countries (LMICs) have indicated that, in this era of SARS-CoV-2, digital health technologies such as mHealth applications have been utilised for screening, diagnosis, risk assessment, tracking of real-time transmissions, and others in all settings [20,21]. The use of mHealth applications could reduce the spread of SARS-CoV-2 and other infectious diseases in overcrowded emergency rooms and improve patient care [19–23]. With the advent of mHealth, patients with chronic diseases could be managed and treated remotely instead of visiting the hospital in-person. Others with acute disease conditions could also be screened and diagnosed remotely with these mHealth applications rather than visiting the overcrowded emergency rooms. This could minimise their risk of contracting SARS-CoV-2 and other infectious diseases in this current condition. Mobile health applications have also been deployed to support disease surveillance, medication, and treatment adherence, improve communication between clinical staff and their patients, appointment reminders, etc., [24–28].

Despite these significant challenges and the limited resources in Ghana, mHealth interventions' potential in playing a massive transformative role in healthcare provision has received considerable attention [29,30]. Considering the prospects of mHealth applications in resource-limited settings, we conducted a cross-sectional study to determine the availability and use of mHealth applications for disease diagnosis and treatment support by health workers in the Ashanti Region of Ghana. This research focused on the availability of mHealth infrastructure, clinical staff competence, mHealth for diagnostics and treatment, usefulness, ease of use, user satisfaction, and behavioural intention to use mHealth. It is envisaged that the findings of this study will be beneficial to the GoG, donors, non-governmental organisations in health, development partners, and others for improving the quality of healthcare provision by integrating mHealth applications into the normal clinical flow. It is also anticipated that our findings will assist the GoG and other similar settings to implement and sustain digital technologies such as mHealth to promote universal health coverage.

## **2. Methods**

## *2.1. Study Design and Participants*

A cross-sectional survey was conducted in primary healthcare facilities in the Ashanti Region of Ghana. The researchers conducted this survey to examine the availability and use of mHealth applications for disease diagnosis and treatment support by health professionals in the Ashanti Region of Ghana. In this survey, the participants are healthcare professionals who are highly trained clinical staff such as clinicians, nurses, laboratory scientists, pharmacists, physiotherapists, radiologists, and others mandated to provide healthcare services to the public. Healthcare professionals across 100 health facilities gave written consent to take part in this survey. A few participants were assisted in answering the questionnaire, while the majority answered them independently. All the participants were working in healthcare facilities in the Ashanti Region of Ghana during our survey.

## *2.2. Study Setting*

The Ashanti Region is located in the middle part of Ghana (Figure 1). According to the 2010 population census, this region has over 4.70 million inhabitants with a growth rate of 2.7% and is described as Ghana's business hub [31]. It is projected to reach 9.5 million inhabitants in 2040, according to the Ghana Statistical Service 2012 report [32]. This region is the most populated part of Ghana and has several healthcare facilities that cover the entire region [33]. This region is one area with a high prevalence of several communicable and non-communicable diseases in Ghana. For instance, it has the second-highest prevalence rate of non-communicable diseases such as hypertension, stroke, diabetes, cancer, and others in Ghana [34–37]. Although this area is the most populated region in Ghana, it is one of the regions with the lowest tuberculosis prevalence rates [38]. Ashanti Region is one of Ghana's numerous areas with poor healthcare access, especially for people living in poor-resource settings. There are relatively moderate levels of accessibility to general primary healthcare; accessibility to healthcare services remains deficient in several rural districts in this region [39,40]. This is primarily due to the uneven distribution of healthcare facilities since most healthcare facilities are concentrated in urban and semi-urban areas, with few in rural communities [41].

**Figure 1.** Map of Ashanti Region of Ghana.

## *2.3. Sampling Method*

We obtained a list of 530 primary healthcare facilities from the Ashanti Regional Health Directorate (RHD) of the Ghana Health Service (GHS) [33]. The researchers randomly selected 100 primary healthcare facilities from all 43 districts in the region. Because there are many healthcare facilities across the entire region, 100 healthcare facilities were chosen to ensure comprehensive study coverage. To guarantee the uniformity of sampled primary healthcare facilities in all 43 districts, the following approach was employed: the primary healthcare facilities were first stratified into 43 strata, with each stratum denoting a district in the region. The 530 primary healthcare facilities were grouped into four categories: 167 health centres, 154 clinics, 180 sub-district hospitals, and 29 district hospitals. Probability proportionate to size (PPS) was then used to determine the proportion of healthcare facilities from each stratum and category with the formula: nh = (Nh/N) × n, where nh represents the sample size for each stratum h; Nh represents population size for each stratum h; N represents the total population; and n denotes the total sample size. A purposive sampling technique was used to select all the district hospitals. Based on this, 29 hospitals were selected from Category 1, 30 clinics from Category 2, 28 clinics from Category 3, and 13 clinics from Category 4. We also used proportionate stratification to obtain the total number of primary healthcare facilities selected from the four groups in each of the 43 strata. After that, a simple random sampling technique was employed to select all the 100 healthcare facilities for this study (File S1).

## *2.4. Data Collection and Instruments*

The researchers adopted the survey tool from studies conducted by Bauer et al. (2014), Bauer et al. (2017), and Abu-Dalbouch (2013) to match our study population, settings, and study aim [42]. The cross-sectional survey tool (File S2) was piloted in eight health centres and clinics in the Ashanti Region and modified to suit the local context based on the respondents' feedback. This pilot study was conducted to ensure the validity, reliability, and precision of data and remove all forms of ambiguity from the survey tool. We collected data on the category of health professionals, type of facility, number of healthcare professionals, number of patients seen per week, available healthcare infrastructure, healthcare workforce competence, ownership of mobile wireless devices, the usefulness of mHealth, ease of use of mHealth, user satisfaction, and behavioural intention to use mHealth. Data were surveyed and collected by the researcher and three trained research assistants.

## *2.5. Ethics Statement*

This study was given full ethical clearance from the Biomedical Research Ethics Committee from the University of KwaZulu-Natal (Approval No. BREC/00000202/2019) and Ghana Health Service Ethics Review Committee (Approval No. GHS-ERC006/11/19). Regional clearance and recruitment site clearance of participants were obtained before the data collection commenced. All study participants were given written consent forms explaining the study's objective, and they signed the informed consent forms prior to their participation.

## *2.6. Outcome Measures*

The study focused on examining the availability of mHealth technologies for disease diagnosis and treatment support by health professionals in the Ashanti Region of Ghana. The analysis of this study examined two outcome measures.

The first outcome was the availability of mHealth for disease diagnosis and treatment support which stemmed from the question: "Are there mHealth interventions available in this facility to support healthcare delivery?" A binary response (yes/no) was captured.

The second outcome was the use of mHealth applications for disease diagnosis and treatment support, which stemmed from the question: "What do you use mHealth interventions for?" Responses were captured on four options: find health information, disease diagnosis, treat and manage disease conditions, and treat and monitor patients' health conditions.

## *2.7. Explanatory Variables*


## *2.8. Data Management and Analysis*

The completed questionnaires were screened and reviewed by the principal investigator to complete and correct all discrepancies. Data were then captured into a passworded excel spreadsheet. After data cleaning and verification, the data were exported into STATA version 15 which was developed by StataCorp in California, USA. Descriptive statistics such as frequencies, percentages, means, and standard deviations characterise health workers' demographics and clinical features. They were then presented in tables and others. Multivariate logistic regression was employed to explore the influence of the demographic factors on the availability of mHealth for disease diagnosis and treatment support by healthcare workers. Again, this multivariate logistic regression was also used to explore the influence of the demographic factors on the use of mHealth for disease diagnosis and treatment support by health workers. In the multivariate logistic regression model, a *p*-value of 0.05 was set as the statistical significance. Furthermore, the associations were examined using the odds ratio and 95% CI estimates. A Chi-square test at a significance level of a *p*-value of 0.05 was used to assess the relationship between the availability and the use of mHealth for disease diagnosis and treatment support.

## **3. Results**

## *3.1. Characteristics of the Study Participants*

This study received a 100% response rate from the healthcare professionals in the selected healthcare facilities in the region. Completed responses were from 285 participants, with 146 males (51.23%) and 139 females (48.77%). The results revealed that the participants aged 31–40 years were the largest share, with 48.07%, followed by those in the category of 20–30 years, with 42.11% of the responses. Participants aged 41–50 and 51–60 years were the smallest shares, with 9.47% and 0.35%, respectively. The largest group (28.7%) of the respondents in this survey were general nurses, while only 2.46% were midwives. Many (49.12%) of the respondents worked at district hospitals, 35.44% worked at health centres at the sub-district level, and 15.44% worked at rural clinics. Mobile health applications are readily available at the district hospitals, followed by the health centres and the rural clinics having poor availability. The average total number of health professionals in each healthcare facility was estimated at 57.8 (95% CI: 20–98). The average number of patients per week seen by these healthcare professionals was 175.4 (95% CI: 74–372) (Table S1).

## *3.2. Availability of Mobile Health for Diagnostics and Treatment Support in the Ashanti Region*

Results from the frequency table (Table S2) show that mobile wireless devices are available primarily to healthcare professionals with a frequency of 276 (96.84%). Mobile health applications are available with an estimated frequency of 179 (62.81%) and a nonavailability frequency of 106 (37.19%). It is also clear that phone calls are the most predominant mHealth technique being utilised by healthcare professionals, with an estimated frequency of 183 (98.92%). Short message service (SMS) is another mHealth intervention used by healthcare professionals with the second highest frequency of 149 (80.54%). Figure 2 illustrates the availability of the various mHealth applications. Again, simple mobile phones are readily available to healthcare professionals with an estimated frequency of 185 (100%), followed by smartphones with 133 (71.89%) and tablets with 107 (57.84%). It is also observed that there is a higher continuous supply of electric power with an estimated

frequency of 149 (80.54%) and relatively high available support systems of 106 (57.30%). Furthermore, most healthcare professionals have the requisite skills for diagnostics with a high frequency of 132 (71.36%) and competence for treatment procedures with an estimated frequency of 164 (88.65%).

**Figure 2.** Availability of the various mHealth applications.

## *3.3. Use of Mobile Health for Diagnostics and Treatment Support in the Ashanti Region*

The frequency table (Table S3) shows that the high frequency rate of 182 (98.38%) indicates that many healthcare professionals are currently using mHealth applications to promote healthcare delivery. In this region, healthcare professionals use mHealth to support treatment procedures of diseases such as HIV (177, 95.86%), TB (171, 92.43%), hypertension (99, 53.51%), malaria (93, 50.54%), and diabetes (79, 42.70%). Figure 3 demonstrates various diseases that are being treated and managed with mHealth applications. However, only a few healthcare professionals use mHealth to support the treatment of other conditions such as diarrhoea (17, 9.19%), cancer (5, 2.70%), chronic respiratory disease (2, 1.08%), and stroke (0, 0%). In addition, most healthcare professionals use mHealth to search for medical information (117, 63.24%), diagnosis diseases (182, 98.38%), treat and manage disease conditions (162, 87.57%), and treat and monitor patients' health conditions (144, 77.84%).

**Figure 3.** Types of diseases treated and managed with mHealth applications.

Most healthcare professionals agreed that mHealth applications are easy to use when providing healthcare services to their clients. The majority of them confirmed that mHealth applications are easy to use to support disease diagnosis with an estimated frequency of 262 (87.37%). Some other healthcare professionals also indicated that it is flexible to interact with mHealth with an estimated frequency of 273 (95.79%). The survey revealed that healthcare professionals are comfortable using mHealth to support healthcare delivery, with an estimated frequency of 266 (93.33%). In addition, others are very confident in using mHealth with an estimated frequency of 254 (89.12%). Again, some healthcare professionals are delighted with the use of mHealth with an estimated frequency of 218 (76.49%). Moreover, most healthcare professionals would use mHealth to treat and manage patients' disease conditions with a frequency of 254 (89.12%). Furthermore, others intend to use mHealth for disease diagnosis and treatment support with an estimated frequency of 279 (97.89%).

## *3.4. Availability of Health Infrastructure and Healthcare Workforce Competency*

From the multivariate logistic regression model (Table 1), the results illustrate that healthcare workers within the age groups 20–30 (OR = 17.8 (95% CI: 1.49–21.0) and 31–40 (OR = 17.6 (95% CI: 1.45–21.1) had increased odds for toll-free intervention availability when compared to healthcare workers above 40 years. In addition, healthcare workers within the age groups 20–30 and 31–40 had increased odds for mobile apps [OR = 1.46 (95% CI: 0.34–0.18)] and mHealth availability (OR = 2.93 (95% CI: 0.70–12.2) compared to those above 40 years. Male healthcare workers had increased odds for mobile apps' availability (OR = 1.27 (95% CI: 0.53–3.04), mobile wireless devices (OR = 1.26 (95% CI: 0.11–5.16), and toll-free intervention (OR = 1.02 (95% CI: 0.43–2.41) compared to female healthcare workers (Figure 4). The total number of healthcare professionals with access to available mHealth (OR = 1.16 (95% CI: 1.07–1.25) and mobile apps (OR = 1.09 (95% CI: 1.03–1.17) had increased odds. The results also indicate that the number of patients per week significantly affects mHealth intervention availability, mobile apps, and toll-free intervention. As expected, an increase in the number of patients per week increased the odds of mHealth intervention availability (OR = 1.02 (95% CI: 1.01–1.04) and mobile apps (OR = 1.00 (95% CI: 1.00–1.01) to healthcare workers. However, an increase in patients' number reduced the odds of toll-free intervention availability (OR = 0.97 (95% CI: 0.96–0.98).

**Figure 4.** Odds ratio showing the association on the availability of mobile wireless devices and mHealth applications for disease diagnosis and treatment support by health workers in Ashanti region, Ghana.




*p*-value < 0.05: \*\*\* *p*-value < 0.001. Source: Author's computation based on data obtained from the field survey,

**Table 1.** *Cont.*

Furthermore, the results show that health professionals such as doctors and pharmacists significantly influenced the requisite skills for diagnostics and competence to use mHealth for treatment support. Doctors had increased odds for the requisite skills for diagnostics (OR = 1.065 (95% CI: 0.45–2.55) and competence to use mHealth for treatment support (OR = 1.153 (95% CI: 0.27–4.88) as compared to laboratory scientists. Pharmacists had increased odds for disease diagnosis requisite skills (OR = 1.243 (95% CI: 0.56–2.71) compared to laboratory scientists. The results also illustrate those district hospitals and health centres significantly affect the supply of power and support systems.

In addition, district hospitals increased the odds for the supply of power (OR = 59.87 (95% CI: 70.06–5117) and support systems (OR =159.7 (95% CI: 4.51–5660) compared to clinics. However, district hospitals had decreased odds (OR = 0.63 (95% CI: 0.11–0.35) for the competence to use mHealth for treatment support. Health centres had increased odds for the supply of power (OR = 53.53 (95% CI: 5.45–525) and support systems (OR =10.68 (95% CI: 1.05–108) compared to clinics. The total number of healthcare professionals with access to smartphones (OR = 1.073 (95% CI: 1.02–1.12) and competence to use mHealth for treatment support (OR = 1.196 (95% CI: 1.09–1.31) had increased odds. However, the total number of healthcare workers with access to power supply (OR = 0.907 (95% CI: 0.85–0.96) and support systems (OR = 0.948 (95% CI: 0.91–0.89) had decreased in odds. Again, an increase in the number of patients per week increased odds for healthcare workers' competence to use mHealth for treatment support (OR = 1.019 (95% CI: 1.01–1.03) (Figure S1).

## *3.5. Use of mHealth for Diagnostics and Treatment Support*

The multivariate model (Table 2) results show that healthcare workers within the age group 20–30 had increased odds for using mHealth to support the treatment of hypertension (OR = 2.28 (95% CI: 0.74–7.05), diabetes (OR = 3.75 (95% CI: 0.96–14.6), cancer (OR = 6.05 (95% CI: 0.01–5.85), and malaria (OR = 1.04 (95% CI: 0.36–3.05) compared to healthcare workers above 40 years. In addition, healthcare workers within the age group 31–40 had increased odds for using mHealth to manage hypertension (OR = 2.12 (95% CI: 0.67–6.68), diabetes (OR = 5.75 (95% CI: 1.43–23.1), cancer (OR = 11.1 (95% CI: 0.01–12.0), and malaria (OR = 1.24 (95% CI: 0.42–3.67) as compared to healthcare workers above 40 years. Being a male healthcare professional raised the odds for the use mHealth to manage HIV (OR = 2.47 (95% CI: 0.37–16.4) and TB (OR = 1.94 (95% CI: 0.49–7.62) compared to being a female healthcare professional. Both medical doctors and nurses had increased odds (OR = 1.66 (95% CI: 0.30–9.16) and (OR = 1.28 (95% CI: 0.28–5.83), respectively for the use of mHealth to manage TB when compared to laboratory scientists (Figure 5).

**Figure 5.** Odds ratio showing the association on the use of mHealth applications for the management and treatment of HIV and TB conditions by health workers in Ashanti region, Ghana.


**2.**Multivariate analysis results for the use of mHealth for diagnostics and treatment

 support.

**Table** 

*Diagnostics* **2021** , *11*, 1233

The results further show that healthcare workers within the age group 20–30 had increased odds for the use of mHealth for disease treatment (OR = 3.05 (95% CI: 0.58–15.9) and using mHealth once a month for diagnostics [OR = 2.16 (95% CI: 0.55–8.55)] and treatment support (OR = 1.06 (95% CI: 0.25–4.43) compared to those above 40 years. In addition, healthcare professionals within the age group 31–40 had a rise in odds for the use of mHealth for disease treatment (OR = 7.25 (95% CI: 1.32–39.9) and using mHealth once a month for diagnostics (OR = 3.64 (95% CI: 0.68–19.3) and treatment support (OR = 2.68 (95% CI: 0.67–10.7). Being a male healthcare worker increased the odds for using mHealth to treat diseases (OR = 1.48 (95% CI: 0.52–4.17), monitor patients' conditions (OR = 1.22 (95% CI: 0.57–2.59), and mHealth one to six times a week for diagnostics (OR = 1.73 (95% CI: 0.85–3.48) and treatment support (OR = 2.33 (95% CI: 1.03–5.24) compared to being a female healthcare worker.

Medical doctors had decreased odds of using mHealth once a month for treatment support compared to laboratory scientists (OR = 0.38 (95% CI: 0.15–0.19). Again, pharmacists had increased odds for using mHealth application one to six times a week to support treatment (OR = 2.07 (95% CI: 0.98–4.35) compared to laboratory scientists. District hospital increased the odds for the use of mHealth for disease treatment (OR = 1.70 (95% CI: 0.02–13.4) and monitor patients' conditions (OR = 1.60 (95% CI: 0.05–55.6) compared to clinics. In addition, health centre had increased odds for the use of mHealth for disease treatment (OR = 3.96 (95% CI: 0.23–68.5) and monitoring patients' conditions (OR = 1.41 (95% CI: 0.20–9.98) when to compared to clinics. As expected, a rise in the number of patients per week increased odds for using mHealth one to six times for diagnostics (OR = 1.01 (95% CI: 0.99–1.01) and treatment support by healthcare workers (OR = 1.01 (95% CI: 1.00–1.01). However, an increase in the number of patients decreased the odds for using mHealth to treat diseases (OR = 0.99 (95% CI: 0.98–0.99) (Figure S2).

## *3.6. Usefulness of mHealth Interventions*

The results from the multivariate model (Table 3) suggest that healthcare professionals within the age group 20–30 had reduced odds for the use of mHealth to monitor patients' disease conditions (OR = 0.15 (95% CI: 0.02–1.07), manage communicable diseases (OR = 0.15 (95% CI: 0.02–1.07), and provide reminders for medication adherence (OR = 0.32 (95% CI: 0.08–1.24) compared to those above 40 years. In addition, healthcare workers within the age group 31–40 had increased odds for the use of mHealth to manage non-communicable diseases (OR = 1.23, (95% CI: 0.54–2.81) and communicable diseases (OR = 1.41(95% CI: 0.54–3.82) as compared to healthcare professionals above 40 years. However, healthcare professionals within the age group 31–40 had reduced odds for the use of mHealth as reminders for the treatment adherence procedures (OR = 0.41(95% CI: 0.17–0.99) when compared to those above 40 years. Male healthcare professionals had increased odds to use mHealth to monitor patients' disease conditions (OR = 1.76 (95% CI: 0.80–3.85), manage communicable diseases (OR = 1.19 (95% CI: 0.72–2.00), manage noncommunicable diseases (OR = 1.19 (95% CI: 0.72–2.10), and as reminders for medication adherence OR = 1.31 (95% CI: 0.67–2.54) when compared to female healthcare professionals.

Medical doctors had three-fold increased odds of using mHealth as reminders for medication adherence compared with laboratory scientists (OR = 3.32 (95% CI: 1.38–7.97). District hospital reduced the odds for the use of mHealth to monitor patients' disease conditions (OR = 0.41 (95% CI: 0.01–0.78) and as reminders for clinic appointments (OR = 0.18 (95% CI: 0.01–1.02) when compared to clinics. Health centre increased the odds for the use of mHealth to manage communicable diseases as compared to clinics (OR = 1.16 (95% CI: 0.46–2.90). The total number of healthcare professionals who use mHealth as reminders for treatment adherence procedures (OR = 1.04 (95% CI: 1.01–1.08) and clinic appointments (OR = 1.03 (95% CI: 1.00–1.07) had increased odds. A rise in the number of patients per week increased the odds for the use of mHealth to monitor patients' disease conditions (OR = 1.01 (95% CI: 1.00–1.02) and manage communicable diseases (OR = 1.00 (95% CI: 0.99–1.00).




*Diagnostics* **2021** , *11*, 1233

**Table 3.** *Cont.*

The results further indicate that healthcare workers within the age group 20–30 had reduced the odds for the use of mHealth for follow-ups (OR = 0.24 (95% CI: 0.07–0.77), test result notifications (OR = 0.35 (95% CI: 0.13–0.95) and making accurate diagnostic decisions (OR = 0.14 (95% CI: 0.01–0.78) compared with those above 40 years. Again, healthcare professionals within the age group 31–40 had increased odds for using mHealth as reminders for drug collection (OR = 1.43 (95% CI: 0.33–6.09) compared with other healthcare workers above 40 years. Male healthcare professionals had increased odds for using mHealth for follow-ups (OR = 1.56 (95% CI: 0.88–2.76) and treating and managing disease conditions (OR = 1.49 (95% CI: 0.83–2.67) when compared to female healthcare professionals. Both medical doctors and nurses had two-fold increased odds of using mHealth to make accurate diagnostic decisions (OR = 2.77 (95% CI: 0.92–8.33), treat and manage disease conditions (OR = 2.67 (95% CI: 1.23–5.77), and increase effectiveness to treat and manage diseases (OR = 2.10 (95% CI: 0.76–5.83) compared with laboratory scientists.

District hospitals increased the odds for mHealth to treat and manage disease conditions than clinics (OR = 1.36 (95% CI: 0.11–16.8). However, as a district hospital, the odds of using mHealth as reminders to collect drugs reduced (OR = 0.24 (95% CI: 0.001–0.28). In addition, a health centre increased the odds for the use of mHealth to notify patients of their test results (OR = 2.39 (95% CI: 0.95–6.03), treat and manage disease conditions (OR = 3.52 (95% CI: 1.28–9.69), and increase the effectiveness for treatment and management of diseases (OR = 3.88 (95% CI: 1.02–14.7) as compared to clinics. The total number of healthcare professionals who use mHealth as reminders for drug collection had increased odds (OR = 1.05 (95% CI: 1.01–1.10). An increase in the number of patients per week increased the odds for the use of mHealth for follow-ups (OR = 1.00 (95% CI: 0.99–1.01) and to increase the effectiveness to treat and manage diseases (OR = 1.00 (95% CI: 1.00–1.01).

## *3.7. Ease of Use of mHealth Interventions*

In the multivariate logistic regression model (Table 4), the results demonstrate that healthcare professionals within the age groups 20–30 and 31–40 had increased odds for the flexibility to interact with mHealth devices (OR = 1.16 (95% CI: 0.11–11.8) and easy to use mHealth for treatment support (OR = 1.33 (95% CI: 0.17–10.3) compared to those above 40 years. Being a male healthcare worker increased the odds for mHealth being easy to use for disease diagnosis (OR = 1.71 (95% CI: 0.67–4.29) and having the flexibility to interact with mHealth (OR = 4.00 (95% CI: 0.76–20.9) compared to being a female healthcare professional. Medical doctors had nine-fold increased odds of becoming skilful in using mHealth for disease diagnosis and treatment support (OR = 9.56 (95% CI: 1.78–51.1) compared to laboratory scientists. Again, nurses had two-fold increased odds for mHealth being easy to use for disease diagnosis (OR = 2.66 (95% CI: 0.82–8.62) compared to laboratory scientists.

In addition, district hospital had increased the odds for mHealth being easy to use for disease diagnosis (OR = 14.0 (95% CI: 0.16–11.8) and treatment support (OR = 6.69 (95% CI: 0.02–21.4) compared to clinics. Health centres had increased odds for it being easy to learn how to use mHealth devices (OR = 1.32 (95% CI: 1.79–8.65) and become skilful in using such applications for disease diagnosis and treatment support (OR = 1.32 (95% CI: 0.60–24.3). The total number of healthcare professionals increased the odds for flexibly interacting with mHealth devices for disease diagnosis and treatment support (OR = 1.13 (95% CI: 1.00–1.27). A rise in the number of patients per week increased the odds for easily using mHealth for disease diagnosis (OR = 1.00 (95% CI: 0.99–1.04).

## *3.8. User Satisfaction of mHealth Interventions*

The results from the multivariate model (Table 5) show that healthcare workers within the age groups 20–30 and 31–40 had reduced odds for healthcare workers' confidence in using mHealth for disease diagnosis and treatment support (OR = 0.24 (95% CI: 0.04–1.24) and mHealth increasing the quality of healthcare delivery (OR = 0.18 (95% CI: 0.02–2.07) compared to those above 40 years. Being a male healthcare professional increased the odds of healthcare workers' comfort (OR = 1.84 (95% CI: 0.65–5.19) and confidence (OR = 2.33 (95% CI: 1.00–5.43) in using mHealth for disease diagnosis and treatment support compared to being female healthcare professional.


**Table 4.**

Multivariate

 analysis results for the ease of use of mHealth, user satisfaction

 of mHealth, and behavioural

 intention to use mHealth

interventions.

\*\*: *p*-value < 0.05: \*\*\* *p*-value < 0.001. Source: Author's computation

 based on data obtained from the field survey, 2020.


Analysis Results for the User Satisfaction of mHealth Interventions.

> **Table 5.**

Multivariate


**Table 5.** *Cont.*

Again, medical doctors had increased odds of becoming comfortable using mHealth applications for disease diagnosis and treatment support (OR = 1.02 (95% CI: 0.28–3.80) compared to being a laboratory scientist. Again, nurses had increased odds of feeling comfortable with mHealth (OR = 1.06 (95% CI: 0.29–3.80) and improving the quality of healthcare delivery with mHealth (OR = 2.10 (95% CI: 0.55–7.98) when compared to being laboratory scientists. Health centres had increased odds of healthcare workers' comfort (OR = 3.84 (95% CI: 0.87–17.8) and confidence (OR = 3.87 (95% CI: 1.13–13.2) with the use of mHealth for disease diagnosis and treatment support compared to clinics. An increase in the number of patients per week increased the odds of using mHealth to improve healthcare delivery quality (OR = 1.01 (95% CI: 0.99–1.02).

## *3.9. Behavioural Intention to Use mHealth Interventions*

Results from the multivariate model (Table 5) reveal that healthcare professionals within the age groups 20–30 and 31–40 had increased odds for healthcare professionals intending to use mHealth for the treatment (OR = 0.13 (95% CI: 0.02–0.92) and management of patients' disease conditions (OR = 0.35 (95% CI: 0.05–2.38) compared to those above 40 years. Being a male healthcare professional increased the odds of healthcare workers' intention to use mHealth for treating and managing patients' disease conditions (OR = 2.79 (95% CI: 1.19–6.54) and disease diagnosis and treatment support (OR = 1.97 (95% CI: 1.08–3.60) compared to being female healthcare professional.

In addition, pharmacists had increased odds of healthcare workers' intention to use mHealth to treat and manage patients' disease conditions compared to laboratory scientists (OR = 1.45 (95% CI: 0.55–3.82). The odds increased for a district hospital where healthcare workers intend to use mHealth (OR = 2.25 (95% CI: 0.15–32.7) and would always use mHealth for disease diagnosis and treatment support (OR = 3.20 (95% CI: 0.05–24.0) compared to clinics. A rise in the number of patients per week increased the odds for healthcare workers using mHealth to treat and manage patients' disease conditions (OR = 1.00 (95% CI: 0.99–1.01) and their intention to use mHealth for disease diagnosis and treatment support (OR = 1.01 (95% CI: 0.99–1.04).

## *3.10. Association between Health Infrastructure Availability or Healthcare Workforce Competency and Ownership of Mobile Wireless Devices*

A cross-sectional tabulation was done between healthcare infrastructure's availability or healthcare workforce competency and ownership of mobile wireless devices using a chi-square test (Table S4). The chi-square test results illustrate a significant relationship between mobile wireless devices' availability and currently using mHealth to support healthcare provision (*p* < 0.05). Healthcare workers with mobile wireless devices were more likely to use mHealth to support healthcare delivery than those without mobile wireless devices. In addition, the association between mobile wireless devices' availability and their use to assist malaria conditions' treatment is statistically significant (*p* < 0.05). Healthcare workers with mobile wireless devices were more likely to use these devices to treat malaria conditions than those without mobile wireless devices.

Moreover, the chi-square test results also show a significant association between mHealth intervention availability and its use to manage malaria conditions (*p* < 0.05). Healthcare professionals with mHealth were more likely to use such interventions to support malaria management than those without mHealth. The results further illustrate a significant relationship between short message services (SMS) and their use to manage hypertension cases (*p* < 0.05). Healthcare workers who stipulated that they have SMS applications were more likely to use such intervention to manage hypertension conditions than those without SMS services. In addition, the chi-square test results suggest a significant relationship between mobile apps and their use to manage TB (*p* < 0.05), diabetes (*p* < 0.05), and disease diagnosis (*p* < 0.05). Healthcare professionals who indicated that they have mobile apps were more likely to use them for diagnosing diseases and managing TB and diabetes conditions than others with no mobile apps. The chi-square test results demonstrate a significant association between toll-free lines and their usage for managing

TB (*p* < 0.05) and HIV (*p* < 0.05) conditions. Healthcare workers who suggested that they have toll-free lines were more likely to use this intervention to support the treatment of TB and HIV conditions than others without toll-free lines.

## *3.11. Association between Health Infrastructure Availability or Healthcare Workforce Competency and Usefulness of mHealth Applications*

A cross-sectional tabulation was performed between healthcare infrastructure's availability or healthcare workforce competency and the usefulness of mHealth using a chisquare test (Table S5). The chi-square test results suggest a significant relationship between mobile wireless devices' availability and managing non-communicable diseases (NCDs) (*p* < 0.05), communicable diseases (*p* < 0.05), reminders for treatment adherence procedures (*p* < 0.05), clinic appointments (*p* < 0.05), follow-ups (*p* < 0.05), and treating and managing diseases (*p* < 0.05) to support healthcare provision. Healthcare workers with mobile wireless devices were more likely to use these devices to manage communicable and non-communicable diseases, as reminders for treatment adherence procedures, and for clinic appointments and follow-ups than those without mobile wireless devices.

The chi-square test results also show a significant association between the availability of mHealth intervention and its use as reminders for treatment adherence procedures (*p* < 0.05), clinic appointments (*p* < 0.05), follow-ups (*p* < 0.05), and test result notifications (*p* < 0.05) to promote healthcare delivery. Healthcare professionals who indicated that they have mHealth interventions were more likely to use these interventions as reminders for treatment adherence procedures, clinic appointments, follow-ups, and test result notifications than those with no mHealth interventions. The results further illustrate a significant relationship between SMS and their use to manage NCDs (*p* < 0.05) and follow-ups to promote treatment compliance (*p* < 0.05). Healthcare workers with SMS interventions were more likely to use such interventions to manage NCDs and follow-ups than those without mHealth.

## *3.12. Association between Health Infrastructure Availability or Healthcare Workforce Competency and Ease of use of mHealth Applications*

A cross-sectional tabulation was done between the availability of healthcare infrastructure or healthcare workforce competency and ease of use of mHealth applications using a chi-square test (Table S6). The chi-square test results reveal a significant relationship between mobile wireless devices' availability and the ease of using mHealth for disease diagnosis (*p* < 0.05) and treatment support (*p* < 0.05) and its flexibility (*p* < 0.05) to support healthcare services. Healthcare workers who indicated they have mobile wireless devices were more likely to find it easier and more flexible to use them for disease diagnosis and treatment support than those without mobile wireless devices. In addition, the chi-square test results show a significant association between the availability of mHealth intervention and the ease of using mHealth for treatment support (*p* < 0.05) and its flexibility (*p* < 0.05) to promote healthcare delivery. Healthcare professionals with mHealth were more likely to find it easier and flexible to use these mHealth interventions to support patients' disease diagnosis and treatment conditions than others with no mHealth.

The results further show a significant relationship between SMS and its ease of using mHealth for treatment support (*p* < 0.05) and its flexibility (*p* < 0.05) to enhance the provision of quality healthcare. Healthcare professionals with SMS interventions were more likely to find it easier and flexible to use such interventions for disease diagnosis and treatment support than those without mHealth. In addition, the chi-square test results show a significant relationship between phone calls and its ease of using mHealth for disease diagnosis (*p* < 0.05) and treatment support (*p* < 0.05), its flexibility (*p* < 0.05), becoming skilful in using mHealth (*p* < 0.05), and it being easy to learn how to use mHealth (*p* < 0.05). Healthcare workers who indicated that they use phone/voice call interventions were more likely to find it easier and flexible to use such applications for disease diagnosis and treatment support than healthcare workers without access to voice calls.

## *3.13. Association between Health Infrastructure Availability or Healthcare Workforce Competency and User Satisfaction of mHealth*

A cross-sectional tabulation was done between healthcare infrastructure's availability or healthcare workforce competency and user satisfaction of mHealth using a chi-square test (Table S7). The chi-square test results show a significant relationship between mobile wireless devices' availability and confidence (*p* < 0.05) and being completely satisfied (*p* < 0.05) using mHealth for disease diagnosis and treatment support. Healthcare professionals who were confident and completely satisfied with mHealth were more likely to use these mobile wireless devices for disease diagnosis and treatment support than those with no confidence in mHealth. In addition, the chi-square test results illustrate a significant association between the availability of mHealth intervention and comfort with mHealth (*p* < 0.05), confidence in mHealth (*p* < 0.05), and increase quality healthcare (*p* < 0.05). Healthcare professionals with mHealth interventions who were comfortable and confident in mHealth were more likely to use such applications to boost quality healthcare delivery than others with no mHealth interventions.

The results further show a significant relationship between SMS and completely satisfied with mHealth applications (*p* < 0.05). Healthcare workers with SMS applications were more likely to be happy with mHealth for disease diagnosis and treatment support than others with no SMS application access. In addition, the chi-square test results indicate a significant association between phone calls and their comfortability (*p* < 0.05) and increase quality healthcare (*p* < 0.05). Healthcare professionals who suggested using phone call interventions were more likely to be comfortable using mHealth to boost quality healthcare delivery than those without access to voice calls. Again, the chi-square test results illustrate a significant relationship between mobile apps and completely satisfied using mHealth applications (*p* < 0.05). Healthcare workers with mobile apps were more likely to be happy with mHealth for disease diagnosis and treatment support than others with no mobile apps. The chi-square test results reveal a significant association between toll-free intervention and comfort using mHealth (*p* < 0.05). Healthcare professionals who have access to toll-free lines were more likely to be comfortable using mHealth than those without toll-free lines.

## *3.14. Association between Health Infrastructure Availability or Healthcare Workforce Competency and Behavioural Intention to use mHealth*

A cross-sectional tabulation was performed between the availability of healthcare infrastructure or healthcare workforce competency and behavioural intention to use mHealth using a chi-square test (Table S7). The chi-square test results illustrate a significant relationship between mobile wireless devices' availability and always using mHealth for disease diagnosis and treatment support (*p* < 0.05). Healthcare professionals with mobile wireless devices were more likely to use mHealth for disease diagnosis and treatment support than others with no mobile wireless devices. Additionally, the chi-square test found a significant association between mHealth intervention availability and always using mHealth for disease diagnosis and treatment support (*p* < 0.05). Healthcare workers with mHealth interventions were more likely to use mHealth for disease diagnosis than others with no mHealth interventions.

Furthermore, the results also show a significant relationship between SMS and the ability to use mHealth to treat and manage patients' conditions (*p* < 0.05). Healthcare professionals with SMS interventions were more likely to use mHealth to treat and manage patients' needs than others with no SMS intervention access. The chi-square test found a significant association between phone calls and intention to use mHealth for disease diagnosis and treatment support (*p* < 0.05). Healthcare workers who use phone call interventions intended to use mHealth for disease diagnosis and treatment support than those with no access to voice calls. Again, the chi-square test results demonstrate a significant relationship between mobile apps and always use mHealth for disease diagnosis and treatment support (*p* < 0.05). Healthcare professionals with mobile apps were more likely to use mHealth for disease diagnosis and treatment support than those without mobile apps.

## **4. Discussion**

This study aimed to examine the availability and use of mHealth applications for disease diagnosis and treatment support by healthcare workers in Ghana. In this study, 64.91% of healthcare professionals indicated that mHealth applications are available to them, while 35.08% do not have access to mHealth. In addition, 98.38% of healthcare professionals are currently using mHealth applications to support healthcare delivery. The findings show that mobile wireless devices such as simple mobile phones, smartphones, and tablets are readily available to healthcare professionals in Ghana's Ashanti Region. The results also reveal that mHealth applications such as phone or voice calls, SMS, mobile apps, and toll-free lines are available to healthcare workers and are currently being used to support healthcare delivery. The results further illustrate that healthcare professionals predominantly use mHealth applications to screen or diagnose many existing disease conditions in this region.

Additionally, the results demonstrate that healthcare workers in this part of Ghana currently use mHealth to treat HIV, TB, hypertension, diabetes, malaria, and diarrhoea conditions. Again, the results reveal that healthcare professionals continuously use mHealth to support healthcare provision due to the constant supply of power. Moreover, the findings suggest that most healthcare professionals have the requisite skills and competence in using mHealth applications for diagnostics and treatment procedures of disease conditions. Furthermore, the results demonstrate a low-level use of mHealth applications for disease diagnosis and treatment support by healthcare professionals at rural clinics.

A study conducted in the USA largely agrees with this current survey where healthcare workers use mHealth to treat and manage chronic diseases such as HIV, TB, hypertension, and diabetes, among others [42]. This current survey results fully support the findings from similar surveys conducted in primary care clinics in the USA [43]. In their studies, most healthcare workers were comfortable and confident in using mHealth applications to support their patients' healthcare needs [43]. The findings demonstrate that healthcare workers use mHealth applications to promote medication adherence, clinic appointments, and follow-ups. This corroborates with the findings from a similar study conducted by Belcher et al. in Saudi Arabia, where mHealth applications improved the treatment of diabetes, clinic appointments, and check-ups [44].

This current study's limitations include that respondents' inclusion was based on patient consent, which may have introduced selection bias into the study sample. Due to the limited funding for the data collection, only 285 participants were enrolled from this region's numerous primary healthcare clinics. Our current results may not be generalised beyond the Ashanti Region of Ghana among healthcare professionals using mHealth for disease diagnosis and treatment support. Despite all these limitations, our current study is, to the best of our knowledge, the first comprehensive research on the availability and use of mHealth for disease diagnosis and treatment support by healthcare professionals in the Ashanti Region of Ghana. The study helped determine the current availability and use of mHealth applications by healthcare professionals to diagnose and treat diseases in this region. This could guide policymakers in formulating guidelines on the utilisation of mHealth technologies to promote quality healthcare delivery.

This current study achieved its primary objective and demonstrated a gap in mHealth for disease diagnosis and treatment support at rural clinics in Ghana's Ashanti Region. This means that policymakers and implementors should adopt various strategies to facilitate the implementation of mHealth applications for disease diagnosis and treatment support in such resource-constrained settings and enhance their scale-ups. Given this, we recommend a proposed framework for improving the implementation of mHealth for disease diagnosis and treatment support in low- and middle-income countries (LMICs) [12]. The results show that mHealth applications are generally available to healthcare professionals and are being utilised for disease diagnosis and treatment support of patients' conditions. This is a good sign that the continuous use of mHealth should be strengthened to promote quality healthcare delivery as recommended by the World Health Organisation (WHO) 2019 guidelines on digital health [45].

The results demonstrate a low-level use of mHealth applications for disease diagnosis and treatment support by healthcare professionals at rural clinics. To this end, we encourage policymakers to deliberately implement mHealth at rural clinics to support disease diagnosis and treatment procedures of patients' conditions. The findings show that healthcare professionals employed mHealth to treat diseases such as HIV, TB, hypertension, and diabetes. We recommend that more primary studies be conducted focused on using mHealth to treat and manage other diseases such as cancer, stroke, chronic respiratory conditions, asthma, and others in this region. Moreover, the findings indicate that most healthcare professionals use mHealth applications to screen or diagnose several common disease conditions in this region. Hence, we encourage healthcare professionals to use mHealth interventions to screen or diagnose several other neglected tropical diseases to enhance early detection to initiate proper treatment processes.

## **5. Conclusions**

The study revealed that mHealth applications are primarily available to healthcare professionals to promote quality healthcare delivery in the Ashanti Region. The findings show that healthcare professionals use mHealth applications to screen or diagnose, treat, and manage several common disease conditions at primary healthcare clinics. The results also demonstrate a low-level use of mHealth applications for disease diagnosis and treatment support by healthcare professionals at rural clinics. Future studies are recommended to examine the availability and use of mHealth applications for disease diagnosis and treatment support by healthcare professionals at rural clinics.

**Supplementary Materials:** The following are available online at https://www.mdpi.com/article/ 10.3390/diagnostics11071233/s1, File S1: Distribution of primary healthcare facilities sampled in the Ashanti Region, File S2: Survey tool, Table S1: Characteristics of participants from the 100 healthcare facilities surveyed in Ashanti Region, Table S2: Availability of mobile health for diagnostics and treatment support in the Ashanti Region, Table S3: Use of mobile health for diagnostics and treatment support in the Ashanti Region, Table S4: Chi-square test results of the relationship between the available health infrastructure or healthcare workforce competency and ownership of mobile wireless devices, Table S5: Chi-square test results of the relationship between the available health infrastructure or healthcare workforce competency and usefulness of mHealth applications, Table S6: Chi-square test results of the relationship between the available health infrastructure or healthcare workforce competency and ease of use of mHealth applications, Table S7: Chi-square test results of the relationship between the available health infrastructure or healthcare workforce competency and user satisfaction and behavioural intention to use mHealth, Figure S1: Odds ratio showing the association on the availability of mobile apps, toll-free, supply of power, support systems and others for disease diagnosis and treatment support by health workers in Ashanti Region, Ghana, Figure S2: Odds ratio showing the association on the use of mHealth applications for the management and treatment of hypertension, diabetes, cancer, malaria, monitor patients' conditions and others by health workers in Ashanti Region, Ghana.

**Author Contributions:** E.O. and T.P.M.-T. conceptualised the study and developed the analytical strategy. E.O. collected the data, processed the data, performed the statistical analysis, interpreted the results, and wrote the first draft of the study. K.A. and B.T. contributed to the statistical analysis and interpretation of the results. T.P.M.-T. contributed to the analytical strategy and the interpretation of the results and made critical revisions. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** The study was conducted according to the guidelines of the Declaration of Helsinki and approved by the Biomedical Research Ethics Committee of the University of KwaZulu-Natal on 4 November 2019 (Approval No. BREC/00000202/2019) and Ghana Health Service Ethics Review Committee on 13 December 2019 (Approval No. GHS-ERC006/11/19).

**Informed Consent Statement:** Informed consent was obtained from all the participants involved in the study.

**Data Availability Statement:** Data for this study are the property of the University of KwaZulu-Natal and can be made available publicly. All interested persons can access the dataset from the author Ernest Osei via this email address (ernestosei56@gmail.com) and the University of KwaZulu-Natal Biomedical Research Ethics Committee (BREC) using the following contacts: The Chairperson Biomedical Research Ethics Administration Research Office, Westville Campus, Govan Mbeki Building University of KwaZulu-Natal P/Bag X54001, Durban, 4000 KwaZulu-Natal, South Africa (Tel.: +27-31260-4769; Fax: +27-31260-4609; Email: BREC@ukzn.ac.za).

**Acknowledgments:** We thank the University of KwaZulu-Natal for providing us with essential research resources during this study. The authors are also grateful to all the staff members at the primary healthcare facilities who participated in this study. The authors would like to thank the Authorities of the Ashanti Regional Health Directorate, the District Health Management Teams, and all the PHCs managers for permitting us to conduct this study. Finally, we are grateful to the Department of Public Health Medicine staff for their support in diverse ways.

**Conflicts of Interest:** The authors declare no conflict of interest.

## **References**


## *Review* **A Scoping Review of Supply Chain Management Systems for Point of Care Diagnostic Services: Optimising COVID-19 Testing Capacity in Resource-Limited Settings**

**Kuhlula Maluleke 1,\*, Alfred Musekiwa 1, Kabelo Kgarosi 2, Emily Mac Gregor 3, Thobeka Dlangalala 1, Sphamandla Nkambule <sup>4</sup> and Tivani Mashamba-Thompson <sup>5</sup>**


**Abstract:** Background: Point of care (POC) testing has enabled rapid coronavirus disease 2019 (COVID-19) diagnosis in resource-limited settings with limited laboratory infrastructure and high disease burden. However, the accessibility of the tests is not optimal in these settings. This scoping review mapped evidence on supply chain management (SCM) systems for POC diagnostic services to reveal evidence that can help guide future research and inform the improved implementation of SARS-CoV-2 POC diagnostics in resource-limited settings. Methodology: This scoping review was guided by an adapted version of the Arksey and O'Malley methodological framework. We searched the following electronic databases: Medline Ovid, Medline EBSCO, Scopus, PubMed, PsychInfo, Web of Science and EBSCOHost. We also searched grey literature in the form of dissertations/theses, conference proceedings, websites of international organisations such as the World Health Organisation and government reports. A search summary table was used to test the efficacy of the search strategy. The quality of the included studies was appraised using the mixed method appraisal tool (MMAT) version 2018. Results: We retrieved 1206 articles (databases *n* = 1192, grey literature *n* = 14). Of these, 31 articles were included following abstract and full-text screening. Fifteen were primary studies conducted in LMICs, and 16 were reviews. The following themes emerged from the included articles: availability and accessibility of POC diagnostic services; reasons for stockouts of POC diagnostic tests (procurement, storage, distribution, inventory management and quality assurance) and human resources capacity in POC diagnostic services. Of the 31 eligible articles, 15 underwent methodological quality appraisal with scores between 90% and 100%. Conclusions: Our findings revealed limited published research on SCM systems for POC diagnostic services globally. We recommend primary studies aimed at investigating the barriers and enablers of SCM systems for POC diagnostic services for highly infectious pathogens such SARS-CoV-2 in high disease-burdened settings with limited laboratory infrastructures.

**Keywords:** point of care diagnostic services; supply chain management; COVID-19; resourcelimited settings

## **1. Background**

The primary goal of severe acute respiratory syndrome coronavirus type 2 (SARS-CoV-2) testing is to reduce the spread of coronavirus disease 2019 (COVID-19) [1,2]. Due to

**Citation:** Maluleke, K.; Musekiwa, A.; Kgarosi, K.; Gregor, E.M.; Dlangalala, T.; Nkambule, S.; Mashamba-Thompson, T. A Scoping Review of Supply Chain Management Systems for Point of Care Diagnostic Services: Optimising COVID-19 Testing Capacity in Resource-Limited Settings. *Diagnostics* **2021**, *11*, 2299. https:// doi.org/10.3390/diagnostics11122299

Academic Editor: Chao-Min Cheng

Received: 15 October 2021 Accepted: 17 November 2021 Published: 8 December 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

the highly infectious nature of SARS-CoV-2, there is an urgent need for a fast turnaround of results to institute preventative measures such as the isolation of confirmed cases and contact tracing [1,3]. Currently, reverse transcription polymerase chain reaction (RT-PCR) tests are the gold standard for diagnosing COVID-19 [4–6]. The laboratory equipment required to perform RT-PCR is often lacking in resource-limited settings, hindering the fast and accurate detection of SARS-CoV-2 [7,8].

To ease the burden on health facilities and laboratory services, alternative diagnostic methods such as point of care (POC) testing may improve the disease diagnosis [7,8]. POC testing refers to diagnostic testing that enables near-patient disease diagnosis to inform clinical decisions [9]. The benefits of POC tests are numerous, including affordability, ease of use and able to be deployed both at the site of triage and outside healthcare facilities to guide disease management [4,10]. POC tests deliver prompt results; therefore, they are of utmost importance in containing highly infectious diseases such as COVID-19 [10].

The WHO recommended scaling up testing programmes for SARS-CoV-2 by testing all suspected cases [1,10]. This recommendation was prompted by a resurgence of COVID-19 and the limited testing capacity in settings that have poor access to laboratory infrastructures [11]. The use of POC testing would significantly increase the testing capacity and allow for more accurate reporting and management of SARS-CoV-2. Rapid antigen tests also allow for the decentralisation of SARS-CoV-2 testing, thus increasing testing coverage, which may allow policymakers to institute effective adaptive policy responses [1]. To ensure the equitable availability and accessibility of POC tests, efficient supply chain management (SCM) is necessary. Supply chain refers to resources and processes needed to deliver goods and services to consumers with complete satisfaction in a cost-optimized manner [12,13]. SCM is a multifaceted system that involves production, selection, quantification, procurement, storage, distribution, redistribution, quality assurance and inventory management [10,14].

Evidence in supply chain systems for POC diagnostics is not clear nor readily available. This scoping review is aimed at mapping the evidence of SCM systems of all existing POC diagnostic services in order to reveal gaps to guide future research. It is also anticipated that the results of this review will help guide POC diagnostics implementers in implementing sustainable SCM for POC diagnostics to help manage highly infectious pathogens such as SARS CoV-2 in resource-limited settings. For the purposes of this study, resource-limited settings are defined as settings characterised with having limited access to laboratory infrastructures and limited capability to provide care for life-threatening illness and limited basic critical care resources.

## **2. Methodology**

This scoping review was conducted as part of a multi-phase PhD study aimed at developing a novel approach for improving the SCM of SARS-CoV-2 POC diagnostic services in resource-limited settings. This scoping review protocol was registered with the Open Science Framework (OSF) under the title: "A Scoping Review Protocol for supply chain management systems for point of care diagnostics services: Optimising COVID-19 testing capacity in resource-limited settings" (File S1). The published methodology was made available on 19 September 2021 for public comments via the link below: https: //doi.org/10.17605/OSF.IO/RHGEF.

This review was guided by the methodological framework proposed by Arksey and O'Malley 2005 [15] and further advanced by Levac et al., 2010 [16]. According to this framework, we conducted the review in the following five stages: (i) identify the research question; (ii) identify relevant studies; (iii) select eligible studies; (iv) charting the data and (v) collating, summarising and reporting the results. Arksey and O'Malley 2005 proposed a sixth optional stage comprising consultations with key stakeholders to provide insights beyond those found in the literature [15]. This scoping review did not include consultations with stakeholders.

The results of the scoping review were presented according to the Preferred Reporting Items for Systematic Reviews and Meta-Analysis Extension for Scoping Reviews (PRISMA-ScR) [17].

## *2.1. Identification of the Research Question*

The research question for this study was: What is the evidence in SCM systems for POC diagnostics services globally? To determine the eligibility of the proposed research question for a scoping review, we used the Population, Concept and Context (PCC) framework, as depicted in Table 1.


**Table 1.** PCC framework for determining the eligibility of the research question.

## *2.2. Identification of Relevant Studies*

We conducted a comprehensive and reproducible literature search using the following electronic databases: Medline Ovid, Medline Elton B. Stephens Company (EBSCO), Scopus, PubMed, PsychInfo, Web of Science and EBSCOHost. We also searched for grey literature, including dissertations/theses, conference proceedings, websites of international organisations such as WHO and government reports. We identified additional relevant studies by manually searching all references cited in the included studies to identify studies not indexed in electronic databases. Language restrictions were not applied to minimise the risk of excluding relevant studies.

This comprehensive search strategy was codeveloped by the principal investigator (PI), subject specialist and university librarian to ensure the correct use of indexing terminology and Medical Subject Headings (MeSH) terms. The following keywords or MeSH terms were used: (1) "supply chain management" or "supply chain" or "supply chain flow" or "supply chain systems", (2) "point of care" or "point of care testing" or "point of care diagnosis" or "point of care diagnostic services" and (3) "SARS-CoV-2" or "COVID-19" or "Coronavirus". The keywords were refined to suit each database. Each search was documented in detail, showing the keywords/MeSH terms, date of search, electronic database and number of retrieved studies, and the results of the search were tabulated in File S2.

The search strategy was optimised by adopting the search summary table (SST) outlined by Bethel et al. [19] as a guide. The SST was used to improve and report on the effectiveness of the search strategy.

## *2.3. Selection of Eligible Articles*

This scoping review was guided by inclusion and exclusion criteria to ensure the correct identification and selection of relevant articles.

## 2.3.1. Inclusion Criteria

The included articles met the following criteria:


## 2.3.2. Exclusion Criteria

Articles were excluded from the scoping review if they had the following characteristics:


## *2.4. Selection of Sources of Evidence*

The articles were screened in three stages, namely title, abstract and full-article screening. Reviewers used a screening tool (File S3) developed by the PI and pilot tested by the reviewers. The eligible articles were exported to an Endnote 20 library, and the duplicates were removed. The PI screened abstracts in parallel with the co-reviewer (EM). After screening the abstracts, the reviewers discussed any discrepancies in the selected articles until a consensus was reached. Two reviewers (KM and EM) then screened the full texts of articles selected during the first stage. A third screener (TD) resolved any discrepancies in the selected articles after full-text screening. Both abstract and full-article screening were guided by the screening tool that factored all aspects of the inclusion/exclusion criteria and the PCC elements.

The level of agreement between screeners' results after screening the abstracts and full articles was determined by calculating Cohen's kappa statistics. The kappa statistics were interpreted as follows: values <0.1 indicate no agreement and 0.10–0.20 indicate none to slight, 0.21–0.40 as fair, 0.41–0.60 as moderate, 0.61–0.80 as substantial and 0.81–1.00 as almost perfect agreement.

## *2.5. Charting the Data*

Data were captured from each included article using a data charting form. Two independent reviewers (KM and TD) piloted the data charting form and recommended modifications that were implemented. The following data were extracted from the included articles: author and year of publication, title of study, aim of study, country, study design, study setting, study population, type of point-of-care test investigated, stage of SCM investigated, main findings and other significant findings.

## *2.6. Collating, Summarizing and Reporting the Results*

We thematically analysed the data extracted from the included articles. The themes were narratively summarised. The following themes emerged from the data: availability and accessibility of POC diagnostic services; reasons for stockouts of POC diagnostic tests (procurement, storage, distribution, inventory management and quality assurance) and human resources capacity in POC diagnostic services.

## *2.7. Quality Appraisal*

We used the mixed method appraisal tool (MMAT) version 2018 to evaluate the quality of the included articles [20]. Using MMAT, we appraised the methodological quality of five categories of research: qualitative research, randomised controlled trials, nonrandomised studies, quantitative descriptive studies and mixed methods studies [20]. Two independent reviewers (KM and TD) carried out the quality appraisal process. The following percentage scores were used to grade the quality of evidence: (i) ≤50% represented low-quality evidence, (ii) 51–75% represented average-quality evidence and (iii) 76–100% represented high-quality evidence.

## *2.8. Ethical Considerations*

This scoping review relied on a synthesis of the existing literature, and therefore, ethical approval was not required.

## **3. Results**

## *3.1. Screening Results*

Our screening search involved title screening and returned a total of 882 results, which consisted of 868 articles and 14 articles from grey literature (Figure 1). Following the removal of 735 duplicate articles, 147 remained. The 147 articles were included in abstract screening. During abstract screening, 47 articles met the inclusion criteria for full-article screening, and 100 were excluded. Of these 47 articles that were included in full-article screening, 22 did not meet the inclusion criteria, and they were excluded. The data were extracted from the remaining 25 articles. The reasons for excluding the 22 articles included were as follows: 12 articles focused on laboratory-based POC diagnostic services, 3 articles did not discuss any aspect of the SCM system, 4 articles focused on the SCM of health systems and 3 articles focused on pharmaceutical and hospital pharmacy SCM. An additional search was performed on 28 October 2021, and it returned 324 articles. Based on the title screening, 29 articles were found eligible and screened for abstracts. After abstract screening, we excluded 14 articles, and 15 articles met the inclusion criteria for full-article screening. Of these, we excluded 9 articles. Additional data were extracted from the remaining six articles. The reasons for excluding the nine articles included: three articles focused on laboratory-based POC tests, and six articles focused on evaluating the performances of the POC tests. After full-text screening, there was a high agreement of 96.77% vs. 50% expected by chance (Kappa statistics = 0.93, *p* < 0.05). In addition, McNamar's chi-square statistics suggested no significant differences in the proportions of yes/no answers by the reviewers (*p* > 0.05) (File S4). The search strategy was continuously improved. An updated search (rerun) was essential, because SARS-CoV-2 is a novel virus, and new research is published frequently (File S5).

**Figure 1.** PRISMA-ScR flow chart showing the literature search and selection of articles.

## *3.2. Characteristics of the Included Articles*

The characteristics of the included articles are summarised in Table 2. The eligible studies were published between 2011 and 2021. The 31 articles included 15 reviews [10,12,21–32], 1 website of an international organisation [33], 2 randomised controlled trials [34,35], 1 cross-sectional study [36], 1 case study [37], 1 cohort study [38] and 10 studies that used mixed method approaches ranging from focus group discussions to interviews to direct observations [39–48]. The included articles presented evidence of research conducted in the following countries: Mozambique [34,36,44], Zimbabwe [49], Ghana [12,21,22,40,45,48], Malawi [42], Namibia [41], India [43], Sierra Leone [37], Burkina Faso [38], The Philippines [38], Senegal [38,47], Ethiopia [38], Uganda [39,46], Brazil [39], Peru [39], China [39], Tanzania [39] and Zambia [35,39]. Figure 2 shows the distribution of eligible articles by country. Peru, Brazil, China, Namibia, The Philippines and India are middle-income countries, and Mozambique, Ghana, Malawi, Sierra Leone, Burkina Faso, Senegal, Ethiopia, Uganda, Tanzania, Zimbabwe and Zambia are low-income countries. Two articles were conducted in more than one study setting. A cohort study by Albertini et al. was conducted in Senegal, Burkina Faso, Ethiopia and The Philippines [38]. A mixed methods qualitative study by Mabey et al. was conducted in Peru, Brazil, China, Uganda, Zambia and Tanzania [39].

**Figure 2.** World map showing the distribution of the articles included in the scoping review on supply chain management systems for point-of-care diagnostic services.



**Table 2.** *Cont.*


by building a sustainable supply chain

logistics.


diagnostics

smartly and

efficiently.





pneumonia.



**Table 2.** *Cont.*


**Table 2.** *Cont.*


**Table 2.** *Cont.*


**Table 2.** *Cont.*

Evidence on the following POC diagnostic tests were provided: COVID-19 [27, 28,33]; HIV [10,12,22–26,34,36,42,49]; malaria [10,12,22,23,34,35,38,40,41,43,44,46–48]; syphilis [10,12,23,32,34,39,42,45]; tuberculosis [30,31] and others comprising of anaemia [34,40], UTI [37], diabetes [12,31], blood glucose [22], preeclampsia urinalysis [37], urine pregnancy [22], urine protein [22,34], haemoglobin [22], dyslipidaemia [12], dengue [30], meningitis [29], hepatitis C [29] and hepatitis B [12,22,29–31]. Figure 3 shows the types of POC diagnostic tests in the included articles. Malaria POC diagnostic tests were investigated in 14 articles, HIV POC diagnostic tests in 11 articles, syphilis POC diagnostic tests in 8 articles, COVID-19 POC tests in 3 articles, TB POC tests in 2 articles and the other POC diagnostic tests were investigated in 12 articles.

**Figure 3.** Distribution of the types of POC diagnostic tests in the articles included in the scoping review of supply chain management systems for point-of-care diagnostic services.

The eligible studies provided evidence on at least one aspect of the SCM system. The aspects of the SCM systems covered were procurement and stockouts [10,12,21,24–26,33, 34,36,41–45,47,49], quality assurance [10,23,24,27,29,31,32,37,46,48,49], safe disposal [46], inventory management [10,12,21,25,35,36], selection [10,22,28,33,46], distribution [12,21, 22,34,38,41,48], quantification [12], storage [12,21,38,41,48], human resource capacity [30,35, 40,47] and the availability and accessibility of POC diagnostic services [10,12,23,32,39,47,48]. Figure 4 shows the aspects of the SCM systems described in the included articles.

**Figure 4.** Aspects of supply chain management systems in the articles included in the scoping review on supply chain management systems for point-of-care diagnostic services.

## *3.3. Quality of Evidence*

From the 31 included articles, 15 primary studies were appraised for methodological quality using MMAT version 2018 [20]. The remaining 10 articles were excluded because they were not primary studies [10,12,21–26,32,49]. After categorising each study, we rated the criteria of the chosen categories to establish whether the criteria for each study design had been met. The 15 studies that were appraised for methodological quality scored between 90% and 100%, which showed high methodological quality. Twelve studies scored the highest quality score of 100% [34–36,39–45,47,48]. One study scored 95% [38]. The remaining two studies scored 90% and 92% [37,46] (File S6).

## *3.4. Main Findings*

All the included articles presented evidence of at least one aspect of SCM of POC diagnostic services globally. The following themes emerged from the included articles: availability and accessibility of POC diagnostic services, reasons for stockouts of POC diagnostic tests (procurement, storage, distribution, inventory management and quality assurance) and human resources capacity in POC diagnostic services.

## 3.4.1. Accessibility and Availability of POC Diagnostics Services

Eight articles reported evidence on the availability and accessibility of POC diagnostic services [10,12,23,27,34,39,47,48]. These articles showed that the accessibility and availability of POC diagnostic services in resource-limited PHC clinics contributed to improved health outcomes [12]. A review conducted by Peeling [23] showed that, in Peru, it would take five to six visits spread over 27 days to screen and treat pregnant women for syphilis. With the introduction of rapid diagnostic tests (RDTs), the testing and treatment of infected mothers was done on the same day [23]. A randomised controlled trial conducted in Mozambique found that introducing POC diagnostic testing for pregnant women improved antenatal care [34]. A significant improvement in the accessibility of antenatal POC diagnostic tests was reported in all implementation sites. In the first visits, 5519 (14.6%) out of 37,826 women were screened for anaemia compared to 30,057 (97.7%) out of 30,772 in the intervention, 3739 (9.9%) out of 37,826 women were screened for proteinuria during the control period compared with 29,874 (97.1%) out of 30,772 in the intervention period and 17,926 (51.4%) out of 34,842 received mebendazole during the control period compared with 24,960 (88.2%) out of 28,294 during the intervention period [34].

In a survey conducted by Poole et al., 2021 highlighting the current state of RDTs globally, most of the respondents reported 24/7 availability of tests [29]. High-income countries reported a higher availability of the rapid influenza test compared to low-income countries [29]. Low-income countries reported a high availability of HIV and hepatitis rapid tests. The reason given for this pattern is that hepatitis and HIV are more prevalent in developing countries, where public health interventions are less likely to identify and treat patients early during illness [29]. Another reason is that the priorities for treatment are different: influenza management in secondary care is a less-pressing need in resource-limited settings where patient isolation facilities are less readily available [29]. Additionally, the cost of identifying an influenza case is less than HIV or hepatitis, where early identification and treatment make a greater difference. The cost of each test is likely to also be a factor in the difference of availability, with multiplexed assays generally being considerably more expensive and requiring more complex logistical support. Methods for reducing the costs of many RDTs are lacking, which limits their availability in low-income settings [29].

Decentralisation was an important factor promoting the accessibility and availability of POC diagnostic services. The decentralisation of POC diagnostic services increased detection, patient management and prompt treatment initiation when necessary [23,32,39]. Mabey et al., 2012 reported on the introduction of POC testing for syphilis in various urban (China and Peru) and rural settings (villages in East Africa). The proportion of antenatal care attendees increased by more 90% in areas that previously had some testing [39]. In areas where syphilis POC diagnostic testing had previously not been available, the

screening rate increased by more than 50% [39]. In all settings, more than 90% of those who tested positive received syphilis treatment [39].

In most of the studies, POC diagnostic services relied solely on government supplies [35,37,39,41–43,45]. Two articles reported that the supply of POC diagnostic tests was insufficient and sporadic [12,48]. This affected malaria testing in a study conducted in Ghana, which determined the factors that influenced the implementation of POC diagnostic tests in routine malaria management at PHC facilities [48]. At the time of the study, four study sites had limited malaria RDTs, while two sites had none [48]. As RDTs became less available, the rate of blind treatment increased. Other patients opted to buy test kits privately, causing financial strain [48]. The RDT supplies were interrupted due to frequent and prolonged RDT stockouts [48].

## 3.4.2. Reasons for Stockouts of POC Diagnostic Tests

The reviewed articles revealed that stockouts of POC diagnostic tests were caused by problems in procurement, inventory management, storage, distribution and quality assurance.

## • Procurement of POC diagnostic tests

Sixteen articles reported stockouts due to procurement issues [10,12,21,24–26,34,36,40– 45,47,49]. Of the 16 articles, one article reported an adequate inventory of test kits more than 89% of the time across the 75 facilities from a study conducted in Mozambique [36]. In this study, the HIV RDT stock levels were well-maintained due to technical support received from a nongovernment organisation affiliated with the Global Fund [36]. The stock levels were also monitored during monthly visits and followed up with planning and coordinating supply chain logistics with health facilities [36].

The other 15 articles reported poor procurement processes. Two qualitative studies conducted by Magesa et al., 2019 and Maddox, et al., 2017 reported that more than 60% of the key informants from all levels of the supply chain reported stockouts due to a lack of proper knowledge and training in procurement processes [41,42]. A randomised control trial conducted by Betrán et al. [34] reported that, during the intervention period, two clinics had a single 3-day period with no kits due to clinics making late requisitions to the health directorate [34]. According to Hussain, Dandona, David and Schellenberg [43], malaria diagnoses were compromised by a failure of the supply chain due to poor communication or procurement systems compromising the availability of POC tests at PHC facilities. Almost half (48%) of the facilities did not have test kits. Similar findings were reported by Hasselback et al. [44], where 59% of health facilities reported stockouts due to poor communication between health facilities and suppliers [44]. This resulted in a high rate of referral to private healthcare facilities, which hindered the early diagnosis and complete treatment of malaria [43].

The WHO, in a document that aims to bring clarity on the process of requesting and receiving globally sourced COVID-19 critical diagnostics supplies, stated that low- and middle-income countries have continued to experience restrictions in test access due to competition for limited volumes with high-income countries [33]. Manufacturers have also faced challenges in scaling up manufacturing to meet all testing needs. Prices for diagnostic products remain high, and some national governments continue to face restricted access to tests. To access POC tests, purchasers may place orders directly with the companies or utilize one of the available multilateral procurement channels. The funded demand and requests are being followed closely to determine whether these tests may be constrained in volume availability. If they become constrained, an allocation model using the same principles as above will be implemented to ensure equitable distribution [33].

• Inventory management of POC diagnostic tests

Consumption data was important to guide the quantity of inventories procured at all implementation sites [47]. Adequate inventory was promoted by a well-managed and monitored system to enable clear communication between PHC facilities and regional or provincial offices [24,41,45]. Lack of consumption data contributed to stockouts of malaria RDTs as a results of a poorly functioning Facility Electronic Stock Card (FESC) in Namibian PHC facilities [41]. A lack of consumption data resulted in a high rate of expired medicine and wastage of malaria RDTs due to overstock, while other study sites experienced stockouts due to underestimated consumption [41]. In their review, Peeling et al., 2015 concluded that real-time data monitoring via electronic readers improved inventory management [23]. Operational data on stocks, device usage and conditions were uploaded via Wi-Fi or cellular networks and transmitted to central databases [23]. Linking the data to SCM software helped avoid stockouts and improved the efficiency of the health system [23].

In Mozambique, 15 health facilities were surveyed over 120 time points [44]. Stockout patterns varied by data source, with an average of 59% of health centres reporting stockouts on stock cards every month, preventing the proper documentation of consumption data [44]. Each ten-unit increase in monthly-observed consumption was associated with a nineunit increase in lost consumption, indicating higher rates of stockouts at higher levels of observed consumption [44]. Stockouts were caused by the inaccurate tracking of lost consumption, insufficient sophistication in inventory management and replenishment and poor process compliance by facility workers, all stemming from inadequate attention to design and implementation of the inventory management system [44].

• Storage of POC diagnostic tests

Six articles mentioned that RDTs should be stored under recommended temperatures, because exposure to adverse environmental conditions had the potential to degrade POC diagnostic tests [12,21,37,38,41,48]. A study investigating storage and temperature conditions in Burkina Faso, Senegal and Ethiopia revealed that malaria RDTs were being stored at temperatures exceeding the recommended RDT manufacturer temperature limit [38]. In three of the eight facilities in Burkina Faso, temperatures rose above the recommended RDT manufacturer temperature limit of 40 ◦C [38]. In 11 of the 13 facilities in Ethiopia, temperatures exceeded the recommended RDT manufacturer temperature limit of 30 ◦C. In five out of ten facilities in Senegal, temperatures exceeded the recommended RDT manufactured limit of 40 ◦C [38]. In this study, RDTs were exposed to unfavourable conditions for only brief periods, making it difficult to detect if product shelf lives were shortened [38]. When RDTs are exposed to extreme conditions for some time, it is costly to retest a withdrawn batch, and the continuity of diagnostic services is disrupted [38,46]. Ways to lower temperatures were explored in Uganda [46]. This was done by implementing techniques such as underground storage and the use of evaporative cooler boxes. These studies concluded that RDTs should be selected according to the expected field conditions [38,46].

An association between the limited storage capacity contributed and stockouts was reported in Namibia [41]. Most clinical settings (*n* = 16, 94%) reported problems with storage resulting in the expiration of diagnostic tests. Stock was stored in the wrong place, and staff were not always aware that there was stock. Some study sites would pile up boxes on the floor, compromising the qualities of the diagnostic tests [41].

Evidence has reported the robustness of syphilis rapid tests, because they do not require special storage or transport conditions [32]. Syphilis tests can be stored at temperatures ranging from 8 to 30 ◦C [32]. If clinics are warmer than 30 ◦C, the shelf life of rapid tests is reduced, and sensitivity is lost [32]. In settings that rely on POC diagnostic tests, it is important to conduct periodic quality control checks to ensure ongoing validity. Health authorities should visit study sites to ensure that POC diagnostic tests are functioning as expected [32].

In Ghana, malaria RDTs are not always distributed timeously, and there is limited storage space at the district office, resulting in RDTs being left outdoors, exposed to sunlight and other weather conditions [48]. Providers were concerned with compromised test qualities due to poor storage, and they had little confidence in the accuracy of the test results [48].

## • Distribution of POC diagnostic tests

Nine articles discussed the distribution of POC diagnostic tests and highlighted that regional or national health directorates were responsible for distributing POC diagnostic tests to PHC clinics [12,21,22,34,38,41,48]. The timely distribution of POC diagnostic tests to PHC clinics often depends on the availability of tests at the regional or national health directorate. In Ghana, Kuupiel, Tlou, Bawontuo, Drain and Mashamba-Thompson [22] found that 90% of the PHC clinics received POC diagnostic supplies within 24 h after requisition. While the distribution of POC tests was adequately managed, PHC staff revealed documentation challenges that would limit their ability to forecast demands. They further recommended that PHC staff require training on the documentation of test stock levels to aid forecasting demands to ensure the continued supply of diagnostic tests to match consumption [22].

In Ghana, healthcare providers reported their perspectives and experiences in the rollout of rapid POC tests for antenatal syphilis screening [45]. Almost half (6/13) of the facilities that had started antenatal syphilis screening did not have any syphilis test kits, while HIV test kits were available in all the 14 facilities that were screening pregnant women for HIV [45]. In some facilities, stockouts of syphilis test kits were quite frequent, but pregnant women were not referred to other facilities for testing and were only screened when the test kits became available [45]. In one region, healthcare providers blamed stockouts of test kits on inadequate regional supplies, while the regional staff blamed stockouts on the lack of returns from districts and healthcare facilities and, less commonly, insufficient supplies from national headquarters [45].

Mozambique experiences rainy seasons, which are associated with a high prevalence of malaria and distribution difficulties. Rural areas are not easily accessed due to poor road infrastructure [44]. Large trucks cannot drive on wet dirt roads sometimes, causing motorcycles to be the primary mode of distribution, which does not allow for the same protection as trucks [37]. The use of malaria RDTs increases by more than 300% in the rainy season; therefore, it is important to plan the distribution aspects to prevent stockouts during such a critical season [44].

• Quality assurance of POC diagnostic tests

Eight articles discussed the importance of following quality assurance processes when handling POC diagnostic tests [10,23,24,32,37,46,48,49]. These articles showed that poor regulatory mechanisms further limited RDT implementation. Quality assurance processes were seldom in place, preventing the ongoing monitoring of POC diagnostic tests. Infrequent quality assurance and control visits to facilities by authorities further undermined providers' willingness to use RDTs [10,23,24,32,37,46,48,49].

Evidence revealed that the current setup of RDTs appears to be more laboratorycentred. Governance and quality control were reported to be the responsibility of laboratories in the vast majority of those surveyed [29]. A total of 90% of those who responded to the survey reported that tests were carried out in their institution by laboratory staff [29]. Simpler tests, RDTs, were conducted more during point-of-care. While there are several existing international regulatory processes for drugs and medications, providing safeguards for their safety and efficacy, they are often lacking for RDTs [29]. As a result, diagnostic tests are often sold and used in the developing world without any evidence of effectiveness. For example, a study conducted to evaluate the performance of RDTs by Mak et al. [26] reported the sensitivity of an RDT for SARS-CoV-2 of 11.1–45.7% when the manufacturer had claimed it was 98%. This is indicative of poor-quality assurance mechanisms in the sites surveyed.

## 3.4.3. Human Resource Capacity in POC Diagnostic Services

Four articles discussed the importance of human capacity in POC diagnostic services [35,37,40,47]. Two studies reviewed how the services of community healthcare workers (CHWs) are used to assisting in providing POC diagnostic services in resource-limited

settings. In Sierra Leone, Ekambaram, Gomanie and Mehta [37] reported that CHWs had no formal training and that human error contributed to losing POC tests [37]. In Senegal, Blanas et al. [47] reported that most lay health workers acquired important skills, but few did not understand the RDT algorithm soon after the training. Although the scores improved by 10–20% after two months of field training, half of the CHWs still could not interpret the RDT algorithm correctly, and almost half could not prescribe artemisinin-based combination therapy (ACT) correctly [47].

Hamer et al. [35] evaluated the ability of CHWs to provide high-quality, safe, integrated care for malaria and pneumonia in two rural districts in Zambia. Community health workers were able to manage malaria using RDTs at the community level. The CHWs performed RDTs with 90% accuracy and with 93% correctness after a 3-h training session assisted by visual job aids and RDT package inserts. With enough training, CHWs were able to handle RDTs without significant risks to themselves and their patients, reducing wastage linked to the improper use of RDTs [35]. In Zambia, CHWs also contributed to successful SCM by keeping accurate records and engaging with supervisors to ensure adequate supplies [35].

A review conducted by Kumar et al. [30] further emphasised the importance of human resource capacity in the improvement of diagnostic accuracy and precision. They reported that appropriate sample handling approaches are important in reducing errors during sampling, testing and interpretation of the POC test. For an infectious disease diagnosis, the sample could take different forms, such as urine, serum, blood, plasma, stool or saliva. The different physical properties and chemical compositions of these samples demand proper and appropriate approaches that can accommodate the target analyte in an acceptable form [30]. The Lancet commission also found that the reliability of the diagnostic test depends not just on the performance characteristics of the actual test but on all the elements involved in the testing—for example, sample collection and preparation (PALM), result interpretation and result communication [31]. All these elements rely on users being adequately trained and having ongoing access to quality control materials and technical support and maintenance for instruments. It is therefore pivotal to incorporate adequate quality assurance and quality control into the point-of-care testing protocols. Additionally, these point-of-care diagnostics should only be used in situations in which there are referral pathways and there is healthcare provider buy-in and patient trust [31].

## **4. Discussion**

We conducted a scoping review to map the evidence on SCM systems for POC diagnostic services with the goal of optimising the COVID-19 testing capacity in resource-limited settings. Our scoping review results show that there is limited published research on SCM systems of POC diagnostic services. Studies have been conducted in sixteen low- and middle-income countries. This is a major public health concern and requires immediate action from all relevant stakeholders, especially since 47% of the global population has little to no access to diagnostic services [31]. The COVID-19 pandemic has emphasised the crucial role of diagnostics in healthcare and that, without access to diagnostics, the delivery of universal health coverage, antimicrobial resistance mitigation and pandemic preparedness cannot be achieved, thereby hampering the progress towards achieving sustainable development goal (SDG) 3 that aims to promote good health and well-being for all by 2030 [50]. Our scoping review findings also demonstrate that, for the continuum of POC diagnostic services, POC diagnostic tests must be available and accessible to all who need them through sustainable SCM systems [34,43,47,48]. Sustainable SCM systems are influenced by several factors, some of which are procurement, inventory management, storage, distribution, quality assurance and the human resource capacity, as revealed in most of the study settings explored [34–36,38,39,41,46,47]. Weak SCM systems may lead to significant stockouts of POC diagnostic tests [21,36,43]. Stockouts result in the reduced use of POC diagnostic tests in resource-limited settings, which negatively impacts health outcomes [35,39,45].

The WHO has been the main driver in ensuring that POC diagnostic tests are available in resource-limited settings through their prequalification program and bulk procurement systems [14,32]. The prequalification program facilitates access to medicines and medical equipment that meets unified standards of quality, safety and efficacy for HIV/AIDS, malaria and tuberculosis, with the aim of reducing widespread disease in countries with limited access [14,32]. Currently, the Foundation for Innovative New Diagnostics (FIND), in collaboration with WHO and other organisations, is playing a pivotal role in scaling up the development and delivery of COVID-19 tests through its Access to COVID-19 Tools (ACT) Accelerator and provide sustainable solutions beyond the COVID-19 pandemic [51]. To provide sustainable solutions in providing POC diagnostic tests and to avoid past uncoordinated procurement issues, various stakeholders have joined forces to speed up the end of the pandemic by supporting the development and equitable distribution of the tests, treatments and vaccines the world needs to reduce mortality and severe disease, restoring full societal and economic activity globally in the near term, and facilitating a high-level control of COVID-19 in the medium term [51]. The collaboration supports the coordinated, uninterrupted provision of timely, high-quality diagnostic tests in resourcelimited settings [26]. The collaboration serves as a platform for information exchange, as well as alignment, on procurement principles, planning and addressing key SCM issues [26]. They also ensure that there are adequate funds to purchase projected POC diagnostic tests to make them available for use at the POC in a timeous manner.

Effective SCM is dependent on the skilled human resource capacity for POC diagnostic services [21]. Staff in PHCs should be trained in various aspects of POC diagnostic test SCM [21]. Staff should also be trained in how to perform POC diagnostic testing accurately to prevent wastage [21]. Training is also important to aid in stock management and the safe disposal of used test kits to ensure personnel and environmental safety [21]. Training programs complemented with strengthened monitoring and supervision support at the clinics may ensure compliance with SCM guidelines, and acceptable standards will further enable the suitability of diagnostic services.

Storage, distribution and inventory management are equally important in the SCM system. Regional or provincial warehouses should have enough stock to ensure adequate distribution to all PHC facilities. Regional warehouses and PHC storage facilities need to meet storage requirements. While distributing POC diagnostic tests, harsh environmental conditions should be avoided by using refrigerators and cold chain facilities [21]. Improved inventory management will ensure that enough tests are available to meet demand. Health facilities should have set minimum and maximum levels and follow the First Expired, First Out principle, thus preventing tests from expiring. Inventory management linking national, regional, district and PHC facilities could potentially benefit from the adoption of digital technology, such as blockchain. Blockchain is an emerging digital technology that has unique characteristics, such as immutability, decentralisation and transparency, that can be used for the coordination of large-scale operations such as population-level mass screening, rapid contact tracing and supply chain management [52]. This modern technology is widely used in high-income countries [53]. In resource-limited settings, the adoption of blockchain digital technology may be costly; therefore, low-cost blockchain may be more sustainable [54].

The adoption of real-time technology such as the Stock Visibility System (SVS) can bring visibility to the supply chain process, make it seamless, and can facilitate effective and efficient inventory management at all levels [55]. The SVS facilitates the detection of high- and low-consumption PHC clinics to enable the redistribution of tests to prevent the overstocking and expiration of POC tests [55]. The distribution of SARS-CoV-2 POC diagnostic tests can be optimised by using unmanned aerial vehicles (UAVs) or drones that can be operated either autonomously or remotely by humans [56]. The use of drones has been extensively adopted for the faster and safe transfer of essential products, assisting authorities to deal with and possibly overcome the constraints and health emergencies imposed on society by the COVID-19 pandemic [57,58]. Drones have been shown to prevent the spread of coronavirus infection by limiting person-to-person contact and stopping the unwanted movement of people during the lockdown [58]. Their independence on road infrastructure and remote operations enables them to be a viable option for various supply chains in hard-to-reach settings [56]. The use of drone technology has been proven to be effective in the delivery of much-needed medical supplies to Rwanda, Lesotho and Ghana's rural hospitals [59–61]. Drone technology has also been shown to be effective in inventory management more efficiently than humans through the movement of items, locating specific inventory, surveying large areas and inspecting labels while saving time and eliminating human error [56]. In addition to diagnostic supplies to remote and hardto-reach settings, the concept of using inventory drones can be adapted and implemented to help improve the efficiency in monitoring stock levels of POC diagnostic tests at storage facilities. As with other health technologies, the effective and sustainable implementation of this technology will need to be context-specific and include the involvement of all key stakeholders.

## *4.1. Implications for Practice*

All the primary studies included in this scoping review were conducted in resourcelimited settings in LMICs, as these settings have been reported to have weak health systems and limited coverage or access to diagnostic services. Resource-limited settings have serious SCM barriers. Stockouts of POC tests are a major challenge, and most PHC facilities do not procure POC tests on their own but rely on making requisitions at the district or national level. Distribution to PHC facilities is normally delayed, negatively affecting health outcomes, because patients cannot be tested in a timely manner. A sustainable, wellmanaged SCM system is important in resource-limited settings, and digital technology may be a viable option. The following SCM measures can be implemented to ensure sustainability: strengthen procurement systems, appropriate forecasting, the efficient training of SCM and health personnel, equitable and timely distribution, efficient inventory management and adequate quality assurance.

## *4.2. Implications for Research*

We found limited published research on the SCM of POC diagnostic services in resource-limited settings. Our scoping review revealed that there are currently no primary studies on the SCM systems of COVID-19 POC diagnostic services. We therefore recommend further research to investigate the COVID-19 SCM of POC diagnostic services, explore SCM systems utilised in high-income countries with the aim of adopting sustainable solutions for LMICs and systematic analyses of the impacts of the COVID-19 pandemic on SCM systems. We also recommend primary studies investigating the barriers and enablers of SCM systems in resource-limited settings to provide sustainable solutions to SCM challenges.

## *4.3. Strengths and Limitations*

This scoping review is one of the few to comprehensively map the evidence of SCM systems for POC diagnostics services globally with the aim of optimising COVID-19 POC diagnostic services. This scoping review revealed significant gaps in the literature on SCM systems of COVID-19 POC diagnostic services. The use of a scoping review to map evidence allowed the incorporation of different study designs, and we used a transparent and reproducible method to identify, chart, analyse and appraise the articles [15]. The strength of this scoping review is that a comprehensive literature search in relevant electronic databases was conducted. The search included all articles published from inception to date, and no language restrictions were applied. In order to be as comprehensive as possible, the scoping review utilised many keywords and used Medical Subject Heading terms. Despite attempts to be as comprehensive as possible, other published and grey literature may have been missed during the literature search, because COVID-19 is a novel virus, and more research is being published frequently.

This scoping review mapped evidence on the SCM of POC diagnostic services while optimising the COVID-19 testing capacity in resource-limited settings; however, there were no primary studies retrieved that assessed the SCM of COVID-19 POC diagnostic services. It is recommended that primary studies to investigate the SCM of COVID-19 diagnostic services in resource-limited settings are conducted.

## **5. Conclusions**

This scoping review revealed that there is limited research on the SCM of POC diagnostic testing in resource-limited settings. POC diagnostic services are fundamental in the timely control and management of COVID-19. The supply chain of POC diagnostic testing should be optimised to ensure access to all patients. The adoption of digital technology can play a crucial role in mitigating the SCM challenges arising from the COVID-19 pandemic. There is currently no research focusing on the SCM of POC tests in the control and management of COVID-19. Therefore, there is an urgent need to conduct primary studies that will investigate the SCM for SARS-CoV-2 POC testing services in order to reveal the research gaps and provide evidence-based solutions for policymakers and implementers of this service.

**Supplementary Materials:** The following are available online at https://www.mdpi.com/article/ 10.3390/diagnostics11122299/s1: File S1: Scoping Review Protocol. File S2: Electronic database search results. File S3: The screening tool. File S4: Full article screening results and agreement. File S5: Search summary table. File S6: MMAT quality appraisal.

**Author Contributions:** K.M. conceptualised and wrote the draft manuscript under the supervision of T.M.-T. and A.M. S.N. contributed to the development of the search summary table. K.K. optimised the search strategy and performed the literature search. K.M., E.M.G. and T.D. contributed to the title and abstract screening and full-text screening of the eligible articles. T.M.-T. and A.M. critically reviewed and provided input to revise the manuscript. All the authors contributed to the reviewed draft version of the manuscript and approved the final version. All authors have read and agreed to the published version of the manuscript.

**Funding:** This publication was made possible by funding from UNICEF and with support from Future Africa, the University of Pretoria.

**Institutional Review Board Statement:** Ethics approval is not applicable for this scoping review protocol.

**Informed Consent Statement:** Not applicable.

**Data Availability Statement:** All data supporting the conclusions of this scoping review are available through a detailed reference list. The original datasets were not presented, because this scoping review used existing literature.

**Acknowledgments:** The authors would also like to extend their appreciation to Cheryl Tosh for editing and Ninety-One SA (Pty) Ltd. for providing the resources to assist in conducting this scoping review.

**Conflicts of Interest:** The authors declare that they have no competing interests.

## **Abbreviations**


## **References**


## *Article* **Multi-Level Stakeholder Perspectives on Determinants of Point of Care Ultrasound Implementation in a US Academic Medical Center**

**Anna M. Maw 1,\*, Megan A. Morris 2, Juliana G. Barnard 2, Juliana Wilson 3, Russell E. Glasgow 4, Amy G. Huebschmann 5, Nilam J. Soni 6, Michelle Fleshner 1, John Kaufman <sup>1</sup> and P. Michael Ho <sup>7</sup>**


**Abstract:** There is growing interest from multiple specialties, including internal medicine, to incorporate diagnostic point of care ultrasound (POCUS) into standard clinical care. However, few internists currently use POCUS. The objective of this study was to understand the current determinants of POCUS adoption at both the health system and clinician level at a U.S. academic medical center from the perspective of multi-level stakeholders. We performed semi-structured interviews of multi-level stakeholders including hospitalists, subspecialists, and hospital leaders at an academic medical center in the U.S. Questions regarding the determinants of POCUS adoption were asked of study participants. Using the framework method, team-based analysis of interview transcripts were guided by the contextual domains of the Practical Robust Implementation and Sustainability Model (PRISM). Thirty-one stakeholders with diverse roles in POCUS adoption were interviewed. Analysis of interviews revealed three overarching themes that stakeholders considered important to adoption by clinicians and health systems: clinical impact, efficiency and cost. Subthemes included two that were deemed essential to high-fidelity implementation: the development of credentialing policies and robust quality assurance processes. These findings identify potential determinants of system and clinician level adoption that may be leveraged to achieve high-fidelity implementation of POCUS applications that result in improved patient outcomes.

**Keywords:** point of care ultrasound; implementation science; adoption

## **1. Introduction**

Point of care ultrasound (POCUS) is ultrasound imaging that is acquired and interpreted by a clinician at the bedside. Driven by growing clinical evidence [1–3], there is increasing interest in the integration of POCUS use into routine clinical care by multiple specialties. Though emergency medicine [4] and critical care [5,6] were the first specialties to

**Citation:** Maw, A.M.; Morris, M.A.; Barnard, J.G.; Wilson, J.; Glasgow, R.E.; Huebschmann, A.G.; Soni, N.J.; Fleshner, M.; Kaufman, J.; Ho, P.M. Multi-Level Stakeholder Perspectives on Determinants of Point of Care Ultrasound Implementation in a US Academic Medical Center. *Diagnostics* **2021**, *11*, 1172. https://doi.org/ 10.3390/diagnostics11071172

Academic Editor: Tivani P. Mashamba-Thompson

Received: 9 April 2021 Accepted: 22 June 2021 Published: 28 June 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

integrate POCUS into their training standards, multiple other medical and surgical specialties are following their lead, including internal and hospital medicine whose professional societies now officially endorse diagnostic POCUS use [7,8].

In spite of the growing evidence and interest from the clinical community [9], POCUS use has not yet been implemented into practice broadly. Prior studies indicate the cost of equipment and training opportunities were the most commonly reported barriers to adoption by clinicians [10]. However, recently, the cost of ultrasound equipment has dropped dramatically, allowing handheld ultrasound devices to be purchased directly by clinicians. Additionally, more professional societies have developed courses and training pathways for generalists. While recent surveys have published determinants of clinician adoption of POCUS [11], barriers likely vary significantly by local setting. Therefore, implementation efforts must begin with assessing current determinants experienced by local clinicians as implementation will certainly fail without their buy-in.

The growing interest from clinicians across specialties and their ability to directly purchase personal handheld ultrasound devices has compelled hospitals to consider the need to develop local policies and invest in infrastructure to ensure the security of POCUS images as protected health information and quality assurance of these images. Accordingly, attention has now broadened to include determinants of adoption at the health system level, in order to ensure high-fidelity and sustainable POCUS use [12–15]. For instance, in January 2020, the Joint Commission endorsed a statement by the Emergency Care Research Institute (ECRI), a medical safety advisory group, that many healthcare facilities do not currently have adequate infrastructure, policies, and processes in place to ensure optimal and safe clinical use of POCUS [16,17]. However, determinants of POCUS adoption at the health system level, including credentialing policies, quality assurance processes, and information technology infrastructure, as well as the value proposition of POCUS implementation from the system perspective, are not well described.

Given that successful implementation of any technology requires adoption at both the clinician and health system level, prior studies that have focused on determinants of clinician adoption are necessary but not sufficient. Determinants of adoption at the system level will need to be addressed in order to facilitate adoption at the clinician level. The purpose of this qualitative study was to understand the current determinants of POCUS adoption at *both* the clinician and system levels in order to identify implementation strategies that could facilitate adoption at academic medical centers.

## **2. Materials and Methods**

## *2.1. Study Design*

We performed a qualitative study to capture broad perspectives on determinants of POCUS implementation at both the system and clinician levels. Data were collected for this study from semi-structured interviews of stakeholders in diverse professional roles within a hospital system. This study is part of a larger study investigating the implementation of point of care lung ultrasound by hospitalists in the care of hospitalized adults during the COVID-19 pandemic.

## *2.2. Conceptual Framework*

Given the multi-level institutional and external factors that affect the adoption of POCUS by clinicians and health systems, we selected the Pragmatic Robust Implementation and Sustainability Model (PRISM) to frame our investigation. PRISM is a pragmatic multi-level contextual model, includes relatively specific domains relevant to POCUS, and is tied to implementation outcomes in the RE-AIM framework [18–20]. The RE-AIM [20] framework was developed to promote external validity and equity in research on health interventions and assesses both implementation and effectiveness outcomes. The contextual domains of PRISM include known drivers of implementation [19] in the External environment (i.e., national policies, guidelines, and incentives) and the Internal setting (i.e., multi-level organizational characteristics, perspectives, implementation and

sustainability infrastructure). The use of PRISM was recommended for the planning stages of implementation of health interventions to help identify determinants (i.e., barriers and facilitators) that will inform the creation and selection of implementation strategies, thereby enhancing adoption, implementation and maintenance of evidence-based practices [18]. The contextual domains of PRISM were used to guide the interview protocol, data coding, and analysis.

## *2.3. Study Sample and Setting*

We interviewed multi-level stakeholders at University Colorado Hospital, a 550-bed quaternary care academic medical center in Aurora, Colorado. Interviews were performed as part of a pilot study investigating lung ultrasound implementation by hospitalists during the COVID-19 pandemic. A key stakeholder was defined as an individual who has an influence on POCUS adoption. Stakeholders included hospitalists, subspecialists, radiographers, administrators, information technologists, clinical and non-clinical hospital leadership. Patients and trainees were not recruited because there is already a body of literature demonstrating both types of individuals support POCUS use and are drivers of adoption [21,22].

This study was approved by the Institutional Review Board of the University of Colorado in April 2020. Postcard consent was obtained from all participants. We used purposeful sampling for initial study recruitment and snowball sampling to complete enrollment.

## *2.4. Data Collection*

Between July 2020 and January 2021, two investigators (JGB and AMM) conducted semi-structured interviews with key stakeholders to understand their perspective on POCUS implementation in their local setting and more broadly. We developed interview guides for each stakeholder demographic (Supplementary Material File S1) which were guided by the contextual domains of PRISM and evolved over the course of data collection. Interviews were conducted by phone or video conferencing. Data collection continued until preliminary analyses indicated thematic saturation when no additional themes were emerging from the interviews.

## *2.5. Data Analysis*

Interviews were audio-recorded and transcribed verbatim by a professional transcription service. A mixed deductive and inductive coding process aligned with a framework method approach was used [23]. The framework for deductive codes included the contextual domains of PRISM for the external and internal environment (Table 1). We additionally allowed for new codes that inductively arose from the data. Two investigators (AMM and JGB) began the analysis by immersing in the data and then developing the initial coding framework based on the PRISM domains, which was independently applied to a subset of transcripts. The research team then met multiple times to reconcile coding, refining and further developing the coding framework until a final coding framework was agreed upon. One investigator (AMM) applied the framework to the remaining transcripts, with a second investigator (JGB) double coding 20% of the transcripts to ensure consistency of application of the codes across the transcripts. All discrepancies were reconciled through consensus. The codebook and analysis were reviewed by other research team members (MF and JK) and MAM, a doctoral-trained qualitative expert. Coded data were analyzed within and across different stakeholder groups to identify major and minor themes that represent the participants' perceptions of POCUS implementation determinants.


**Table 1.** Coding Framework using Contextual Domains from the PRISM.

## **3. Results**

Of the 36 stakeholders invited, a total of 31 key hospital stakeholders participated in interviews that lasted 20 to 45 min (Table 2). In addition to hospitalists, seven subspecialty clinicians from cardiology (1), nephrology (1), anesthesia (1), pulmonology/critical care (1), and emergency medicine (3) were interviewed. Recruited clinicians had a broad spectrum of POCUS experience ranging from novice to experts who routinely use POCUS for diagnosis of multiple disease processes including pneumothorax, pneumonia, pleural effusion, and decompensated heart failure. Hospital leaders included quality and safety leaders, clinical operations leaders including radiology informatics, a hospital medicine clinical leader, and a leader from the health system's POCUS task force committee. Hospital administrators included an administrator for the division of hospital medicine and radiography. Support staff interviewed included two information technologists and a radiology technician.

**Table 2.** Participant Demographics.


Although our data fit well within the contextual domains of PRISM, there were two domains within the external environment in which no data were collected: policy and incentives. No themes regarding external incentives related to POCUS emerged spontaneously from these data, and no question on our interview guide explicitly inquired about this topic, although views regarding clinical practice guidelines were collected which is an alternative domain within the external environment of PRISM, in contrast to policy.

## *3.1. Themes*

Cutting across our PRISM codes and domains, three dominant themes emerged from stakeholder interviews about a system- and clinician-level adoption: clinical impact, efficiency and cost (Table 3). Although these three themes were central in discussions of adoption, the relative importance and relationship between themes differed by stakeholder level. Subthemes also varied slightly by stakeholder level. Hospital leaders generally focused on determinants of system-level adoption and clinicians were focused on determinants at the clinician level. Cost or financial impact was perceived as the fundamental arbiter of system-level adoption with quality metrics and measures of efficiency, such as length of stay, acting as a surrogate for cost. In contrast, although clinician stakeholders acknowledged that high-value practices were desirable, they seemed more concerned with the potential costs to patients as opposed to costs to the hospital. Additionally, clinician stakeholders often placed more emphasis on clinical impact in terms of patient outcomes and personal workflow efficiency with regard to their own likelihood of adoption. Table 4 offers additional quotations supporting the themes discussed.


**Table 3.** Themes and subthemes.

## 3.1.1. Theme 1: Clinical Impact

Across stakeholder types, the perception that POCUS use had the potential to both benefit and harm patients was a central theme in the discussion of determinants of both clinician and system adoption. Ensuring adequate training was a subtheme among clinicians when discussing determinants of personal adoption. The creation of hospital policies around POCUS credentialing, privileging and quality assurance infrastructure was a subtheme among hospital leaders. Patient perceptions of POCUS were seen as a determinant of clinician and health system adoption by both clinicians and hospital leaders. Many clinicians who had experience using POCUS commented they perceived it enhanced their rapport with patients and practice experience.


**Table 4.** Participant quotations illustrative of themes.


**Table 4.** *Cont*.

## 3.1.2. Potential for Both Clinical Benefit and Harm

*Clinician-level adoption:* Many clinician stakeholders reported that they perceived clinical decision-making could be enhanced by POCUS because of the improved accuracy in diagnosing decompensated heart failure, pneumonia, pleural effusion, and pneumothorax, and in doing so, would expedite initiation of appropriate therapies. This perceived benefit to patient care made POCUS adoption attractive to clinician stakeholders across specialties. For instance, clinician J1 said: "*If you're concerned, you don't have time to get radiography, and so having access to ultrasound– it's essential to make the diagnosis*".

However, many clinician interviewees considered patient harm from POCUS misuse due to inadequate training as an important potential pitfall of POCUS use. Another potential pitfall mentioned was if clinicians apply ultrasound outside the scope of either its intended use of their specialty's practice. For instance, clinician E2 was quoted as saying "*People using ultrasound in the, quote, 'wrong way'* ... *[end up] saying a lot more than they are qualified to say*". Many clinicians interviewed expressed that access to adequate training and clear guidelines for use were important prerequisites to their personal adoption of POCUS.

*System-level Adoption:* The need to ensure the quality of care with appropriate credentialing and quality assurance mechanisms emerged as important perceived determinants of adoption at the system level. Hospital leader interviewees emphasized the importance of robust quality processes to ensure high-fidelity system-level adoption of POCUS. For instance, when discussing the reasons why the hospital had recently decided to form a POCUS task force committee, hospital leader G7 said it was an attempt to safeguard appropriate use, "*How do we assure that people are utilizing the tool correctly and not tinkering with a diagnostic procedural skill without a clear understanding?*".

## 3.1.3. Patient and Physician Experience Related to POCUS Use

*Clinician-level adoption:* Many clinicians reported improved patient–physician rapport and experience with POCUS use as an important advantage of POCUS. The additional clinician time at the bedside required to perform POCUS exams allowed for additional conversation and rapport building between the clinician and patient, as well as augmented history gathering. Clinician A27 stated: "*I think it also increases the amount of time that I'm with patients, and so there's probably some either measurable or unmeasurable intangible thing there that it builds rapport*". Clinicians reported reviewing POCUS images with patients as they are acquired in real-time which allowed for enhanced conversations regarding their medical issues, offering an opportunity for patient education.

*System-level Adoption:* The potential impact of POCUS adoption on patient satisfaction scores was perceived by hospital leaders as an important determinant of system-level adoption. Patient satisfaction scores are a reportable metric and could potentially impact the number of patients seeking care, thereby influencing hospital revenue. Hospital leader G6 stated: "*They [hospital executives] care about their rankings so probably reported patient experience scores, and then they care about the brand. I suppose you could make a case that you could market this as, 'We are technologically innovative in a way that other health systems are not.' That might be an appeal*".

## 3.1.4. Theme 2: Efficiency

*Clinician-level Adoption:* Many stakeholders felt that clinicians had to tolerate a period of reduced efficiency in their personal clinical workflow in order to adopt POCUS, but once the skill was mastered, it would improve the efficiency of their diagnosis and management of patients. For instance, a novice POCUS user may take more time to perform and incorporate a lung ultrasound exam into clinical decision-making compared to traditional approaches using auscultation and chest X-ray. However, as a clinician gains experience, incorporating point of care lung ultrasound was perceived as improving efficiency in diagnosis and management. In the words of clinician A18: "*I think if you're able to invest that time you actually may come out ahead in terms of providing appropriate care for your patients. It's kind of like your catch 22*". Multiple clinician stakeholders observed that to complete the process of adoption, clinicians needed to believe that the required time investment in practice would eventually pay off in more accurate and quicker clinical decisions.

Many clinicians interviewed believed POCUS increases clinician time at the bedside with the patient. As a result, even clinicians who had integrated POCUS into their daily practice reported performing fewer exams when they were caring for a large number of patients. "*If I've got a much higher patient census, more patients I have to take care of, I'm less likely to use it 'cause I don't have time*", Clinician A7 stated. When the patient volume was high clinicians reported relying instead on radiology performed tests such as chest X-rays because they take less clinician time to use as they only require a moment to place the order. However, they also acknowledged that their clinical decision-making was delayed with the use of chest X-rays instead of POCUS as they then had to wait for the technician to acquire the images and radiologists to interpret them.

*System-level adoption:* Increased hospital efficiency was identified by hospital leaders as one of the most important facilitators of system-level adoption for all POCUS applications. Hospital leader G6 said: "*Honestly, my experience is if you want to get anything done in hospitals, you have to go after the efficiency piece*". Length of stay was considered one of the most important measures of system efficiency by hospital leaders. A decrease in ordering low-yield radiology tests, such as portable chest X-rays, was also considered a potentially valuable benefit of POCUS use. Support Staff D2 said: "*I think POCUS would probably be utilized a little more judiciously [because it's performed by clinicians], rather than pressing a button and saying, 'X-ray for 14 days every day while the patient's here'*".

## 3.1.5. Theme 3: Cost

*Clinician-level Adoption:* Many clinician stakeholders reported that providing "highvalue care" was important and perceived POCUS use as facilitating high-value care because of its potential to expedite diagnosis, appropriate treatment, and discharge while potentially avoiding the need for additional diagnostic tests, such as chest X-ray. Clinician J1 said: "*For me, I think the idea of cost containment and medicine is huge, especially in the intensive care unit where resource allocation is just so extreme. If you had outcome data to suggest that this [POCUS] is beneficial, and we're actually containing the cost for the hospitalization, that's a no brainer then."* In contrast, some clinicians worried that POCUS utilized unnecessarily would incur an extra expense for patients. Clinician A7 stated: "*A group of hospitalists and providers are concerned that, even though it's a pretty low-cost test, that if we're charging patients for a test that isn't really changing our management at all...then that's a test we shouldn't be doing*".

*System-level Adoption:* Hospital leaders perceived financial impact on the health system as the single most important determinant to system-level adoption, even though fully understanding the financial impact was a complex task. Although patient satisfaction, quality of care, and hospital efficiency were considered important aspects of system-level adoption, their impact on the financial health of a hospital was critical to their importance to the health system. Hospital leader G6 stated: " ... *it's corporate America, and it's about the bottom line*".

A concern expressed by hospital leaders was a lack of clarity about who should pay for the implementation of a new intervention, particularly one that involves clinician

training such as POCUS. Some stakeholders stated that in academic settings, hospital executives may question whether the hospital and health system should bear the cost of clinician training or whether it should be paid by the medical school or clinicians themselves. Hospital Leader G6 demonstrates this perception with the following quote, "*I think university hospital would say, 'Wow, that sounds like education. The dean does education. The dean should pay for this'*".

## **4. Discussion**

We sought to understand the current determinants of POCUS adoption at the system and clinician level in a quaternary care academic medical center in the United States. Our results suggest that determinants of adoption at both stakeholder levels are well aligned with cost, efficiency, and clinical impact. Most stakeholders interviewed believe that POCUS has the potential to improve outcomes, patient and clinician experience, efficiency, and cost. However, many study participants emphasized that adequate training, credentialing policies, and quality assurance processes must be implemented to realize these potential benefits and prevent potential patient harm from inappropriate POCUS use.

The themes and sub-themes that emerged from these data on improving outcomes, patient and clinician experience, and reducing cost, map nicely onto the Quadruple Aim [24,25] that seeks to improve patient outcomes, patient experience/satisfaction, health care system costs, and health care clinician/staff satisfaction. The Quadruple Aim is a framework used by many health care agencies including the Agency for Healthcare Research and Quality to guide optimization of the health care system. The Quadruple Aim framework may be used as a tool in future studies to understand the extent to which the implementation of POCUS can contribute to high-value sustainable care.

There have been many surveys assessing barriers to POCUS adoption from the clinician's perspective [11,26–30]. This study expands notably upon those data. It is the first qualitative study that explores the determinants of clinician- and system-level adoption from the perspective of both clinical and non-clinical stakeholders in a high-resource health system. Given the call for robust quality assurance procedures in U.S. hospitals [16], POCUS adoption will require changes to health system infrastructure in addition to clinician behavior change. Our study adds to the literature on perceived advantages and barriers to POCUS use at both system and clinician levels. Given the concerns that were raised by clinicians and health system leaders regarding the importance of implementing POCUS in a manner that ensures patient safety and the cautions published by national organizations regarding the need for robust quality assurance processes [16], further evaluation of the determinants of high-fidelity implementation is warranted.

In addition to its potential to improve patient health outcomes, many clinician interviewees perceived a benefit from the enhanced patient–physician experience associated with POCUS use. While improved patient experience associated with POCUS was suggested in the literature previously [21,22], to our knowledge, the positive impact on clinician experience, while described in multiple editorials, has not before been demonstrated in primary data. The impact of POCUS use on clinician experience should be explored and characterized in future studies. This is particularly important given the epidemic of burnout among clinicians and the growing recognition of clinician experience as an underpinning of quality health care [24].

In addition to clinical outcomes, we learned that POCUS's effect on efficiency may be an important determinant of both clinician- and system-level adoption. This emphasis on efficiency is not surprising given the hospitalist movement was born out of a desire to increase the efficiency of hospitals and control costs [31]. Although some evidence is emerging that POCUS use can improve clinician and system efficiency [2,32,33], more studies are needed to understand which applications and in what context POCUS is most likely to provide this benefit.

Finally, the financial impact of POCUS on the health system and the patient was considered an important determinant to system and clinician adoption by many participants. Hospital leaders as well as some clinicians interviewed considered cost as the most important determinant of adoption at the systems level. Notably, many clinician participants considered high-value care an important aspect of clinical decision-making. The generalizability of this finding should be explored in future studies as the concept of high-value care is relatively new and the extent to which it has been internalized by clinicians broadly is unknown.

The findings on POCUS adoption by a health system and its clinicians captured in the present study mirror professional society conversations around this topic currently [8,15]. As the evidence for the utility of POCUS has grown with multiple studies demonstrating improved accuracy [34], expedited diagnoses [2], and an associated reduction in additional testing and overall costs [32,33,35], there is now an increasingly recognized need for guidelines in training standards and quality assurance to guide implementation [13,14,16].

From an implementation science perspective, given that high-fidelity implementation of complex health interventions, such as POCUS, are known to be highly contextdependent [36], traditional guidelines that offer generic recommendations will likely be insufficient to ensure the benefits of POCUS are realized locally. Instead, guides may help local interested parties assess determinants of adoption unique to their environment, develop quality assurance infrastructure, and evaluate the effectiveness of local processes. Such adoption (and potentially implementation, adaptation and sustainability) guides can help ensure the potential benefits of POCUS use are reproduced in diverse real-world practice settings. The PRISM contextual model seemed to work well in categorizing the themes that emerged, but we note that not all PRISM domains were discussed and that alternative models of contextual factors related to adoption may also have fit the emergent data such as Rodger's Diffusion of Innovation Theory [37], Normative Process Theory [38] and Consolidated Framework for Implementation Research [39]. Finally, this study focused on determinants of adoption. Future research should explore determinants of the successful implementation, adaptation and sustainment of POCUS, which may or may not be the same.

## *Limitations and Future Directions*

This study has several limitations that should be acknowledged. First, the study population was limited to stakeholders at one academic medical center in the United States. Implementation of complex health interventions, such as POCUS, is highly contextdependent and therefore the generalizability of our results may be limited to similar settings. However, although specific barriers unique to each practice setting must be identified in order to create effective dissemination strategies, we anticipate the dominant themes found in these data, namely cost, efficiency, and clinical impact, will be important in the vast majority of practice settings in the U.S. Another limitation is that we did not interview executive-level managers who are the ultimate decision-makers of a health system. Finally, we chose to forego interviews of patients in this study because there is evidence demonstrating patients generally support POCUS use by clinicians [21,22]. However, gaining a better understanding of the value proposition of POCUS use from the patient perspective, particularly their input on whether this interaction with clinicians does generally increase their engagement and satisfaction as study participants theorized, may be important to helping the health system leaders decide whether to invest in implementation.

To our knowledge this is the first qualitative study to explore clinician- *and* systemlevel determinants of POCUS adoption in an academic medical center in the U.S. In future work, we hope to determine how differences in contextual factors affect implementation determinants of POCUS applications by extending our work to stakeholders in diverse hospital systems.

## **5. Conclusions**

Determinants perceived to be important to both clinician- and system-level adoption of POCUS use included clinical impact, efficiency, and cost at an U.S. academic medical center. Future studies should focus on how determinants vary across different types of hospital systems so that contextually sensitive implementation strategies may be identified and employed in the pursuit of high-fidelity and sustainable implementation of POCUS applications that optimize patient outcomes.

**Supplementary Materials:** The following are available online at https://www.mdpi.com/article/10 .3390/diagnostics11071172/s1, Supplementary Material File S1. Interview guide for Hospitalists.

**Author Contributions:** Conceptualization, A.M.M.; methodology, M.A.M., J.G.B., R.E.G., A.G.H.; formal analysis, A.M.M., J.G.B., M.A.M., M.F., J.K.; investigation, A.M.M., J.G.B., M.F., J.K., J.W.; writing—original draft preparation, A.M.M.; writing—review and editing, A.M.M., M.A.M., J.G.B., J.W., R.E.G., A.G.H., N.J.S., M.F., J.K., P.M.H.; supervision, P.M.H., A.G.H., R.E.G., M.A.M., N.J.S.; funding acquisition, R.E.G., A.M.M. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was funded by the National Heart, Lung, and Blood Institute, K12HL137862.

**Institutional Review Board Statement:** The study was conducted according to the guidelines of the Declaration of Helsinki, and approved by the Institutional Review Board of the University of Colorado (protocol code 20-0904 and 15 April 2020).

**Informed Consent Statement:** Informed consent was obtained from all subjects involved in the study.

**Data Availability Statement:** The data presented in this study are available on request from the corresponding author.

**Conflicts of Interest:** The authors declare no conflict of interest.

## **References**


*Article*
