**Use of a Smartphone-Based Augmented Reality Video Conference App to Remotely Guide a Point of Care Ultrasound Examination**

**Davinder Ramsingh 1,\*, Cori Van Gorkom 1, Matthew Holsclaw 1, Scott Nelson 2, Martin De La Huerta 2, Julian Hinson <sup>1</sup> and Emilie Selleck <sup>1</sup>**


Received: 20 September 2019; Accepted: 22 October 2019; Published: 24 October 2019

**Abstract:** Reports on the use of various smartphone-based video conference applications to guide point-of-care ultrasound (POCUS) examinations in resource-limited settings have been described. However, the use of an augmented reality-enabled smartphone video conference application in this same manner has not been described. Presented is a case in which such as application was used to remotely guide a point of care ultrasound examination.

**Keywords:** point of care ultrasound; augmented reality; telemedicine

#### **1. Introduction**

The ability to improve the quality of care to resource-limited settings is often a logistical challenge resulting from the lack of specialized medical practitioners and services. This is often secondary to geographic, demographic, and socioeconomic factors. The implementation of technology in healthcare is often a contributor to this problem rather than a solution. However, recent innovations in smartphone technology and point-of-care ultrasound (POCUS) devices have proven to be key examples of how technological advances are poised to elevate the quality of care in resource-limited settings. Indeed, the use of smartphone devices to provide real-time video conferences has proven to improve rural medicine across many medical specialties [1–5]. Similarly, the advancements in POCUS technology have greatly facilitated the ability to perform ultrasound exams in remote patient care settings.

Point-of-care ultrasound refers to the use of ultrasonography at the patient's bedside for diagnostic and therapeutic purposes [6]. The provider acquires and interprets all images in real-time and then uses that information to diagnose and direct therapies. Of note, POCUS has been identified as the most rapidly growing sector in medical ultrasound imaging [7]. Recent advances in this technology include improved image quality as well as a significant reduction in price, with handheld devices costing approximately 1/20th the price of devices ten years ago (from USD 40,000+ to USD 2000).

Recently, smartphone-based video conference technologies have been used with point-of-care ultrasound. Several studies have demonstrated the ability to remotely educate, guide, and provide image interpretation of POCUS examinations. While these studies have shown promise, a new modality, augmented reality (AR), has recently been implemented in the point-of-care ultrasound education space [8]. The use of AR has demonstrated utility in the remote guidance of POCUS [8]. However, the use of AR to improve remote medical training has mostly been described with the use of specialized equipment that may not be readily available in resource-limited environments.

This report highlights the use of a novel smartphone application (Vuforia Chalk, San Diego CA, USA) to provide AR remote assistance to guide a POCUS examination. The application works on most smartphone devices and web browsers and provides an augmented reality video conference interface that allows each user to notate the other's environment (see Figure 1). The ease of use and widespread applicability across multiple smartphone platforms allows this program to potentially improve the availability of remote AR guidance to teach POCUS in resource-limited settings. This non-sponsored case report was a proof-of-concept evaluation on the feasibility of using this AR application to improve remote guidance of POCUS examinations in a resource-limited environment. Specialty-trained providers from a tertiary care center, Loma Linda University Medical Center (LLUMC), in California, USA, successfully used this application to provide remote AR guidance for a POCUS examination at a rural hospital in Port-au-Prince, Haiti.

**Figure 1.** Overview of Onsite and Remote Augmented Reality Enhanced Video Communication. The white arrow indicates the femur, which was identified for anatomy review during remote guidance communication.

#### **2. Description of the Case**

Faculty from the tertiary care center traveled with a low-cost (\$2000 USD) handheld portable ultrasound device (Butterfly Network, Guilford, CT. USA) and a *Chalk*-enabled smartphone (iPhone 8, Apple Cupertino, CA. USA) to the Hôpital Adventiste d'Haiti. During the visit, the onsite and visiting faculty identified a 35-year-old male patient scheduled to undergo an external fixator removal and replacement. The patient required a regional anesthesia popliteal nerve block. The ability to use ultrasound to perform this block was not routinely available, and local providers had not been trained to perform this procedure. To evaluate the capability of the AR application to provide remote guidance for this procedure a connection was established, via the *Chalk* application, between the visiting faculty's smartphone in Port-au-Prince, Haiti to a remote faculty in CA, USA. Consent was obtained from the patient to report this case.

The connection was established over a mobile 4g hotspot via an iPhone 5 (Apple Cupertino, CA, USA) provided by the onsite faculty. *Chalk* was used to send a call to the expert ultrasonographer (San Diego, CA, USA), who then used the AR platform to guide probe placement on the patient to obtain the appropriate ultrasound image. Once the onsite examiner had obtained the appropriate probe position and ultrasound image, the smartphone camera was adjusted to visualize the ultrasound image from the POCUS device. Specifically, the smartphone POCUS exam was placed on the patient's bed in-between their legs and the second smartphone with the AR platform was held over this device such that the ultrasound image and the AR notations could be visualized by the physician performing the exam (Figure 1). Importantly the user holding the smartphone with the AR platform was not the same person performing the procedure and would adjust the smartphone position to allow appropriate visualization for the proceduralist. The remote expert then highlighted relevant anatomy and identified the nerve on the ultrasound image via the AR platform.

The procedure was performed successfully, and the nerve block demonstrated appropriate efficacy. After the procedure, the one onsite and the one remote provider completed a survey on the image quality of the video connection. In addition, the remote provider completed a survey of the image quality of the ultrasound image viewed from the video conference app and the onsite provider completed a survey of the AR notations created by the remote provider during the guidance of the nerve block. All surveys were scored using a validated 5-point Likert scale [9].

Survey results showed that the image quality of the video communication was rated 5/5 for both video communication and ultrasound image interpretation by the one onsite and one remote practitioner. The onsite practitioner scored the clarity of the AR notations to identify probe placement position on the body as 5/5 and identification of anatomy and nerve on the ultrasound image as 4/5.

#### **3. Discussion**

Technologic advances in medical ultrasound imaging are helping remove the barriers of costs and portability. These innovations are improving the ability to use medical ultrasound in resource-limited settings as a point of care device. However, a barrier that remains is the skill/training of the providers in these settings. Programs to teach POCUS, in-person, have demonstrated to be effective [10], but have a high cost and can be difficult to repeat/grow. The use of real-time video conferencing has demonstrated to be effective for remote POCUS guidance and training [11,12]. The application of such a remote guidance and telecommunication system has demonstrated a positive clinical impact. Kolbe et al. reported a change in management in 48% of patients in a rural village in Nicaragua after the implementation of a remote guidance and telecommunication system between expert sonographers around the world and local practitioners [13]. The application of smartphone-based video conference platforms also has proven clinical utility [12]. Robertson et al. demonstrated successful communication between intensivists at a tertiary care center and non-physician health care providers in a low-income country, which demonstrated successful ability to both educate POCUS image acquisition techniques as well as allow for appropriate image quality for remote clinical interpretation [14].

In recent years, the development of AR has also been applied to POCUS. Wang et al. evaluated the feasibility of using a specific AR hardware/software platform, Microsoft HoloLens (Redmond, WA, USA), to remotely guide novice medical trainers through a trauma ultrasound examination [8]. While this does offer a tremendous opportunity to expand POCUS education, it may be less impactful in resource-limited environments. Advances in smartphone applications now allow for the use of real-time AR enhanced video communication without the need for expensive hardware. This potentially has broad implications in resource-limited areas by improving the ability to provide remote

POCUS education/guidance with low-cost smartphone and POCUS devices. While a limited example, this case demonstrates how these devices can be implemented to provide improved bedside assessment and therapies in a resource-limited environment.

Our report demonstrates the use of a smartphone app that allows for the creation of a real-time augmented reality environment in a manner very similar to common smartphone video conference applications. This proof-of-concept case report presents positive feedback from all of the providers involved and supports further exploration in this area. Of note, none of the onsite physicians involved in the case had used the Chalk app before this event. Additional discussion after the event between the onsite and remote providers highlighted that the use of this platform was an improvement over traditional smartphone video conferencing by allowing both users to provide real-time visual cues over the ultrasound image during the procedure. In addition, the AR interface provided a greater ability for depth perception compared to traditional smartphone video conferencing.

Importantly, there are several limitations for the setup described in this report. While providers reported no issues with screen glare, the use of two mobile devices to achieve the AR guidance resulted in a limitation of the field-of-view of the remote examiner. In addition, the setup described requires another individual to hold and manipulate the AR smartphone. Additionally, the placement of the ultrasound and AR devices will be different based on patient position and care setting. Finally, the AR video communication in this case report did not include the transfer of any protected health information, as this requires secure communication pathways, which have previously been described [15]. Truly, applications such as the one described in this report would require the development of these securities to provide the maximum benefit. Future evaluations of these technologies should seek to address each of these items.

Indeed, the potential widespread availability of a smartphone-based augmented reality training/guidance application makes this platform very exciting for improving healthcare in resource-limited environments by potentially providing a higher level of communication than standard video conferencing. It is the authors' hope that this report can be an example to stimulate formal research in this area.

**Author Contributions:** Conceptualization, D.R., and M.D.L.H.; methodology, D.R., C.V.G., M.D.L.H., S.N., M.H. and E.S.; formal analysis, D.R., J.H. and C.V.G.; software, D.R. and M.D.L.H.; investigation, D.R., C.V.G., M.D.L.H., S.N., J.H., M.D.L.H. and E.S.; writing, D.R., C.V.G., S.N., J.H., E.S.; project administration, D.R., C.V.G., S.N., E.S. and M.D.L.H.

**Funding:** This research received no external funding.

**Conflicts of Interest:** D.R.: Consultant for Fujifilm Sonosite, Consultant for and funded research from General Electric, Medical Advisory Board Member–EchoNous.

#### **References**


© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

### *Editorial* **Blockchain and Artificial Intelligence Technology for Novel Coronavirus Disease 2019 Self-Testing**

**Tivani P. Mashamba-Thompson 1,\* and Ellen Debra Crayton <sup>2</sup>**


Received: 26 March 2020; Accepted: 31 March 2020; Published: 1 April 2020

**Abstract:** The novel coronavirus disease 2019 (COVID-19) is rapidly spreading with a rising death toll and transmission rate reported in high income countries rather than in low income countries. The overburdened healthcare systems and poor disease surveillance systems in resource-limited settings may struggle to cope with this COVID-19 outbreak and this calls for a tailored strategic response for these settings. Here, we recommend a low cost blockchain and artificial intelligence-coupled self-testing and tracking systems for COVID-19 and other emerging infectious diseases. Prompt deployment and appropriate implementation of the proposed system have the potential to curb the transmissions of COVID-19 and the related mortalities, particularly in settings with poor access to laboratory infrastructure.

**Keywords:** self-testing; novel coronavirus disease-19; blockchain; artificial intelligence

The novel coronavirus disease 2019 (COVID-19) has now reached sub-Saharan Africa (SSA) with cases reported in more than 40 SSA countries. SSA health systems are already battling with poor health outcomes and high mortality rates linked to the unique quadruple (HIV, Tuberculosis and non-communicable diseases) burden of disease [1]. In addition, SSA's dense communities, informal settlements and rural and resource-limited settings are at particular risk and are most vulnerable to the COVID-19 outbreak. These populations are underserved in terms of health services and have the potential to become to new COVID-19 epicenters. The global COVID-19 statistics show surprisingly low transmission rates and fewer deaths in resource-limited countries, particularly countries in Sub-Saharan Africa (SSA). While SSA's young population and warm climate may put SSA at an advantage for coping with the COVID-19 outbreak [2], there is growing concern about the impact of COVID-19 co-infections among the people living with other immune-system-weakening conditions such as HIV, TB and diabetes and the struggling health system in resource-limited settings such as SSA countries [3,4].

There is a growing concern about a failure to find and report cases, especially given weak health systems, inadequate surveillance, insufficient laboratory capacity and limited public health infrastructure in African countries [5]. Access to accurate diagnosis, monitoring and reporting of health outbreaks requires a well-resourced healthcare system [6]. Evidence shows that most resource-limited countries lack an effective, rapid surveillance system [7]. These settings also have a limited availability of health technologies for the electronic surveillance of infectious diseases to facilitate the prevention and containment of emerging infectious diseases such as COVID-19 [7]. Universal health coverage, access to high-quality and timely pathology and laboratory medicine (PALM) services is crucially needed to support health-care systems that are tasked with achieving Sustainable Developmental Goals [8]. This calls for the rapid development and deployment of health innovations for accurate diagnosis and electronic surveillance of COVID-19 in underserved populations.

Recent evidence shows that prompt development and deployment of point-of-care (POC) diagnostics for screening in response to the COVID-19 outbreak can help to curb the spread of the disease and to alleviate the burden on the health system [9,10]. The impact of rapid testing on the COVID-19 death rate has been shown in Germany [11]. Emerging health innovations such as blockchain and artificial intelligence (AI) technology can be coupled with POC diagnostics to enable self-testing of patients in isolation as a result of exposure to COVID-19. Blockchain is a digital, public ledger that records online transactions. It involves the digital distribution of ledger and consensus algorithms and eliminates all the threats of intermediaries [12,13]. One of the commonly-known applications of blockchain is the crypto-currency Bitcoin [14], which has been successfully used as an alternative financial sector in emerging economies including countries in SSA [15]. Blockchain technology has shown adaptability in recent years leading to its incorporation in a wide range of applications including biomedical and healthcare systems [16–18]. The use of blockchain and AI in healthcare is evident in the following areas: management of electronic medical records; drugs and pharmaceutical supply chain management; biomedical research; education; remote patient monitoring; and health data analytics [17].

Mobile connected point-of-care diagnostics and self-testing has been successfully implemented in resource-limited settings [19–21]. However, there is limited evidence on the use of blockchain and AI technology for disease diagnosis. Bearing in mind the era of COVID-19 and the evidence on the overburdened healthcare systems and poor disease surveillance systems in resource-limited settings, and taking advantage of the available mobile Health (mHealth) systems, we recommend, a rapid development and deployment of low cost blockchain and AI-coupled mHealth connected self-testing and tracking systems as one of the strategic response strategies for COVID-19 and other immerging infectious diseases (Figure 1).

**Figure 1.** Proposed community-based blockchain and artificial intelligence-coupled mobile-linked self-testing and tracking system for emerging infectious diseases.

The initial step for this system is through a mobile phone or tablet application (app) which could be adapted from existing self-testing apps [22,23]. The app will request a user's personal identifier before opening pre-testing instructions. Following testing, the user will upload results into the app. The blockchain and AI system will enable the transfer of the test result to alert the outbreak surveillance

authorities of all tests performed as well as the number of positive and negative test results. This will help ensure that all positive cases are referred to a quarantine site for treatment and monitoring. The in-built geographic information system (GIS) in mobile devices will enable the tracking of the people who tested positive. This system will also be connected to the local and international databases to ensure appropriate surveillance and control of the outbreak.

The AI component of this technology will enable potent power in data collection (patient information, geographic location of the patient and test results), security, analysis, and curation of disparate and clinical data sets from federated blockchain platforms to derive triangulated data at very high degrees of confidence and speed. With this well-architected integrative technology platform, we assure secure and immutable data sets that enable the collection of high-quality data and can draw deep insights. Local development of these diagnostics can help overcome the supply chain challenges [24] and the cost which can limit accessibility of POC diagnostics in resource-limited settings. This technology can be adapted for use in community-based case finding of other infectious diseases such as HIV, TB and Malaria, which may be exacerbated by the current COVID-19 outbreak. Relevant stakeholders' involvement will be crucial to ensure the efficient development and sustainable implementation of the proposed technology, particularly in underserved populations.

**Funding:** This research received no external funding.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

*Review*
