Next Article in Journal
Morphometric Analysis of the Mandibular Canal and Its Anatomical Variants in a Chilean Subpopulation: Cone Beam Computed Tomography Study
Next Article in Special Issue
Handheld Ultrasound Devices Used by Newly Certified Operators for Pneumonia in the Emergency Department—A Diagnostic Accuracy Study
Previous Article in Journal
Foundational Models for Pathology and Endoscopy Images: Application for Gastric Inflammation
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Diagnostics
Volume 14
Issue 17
10.3390/diagnostics14171913
diagnostics-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Prospective Comparison of Nine Different Handheld Ultrasound (HHUS) Devices by Ultrasound Experts with Regard to B-Scan Quality, Device Handling and Software in Abdominal Sonography

by
Daniel Merkel
1,2,
Christian Lueders
3,
Christoph Schneider
2,
Masuod Yousefzada
2,
Johannes Ruppert
4,
Andreas Weimer
5,
Moritz Herzog
6,
Liv Annebritt Lorenz
7,
Thomas Vieth
8,
Holger Buggenhagen
8,
Julia Weinmann-Menke
9 and
Johannes Matthias Weimer
8,9,*
1
BIKUS—Brandenburg Institute for Clinical Ultrasound, Brandenburg Medical School Theodor Fontane (MHB), 16816 Neuruppin, Germany
2
Immanuel Klinik Rüdersdorf, University Hospital of the Brandenburg Medical School, 15562 Rüdersdorf bei Berlin, Germany
3
Klinik am See, Rehabilitation Center for Internal Medicine, 15562 Rüdersdorf bei Berlin, Germany
4
Department of Medicine, Justus Liebig University Giessen, 35385 Giessen, Germany
5
Center of Orthopedics, Trauma Surgery, and Spinal Cord Injury, Heidelberg University Hospital Heidelberg, 69118 Heidelberg, Germany
6
Else Kröner Fresenius Center for Digital Health, Faculty of Medicine Carl Gustav Carus, Technical University Dresden, 01307 Dresden, Germany
7
Department of Radiooncology and Radiotherapy, Mainz University Hospital, 55131 Mainz, Germany
8
Rudolf Frey Learning Clinic, University Medical Center of the Johannes Gutenberg, University Mainz, 55131 Mainz, Germany
9
I. Department of Medicine, University Medical Center of the Johannes Gutenberg University Mainz, 55131 Mainz, Germany
*
Author to whom correspondence should be addressed.
Diagnostics 2024, 14(17), 1913; https://doi.org/10.3390/diagnostics14171913
Submission received: 30 July 2024 / Revised: 26 August 2024 / Accepted: 28 August 2024 / Published: 30 August 2024
(This article belongs to the Special Issue Recent Advances and Application of Point of Care Ultrasound)
Browse Figures
Review Reports Versions Notes

Abstract

:
Background: The HHUS market is very complex due to a multitude of equipment variants and several different device manufacturers. Only a few studies have compared different HHUS devices under clinical conditions. We conducted a comprehensive prospective observer study with a direct comparison of nine different HHUS devices in terms of B-scan quality, device handling, and software features under abdominal imaging conditions. Methods: Nine different HHUS devices (Butterfly iQ+, Clarius C3HD3, D5CL Microvue, Philips Lumify, SonoEye Chison, SonoSite iViz, Mindray TE Air, GE Vscan Air, and Youkey Q7) were used in a prospective setting by a total of 12 experienced examiners on the same subjects in each case and then assessed using a detailed questionnaire regarding B-scan quality, handling, and usability of the software. The evaluation was carried out using a point scale (5 points: very good; 1 point: insufficient). Results: In the overall evaluation, Vscan Air and SonoEye Chison achieved the best ratings. They achieved nominal ratings between “good” (4 points) and “very good” (5 points). Both devices differed significantly (p < 0.01) from the other seven devices tested. Among the HHUS devices, Clarius C3HD3 and Vscan Air achieved the best results for B-mode quality, D5CL Microvue achieved the best results for device handling, and SonoEye Chison and Vscan Air achieved the best results for software. Conclusions: This is the first comprehensive study to directly compare different HHUS devices in a head-to-head manner. While the majority of the tested devices demonstrated satisfactory performance, notable discrepancies were observed between them. In particular, the B-scan quality exhibited considerable variation, which may have implications for the clinical application of HHUS. The findings of this study can assist in the selection of an appropriate HHUS device for specific applications, considering the clinical objectives and acknowledging the inherent limitations.

1. Introduction

Recently, ultrasound (US) has gained importance in clinical routine and preclinical emergency diagnostics. The minimization and improved digitization of equipment have led to the development of very compact ultrasound devices in the past two decades, which have been established as handheld ultrasound (HHUS) devices in a wide range of medical applications. In particular, from the perspective of point-of-care diagnostics (POCUS) [1], HHUS devices play an important role not only in the field of emergency medicine [2,3], outpatient diagnostics [4,5] and bedside diagnostics [6,7,8] but also under isolation conditions [9], in intensive care medicine [10] and, last but not least, in sonography training [11,12]. Several professional societies have published guidelines on the indication and safety of using HHUS [13,14,15,16].
While there are a few established manufacturers of traditional US devices [17], the variety of available HHUS systems [18] is increasing in parallel with their increasing economic importance [19,20,21]. There are both wired and wireless systems, HHUS with universal “all in one” transducers, as well as HHUS with several attachable transducers and systems that combine two commonly used transducer variants (“two-in-one” devices) [18,22]. The variety of available HHUS devices with very specific functionalities makes the selection of a suitable HHUS for the specific medical application difficult.
Several aspects come into play when selecting a suitable HHUS device [22,23], of which the specific medical question and the environment of the HHUS are the most important. For example, in a pre-clinical emergency setting, the demands placed on an HHUS device are different from those in an infectious disease intensive care unit under isolated conditions or in rural or underserved areas for general medical purposes [4,5,24].
In addition to equipment, weight, transducer shape, battery life, start-up time, heat generation, and other parameters, sufficient B-image quality is undoubtedly a primary criterion when assessing different HHUS devices [15,25]. Very few studies have compared HHUS devices from different manufacturers in terms of inter-system variability [26,27,28,29]; only two have performed an actual head-to-head comparison of HHUS devices on the same test subjects [28,29]. In a gynecological pilot study, three different HHUS devices were compared by the same examiner [29] regarding measurement accuracy. In a further study, four different HHUS devices used by several experts on the same test subject exhibited no significant differences regarding handling, but one of the devices was inferior in terms of B-image quality [28].
This prospective study evaluates a large variety of HHUS devices in a comprehensive head-to-head comparison by several experts. The primary goal is to make a direct comparison of B-image quality, handling, and user-friendliness of the software for a total of nine different HHUS devices in clinical abdominal US. The secondary goal is to rank the devices by cumulatively considering the various evaluation aspects.

2. Material and Methods

2.1. Study Design

This prospective observational study comparing different HHUS devices with regard to B-image quality, handling, and user-friendliness was conducted in July 2024 in a German university hospital. Ultrasound device manufacturers supplied their HHUS devices to a clinical ultrasound laboratory for a period of one week. Ultrasound experts were invited to test the devices on healthy subjects and to evaluate them using a structured questionnaire. The study protocol was reviewed and accepted by the Ethics Committee of the Brandenburg Medical School under the reference number E-01-20220502.

2.2. HHUS Devices

HHUS devices were defined as ultra-compact portable sonography devices consisting only of a transducer, with no peripheral devices, that can be operated with one hand [17,24]. The sonography images generated are transmitted to a separate display either by wire or wirelessly. These devices are approved as medical devices and have a CE label (label for European conformity in commercial products). The device had to be available for a trial period within a previously clearly defined period (1 week). In addition to an internet and literature search [18,26,27,28,30], several sonography device dealers and three university ultrasound centers were contacted to scope the HHUS market. Nine of the total 14 HHUS devices identified this way were included in this study (Table 1; Figure 1 and Figure 2). For three devices, no dealer contact could be established (iSiniQ 30A, mSonics MU1, and Sonostar Uprobe-C4PL), and the remaining two manufacturers (Alpinion minisono and Kosmos) could not supply their devices for the defined study period.

2.3. Examiners

The work of Howard et al. was used to plan the number of raters. A minimum number of 10 raters was defined [31]. The examiners had to be certified medical specialists, complete the testing of all available devices, and consent to participate in this study. They were recruited from an existing collegial network of DEGUM members (German Society for Ultrasound in Medicine).

2.4. Testing of Devices

Several testing supervisors were extensively trained on all devices. All testing took place in the same clinical ultrasound laboratory. All nine HHUS devices included in this study were displayed next to each other in open boxes. Each device had its own display (iPad or tablet) with the current device software installed. The devices were at full electrical charge. Testing was performed in a random order. Before each test, a supervisor switched on the HHUS, connected it to its display, and provided basic operating instructions. During the examinations, the displays were on a separate stand to the left of the test subjects, as is customary in ultrasound examinations. The supervisor was also available during all evaluations later on and assisted with the device settings. The examiner was asked to test each HHUS extensively considering all test parameters on a healthy test subject without a specified time limit and to fill out an evaluation form immediately afterwards. Finally, the examiners were asked to name the top three devices that were their personal favorites based on the overall impression of all criteria.

2.5. Evaluation Form

An interdisciplinary team (ultrasound experts and didactics) designed a written questionnaire based on the current literature containing three main items, “image quality”, “handling”, and “software”, with multiple graded subitems. The assessment was based on a 5-point Likert scale (5 = very good; 4 = good; 3 = satisfactory; 2 = sufficient; 1 = unsatisfactory).

2.6. Statistical Analysis

The data were entered by transferring the evaluation forms into an Excel spreadsheet (Microsoft Excel® Version 16.48, Microsoft Corporation, Redmond, WA, USA). All statistical analyses were performed in Rstudio (Rstudio Team [2020]. Rstudio: Integrated Development for R. Rstudio, PBC, http://www.rstudio.com, last accessed on 15 July 2024) with R 4.0.3 (A Language and Environment for Statistical Computing, R Foundation for Statistical Computing, http://www.R-project.org; last accessed on 15 July 2024). Binary and categorical baseline variables are given as absolute numbers and percentages. Continuous data are given as median and interquartile range (IQR) or as mean and standard deviation (SD). The D’Agostino and Pearson test was used to test for normal distribution. Categorical variables were compared using the chi-squared test, and continuous variables were compared using the T-test or the Mann–Whitney U test. Parametric (ANOVA) or non-parametric (Kruskal–Wallis) analyses of variance were calculated and further explored with pairwise post hoc tests (t-test or Mann–Whitney U). p-values of <0.05 were considered statistically significant. We also used the intraclass correlation coefficient (ICC) type 3,k with the values ranging from 0 (no reliability) to 1 (perfect reliability) to check the consistency between measures of the same class. We have also calculated Krippendorff’s alpha from the raters and the evaluation criteria.

3. Results

3.1. Examiners

A total of 12 examiners were recruited for this study. Supplementary Table S1 shows their baseline characteristics. Most (67%) work in internal medicine and have more than 10 years of professional experience (75%). All had already performed at least 1000 independent ultrasound examinations and are currently performing more than 20 examinations per week in diagnostic ultrasound, in either out-patient (42%) or in-patient settings (58%). Most had no previous experience (75%) with HHUS.

3.2. Results of the Ratings

Table 2 and Figure 3 show the results of the ratings for the main items and subitems. The average grade of the main items and subitems was 3.8, with a 95 percent confidence interval (95% CI) between 3.6 and 3.8. This corresponds to a nominal assessment of all evaluation items between “good” (4 points) and “satisfactory” (3 points). Significant differences were identified between the individual devices and categories, which are explained in more detail below for each category and device.

3.3. B-Mode Image Quality

Table 2 and Figure 3a show the grading for the subitems of B-mode image quality. The average grade was 3.7, with a 95 percent confidence interval (95% CI) between 3.5 and 3.8. This corresponds to a nominal assessment of all evaluation items between “good” (4 points) and “satisfactory” (3 points).
The best results in B-mode quality were achieved using the Clarius C3HD3 (4.3, 95% CI 4.0 to 4.5), Vscan Air (4.2, 95%CI 4.0 to 4.4), and Chison SonoEye (3.9, 95% CI 3.6 to 4.2) devices. This corresponds to a nominal assessment of all evaluation items between “very good” (5 points) and “good” (4 points). The other devices were graded significantly worse than these top three. All significance levels are shown in Supplementary Table S3a.

3.4. Handling

Table 2 and Figure 3b show the grading for device handling. The average grade was 3.7, with a 95 percent confidence interval (95% CI) between 3.6 and 3.9. This corresponds to a nominal assessment of all evaluation items between “good” (4 points) and “satisfactory” (3 points).
The best results in device handling were achieved by D5CL Microvue (4.4, 95% CI 4.2 to 4.5). This corresponds to a nominal assessment of all evaluation items between “very good” (5 points) and “good” (4 points). This grade is significantly better than that of the other devices. All significance levels are shown in Supplementary Table S3b.

3.5. Software

Table 2 and Figure 3c show the grading for the software. The average rating was 3.7, with a 95 percent confidence interval (95% CI) between 3.5 and 3.8. This corresponds to a nominal assessment of all evaluation items between “good” (4 points) and “satisfactory” (3 points).
The best grades for the software were achieved by the SonoEye Chison (4.4, 95% CI 4.3 to 4.5) and Vscan Air (4.2, 95% CI 4.1 to 4.4) devices. This corresponds to a nominal assessment of all evaluation items between “very good” (5 points) and “good” (4 points). Both devices received significantly higher grades than the others. All significance levels are shown in Supplementary Table S3c.

3.6. Overall Value

Table 2 and Figure 3d show the cumulative grades for all subitems.
The best overall grades were achieved by the Vscan Air (4.1, 95% CI 4.0 to 4.2) and Sonoeye Chison (4.1, 95% CI 3.9 to 4.2) devices. This corresponds to a nominal assessment of all evaluation items between “very good” (5 points) and “good” (4 points). Both devices received significantly higher grades than the others. All significance levels are shown in Supplementary Table S3d.

3.7. Final Subjective Overall Assessment

Figure 4 shows the results of the evaluators’ final subjective selection of the three best devices. Half of the evaluators (50%) voted the Vscan Air in first place. Further, two-thirds of the evaluators voted this device in the top three ratings. The Clarius C3HD3 and Chison SonoEye devices also achieved good ratings. These devices were selected as the top three in half of all evaluations.

3.8. Inter-Category Comparison per Device

An inter-category comparison per device is shown in Figure 5 and Figure 6.
The grades achieved in the three main categories, B-mode quality, handling, and software, were significantly correlated, particularly for the Philips Lumify, Vscan Air, and Youkey devices. Notable discrepancies were evident between the D5CL Microvue and TE Air devices. In the case of both devices, the handling was rated as very good, whereas the B-scan quality was only average.
Vscan Air was the only device for which the three main parameters correlated with each other and received high grades. This was also reflected in the final overall assessment of the devices made by the examiners, in which half of all examiners (50%) selected the Vscan Air as their favorite in a final subjective overall assessment (Figure 4).
A presentation of the main parameters as a function of the tested device can be found in Figure 5.
The ICC values for most devices are relatively high (ranges between 0.77–0.95), indicating good to excellent rater reliability (see Supplementary Table S4 Grading of HHUS devices overall, per main item overall and per subitem shown as median and IQR.). Krippendorff’s alpha for the 12 raters and 9 subjects was 0.23. The exclusion of extreme ratings shows an increase up to a Krippendorff’s alpha of 0.31 for 10 raters and 0.47 for 8 raters.

4. Discussion

The quality of nine different HHUS devices was evaluated comprehensively by ultrasound experts for the first time in this study. It was found that the overall B-mode quality of HHUS is better than previously estimated [6,32,33,34,35,36]. It is apparent that there are significant differences in B-scan quality, device handling, and software between HHUS devices from different manufacturers.
The use of HHUS systems for specific questions, as well as in comparison with conventional US systems [30,37,38], has been evaluated more than their inter-system variability. Preliminary studies on HHUS for intensive care and internal medicine detected potentially clinically relevant differences in the B-image quality of different HHUS devices. However, the assessments were not based on a direct head-to-head comparison of HHUS devices but rather on parallel examinations of a high-end device [26,27]. To date, comparisons have been made regarding a maximum of four different HHUS devices on the same test subjects. These were limited to measurement accuracy [29], ease of use [28], and overall satisfaction [28]. We included nine HHUS devices and many more parameters in this head-to-head comparison.

4.1. B-Image Quality

B-image quality, and, thus, the clinical significance of a sonographic examination, depends on many variables, for example, not only on the expertise of the examiner [39,40], the test subject’s ultrasound conditions, and the quality of the device or transducer [41] but also on the presets and post-processing [42,43,44]. Although measurements of phantoms have been established for standardization [45,46] and are required by professional societies [47], assessments of sonographic B-image quality in clinical examinations are predominantly carried out subjectively [23,26]. A systematic study of various HHUS devices showed that their B-image quality can match that of cart-based systems based on phantom measurements but does not correspond to that of high-end devices in terms of perceived detail resolution [48]. The present study did not consider the use of US phantoms to assess B-scan quality, as the primary objective was to investigate the clinical applications of HHUS. Future comparative studies should include systematic standardized US phantoms.
Cart-based US systems use various transducers shaped for their intended use. A convex transducer produces better B-images for abdominal sonographic questions than an “all-in-one” transducer aimed at a plethora of clinical settings. Two of the nine devices achieved the best B-image quality. Both are equipped with separate convex and linear transducers (Clarius C3HD3 and Vscan Air). Both HHUS devices have already impressed in previous studies with regard to B-image quality in abdominal US [26,27,28,48]. Both HHUS devices tested with a universal transducer (Butterfly iQ+ and TE Air) were inferior in terms of B-image quality. A special feature of Butterfly iQ+ is worth mentioning, as it is the only HHUS that uses a special microchip technology as a transducer instead of piezo crystals [1,22].

4.2. Handling and Software

The combination of a transducer, battery, and electronics in one device leads to various shapes, sizes, and weights of these ultra-compact “pocket-sized” HHUS devices (Figure 1).
In a previous study, there were no significant differences in handling between the four devices tested [28]. In contrast, the handling of the Microvue D5CL HHUS was rated significantly better than that of the other eight devices in this study. It is a wireless, extremely compact, lightweight device with two opposing ultrasound transducers. Interestingly, the presence of a wired connection to the monitor had no influence on the rating of handling. The three wired HHUS devices (Philips Lumify, Chison SonoEye, and Butterfly iQ+) ranked third, fifth, and sixth, with no significance to the other wireless devices.
All devices can connect to the internet, allowing for an upgrade or update of the software [41]. Nevertheless, HHUS devices have to meet particularly high requirements in terms of the user-friendliness of the software, as, unlike cart-based US devices, they do not have a conventional control panel for continuous image optimization [42,43,44]. For image optimization, the settings need to be adjusted via a menu within the software, thus requiring good preset settings, which were evaluated separately in this study. Usually, the free hand not holding the transducer is used for image optimization at the cart. In HHUS, this free hand is often occupied with holding the monitor [25], impeding manual image optimization. Solutions to this problem such as Velcro fasteners or tablet stands for the monitors often contradict the “pocket size” feature of HHUS.
SonoEye and the Vscan Air had the best software according to the evaluators. Even though the majority of the software gradings in this study were “acceptable” or “good”, the software was graded “inadequate” by individual reviewers for five of the nine devices tested. One argument to be made here is that, with longer and more frequent use of a specific HHUS device, one can become familiarized with less intuitive software and find hidden submenus more quickly. There was no time limit for performing the tests in this study. Despite the absence of a planned time recording procedure in accordance with the study protocol, each HHUS was tested for an approximate period of between five and ten minutes. This duration is sufficient for the user to become acquainted with the device. However, it is unlikely to be sufficient for an exhaustive exploration of the software’s submenus. We are not aware of any previous studies evaluating HHUS software.

4.3. Overall Satisfaction

As a final part of the assessment, each examiner had to name their top three of the nine HHUS devices tested (Figure 4). We found that 50% of the evaluators chose Vscan Air in first place and in the top three in two-thirds of the cases. This is similar to previous head-to-head comparisons [28] and is consistent with the results of comparative studies with a high-end device [26,27]. The runner-ups were Clarius C3HD3 and Chison SonoEye, each with 50% of the examiners placing them within the top three ratings. The spontaneous final ranking of the devices correlated very well with the numerically determined overall grading (Figure 3).

4.4. Clinical Importance and Perspective

The omnipresent availability of HHUS to medical professionals, as well as increasingly to non-medical professionals, raises questions regarding the limitations and risks of its use. Several position papers from professional societies emphasize the adequate training and expertise of users as prerequisites for the clinical use of HHUS [13,15,49].
It has been proven that assessments such as those of ascites or pleural effusion in an outpatient palliative setting or after cardiac surgery can certainly be carried out with HHUS by non-physician staff [5]. However, one cannot expect to be able to perform difficult oncological diagnostics with HHUS, e.g., in the context of hepatocellular cancer (HCC) screening in cirrhosis patients [50,51]. However, as the B-scan quality of HHUS increases, its field of application will expand to areas that have so far been reserved for high-end sonography. A sharp boundary between the areas of application of HHUS and conventional high-end US has not yet been defined [41]. However, HHUS certainly has much more potential than just detecting liquids [2,3,34]. The use of ultrasound in orthopedic and respiratory clinical settings could be increasingly implemented using HHUS in the future [52,53].
Due to their good networking and continuous possibility of cloud-based data storage, HHUS devices offer ideal conditions for collecting large amounts of data and, therefore, artificial intelligence (AI)-based applications. Several potential applications have already been published, mostly in the form of pilot studies [54,55]. Further developments can be expected in this area in the foreseeable future, which may fundamentally revolutionize the entire field of imaging diagnostics.

4.5. Use in Ultrasound Training

The positive evaluation of HHUS devices in the present study could also be used as an opportunity to adapt future course concepts in ultrasound training.
More integration of HHUS into short formats would enable even greater location-independent learning and teaching [56,57]. In undergraduate sonography training, which has increased significantly in recent years [58,59,60,61,62], HHUS devices could serve as a useful supplement and alternative to conventional ultrasound devices and, thus, reduce the expenditure of resources due to lower acquisition costs, allowing for more widespread implementation [18,63]. Their use in low- and middle-income countries in point-of-care diagnostics may lead to the implementation or maintenance of basic diagnostic capabilities in poorer regions [64,65].

5. Limitations

Not all available devices could be included. Even though the examiners were very experienced, the sample of examiners was very small. Other factors, such as pricing, durability, heat development during prolonged use, and more, were not evaluated. Some examiners had prior experience with HHUS. The device manufacturers were visible on the devices or in the software interface. Each examiner performed all testing on the same test subject for all devices, but not all examiners had the same test subject. Although the devices were each used on the same test subject, it was not mandatory to perform and compare the identical sonographic sections with each device. Another limitation of this study is the discrepancy between the ICC and Krippendorff’s alpha values in the inclusion of 12 raters, due to different assumptions and calculation bases. These differences make a consistent interpretation of inter-rater reliability difficult. To resolve this discrepancy in future studies, an adapted methodology could be used that integrates and harmonizes both measures, e.g., by using adapted weighting strategies or a more differentiated analysis of rater agreement.

6. Conclusions

Handheld ultrasound devices are becoming increasingly prevalent in clinical practice. They are useful when time is a critical factor (in emergency rooms and intensive care units) or when the setting is conducive to portable devices (remote locations, isolation ward, ambulatory use, isolation ward, etc.). This is the first comprehensive study to directly compare nine different HHUS devices in a head-to-head manner. While the majority of the tested devices demonstrated sufficient performance, there were notable discrepancies between them. In particular, the B-scan quality varied considerably, which has implications for the clinical application of HHUS. The findings of this study will assist in the selection of an appropriate HHUS device for specific applications, considering the clinical objectives and acknowledging the inherent limitations.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/diagnostics14171913/s1, Table S1. Baseline characteristics of the 12 examiners.; Table S2. Grading of HHUS devices overall, per main item overall, and per subitem shown as median and IQR. Table S3. p-values of the comparisons between the individual HHUS devices regarding B-mode image quality (a), device handling (b), software (c), and overall grade (d). p < 0.05 marked in green.; Table S4. Intraclass Correlation (ICC) based on the 3 main subjects and 12 raters. ICC type 3.

Author Contributions

Conceptualization: D.M. and J.M.W.; methodology and software: D.M., T.V. and J.M.W.; validation: D.M. and J.M.W.; formal analysis: D.M., J.R. and J.M.W.; investigation: D.M. and J.M.W.; resources: D.M. and J.M.W.; data curation: D.M., C.L., C.S. and J.M.W.; writing—original draft preparation: D.M. and J.M.W.; writing—review and editing: D.M., C.L., C.S., M.Y., J.R., A.W., M.H., L.A.L., T.V., H.B., J.W.-M. and J.M.W.; visualization: D.M. and J.M.W.; supervision: D.M. and J.M.W.; project administration: D.M. and J.M.W. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

The study was conducted according to the guidelines of the Declaration of Helsinki, and approved by the Institutional Review Board of the ethics committee of the Theodor Fontane Brandenburg Medical School under number protocol code E-01-20220502 (30 May 2022). All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and national research committees, and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

The data presented in this study are available on request from the corresponding author. The data are not publicly available because of institutional and national data policy restrictions imposed by the ethics committee since the data contain information that could potentially identify study participants. Data are available upon request (contact via [email protected]) for researchers who meet the criteria for access to confidential data (please provide the manuscript title with your inquiry).

Acknowledgments

We thank all participating nurses, students, and lecturers for supporting our study. We would like to also thank C. Christe for her help in revising the figures. Last but not least, we would like to thank all of the evaluators for their contribution.

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. Baribeau, Y.; Sharkey, A.; Chaudhary, O.; Krumm, S.; Fatima, H.; Mahmood, F.; Matyal, R. Handheld Point-of-Care Ultrasound Probes: The New Generation of POCUS. J. Cardiothorac. Vasc. Anesth. 2020, 34, 3139–3145. [Google Scholar] [CrossRef] [PubMed]
  2. Dewar, Z.E.; Wu, J.; Hughes, H.; Adnani, A.; Christiansen, G.; Ovedovitz, L.; Rittenberger, J.C. A comparison of handheld ultrasound versus traditional ultrasound for acquisition of RUSH views in healthy volunteers. J. Am. Coll. Emerg. Physicians Open 2020, 1, 1320–1325. [Google Scholar] [CrossRef]
  3. Mancuso, F.J.; Siqueira, V.N.; Moises, V.A.; Gois, A.F.; Paola, A.A.; Carvalho, A.C.; Campos, O. Focused cardiac ultrasound using a pocket-size device in the emergency room. Arq. Bras. Cardiol. 2014, 103, 530–537. [Google Scholar] [CrossRef]
  4. Zhou, Y.J.; Guo, L.H.; Bo, X.W.; Sun, L.P.; Zhang, Y.F.; Chai, H.H.; Ye, R.Z.; Peng, C.Z.; Qin, C.; Xu, H.X. Tele-Mentored Handheld Ultrasound System for General Practitioners: A Prospective, Descriptive Study in Remote and Rural Communities. Diagnostics 2023, 13, 2932. [Google Scholar] [CrossRef] [PubMed]
  5. Lo, H.; Frauendorf, V.; Wischke, S.; Schimmath-Deutrich, C.; Kersten, M.; Nuernberg, M.; Nuernberg, D.; Jenssen, C. Ambulatory Use of Handheld Point-of-Care Ultrasound (HH-POCUS) in Rural Brandenburg—A Pilot Study. Ultraschall Med. 2022, 43, 584–591. [Google Scholar] [CrossRef] [PubMed]
  6. Barreiros, A.P.; Cui, X.W.; Ignee, A.; De Molo, C.; Pirri, C.; Dietrich, C.F. EchoScopy in scanning abdominal diseases: Initial clinical experience. Z. Gastroenterol. 2014, 52, 269–275. [Google Scholar] [CrossRef] [PubMed]
  7. Barreiros, A.P.; Dong, Y.; Ignee, A.; Wastl, D.; Dietrich, C.F. EchoScopy in scanning abdominal diseases; a prospective single center study. Med. Ultrason. 2019, 21, 8–15. [Google Scholar] [CrossRef] [PubMed]
  8. Frohlich, E.; Beller, K.; Muller, R.; Herrmann, M.; Debove, I.; Klinger, C.; Pauluschke-Frohlich, J.; Hoffmann, T.; Kreppenhofer, S.; Dietrich, C.F. Point of Care Ultrasound in Geriatric Patients: Prospective Evaluation of a Portable Handheld Ultrasound Device. Ultraschall Med. 2020, 41, 308–316. [Google Scholar] [CrossRef]
  9. Bennett, D.; De Vita, E.; Mezzasalma, F.; Lanzarone, N.; Cameli, P.; Bianchi, F.; Perillo, F.; Bargagli, E.; Mazzei, M.A.; Volterrani, L.; et al. Portable Pocket-Sized Ultrasound Scanner for the Evaluation of Lung Involvement in Coronavirus Disease 2019 Patients. Ultrasound Med. Biol. 2021, 47, 19–24. [Google Scholar] [CrossRef]
  10. Zieleskiewicz, L.; Lopez, A.; Hraiech, S.; Baumstarck, K.; Pastene, B.; Di Bisceglie, M.; Coiffard, B.; Duclos, G.; Boussuges, A.; Bobbia, X.; et al. Bedside POCUS during ward emergencies is associated with improved diagnosis and outcome: An observational, prospective, controlled study. Crit. Care 2021, 25, 34. [Google Scholar] [CrossRef]
  11. Dietrich, C.F.; Goudie, A.; Chiorean, L.; Cui, X.W.; Gilja, O.H.; Dong, Y.; Abramowicz, J.S.; Vinayak, S.; Westerway, S.C.; Nolsoe, C.P.; et al. Point of Care Ultrasound: A WFUMB Position Paper. Ultrasound Med. Biol. 2017, 43, 49–58. [Google Scholar] [CrossRef]
  12. Wong, C.K.; Hai, J.; Chan, K.Y.E.; Un, K.C.; Zhou, M.; Huang, D.; Cheng, Y.Y.; Li, W.H.; Yin, L.X.; Yue, W.S.; et al. Point-of-care ultrasound augments physical examination learning by undergraduate medical students. Postgrad. Med. J. 2021, 97, 10–15. [Google Scholar] [CrossRef] [PubMed]
  13. Cardim, N.; Dalen, H.; Voigt, J.U.; Ionescu, A.; Price, S.; Neskovic, A.N.; Edvardsen, T.; Galderisi, M.; Sicari, R.; Donal, E.; et al. The use of handheld ultrasound devices: A position statement of the European Association of Cardiovascular Imaging (2018 update). Eur. Heart J. Cardiovasc. Imaging 2019, 20, 245–252. [Google Scholar] [CrossRef] [PubMed]
  14. Portability, Insurance, and Accountability Act. Appropriate Use Criteria for Handheld/Pocket Ultrasound Devices. Ann. Emerg. Med. 2018, 72, e31–e33. [Google Scholar] [CrossRef]
  15. Nielsen, M.B.; Cantisani, V.; Sidhu, P.S.; Badea, R.; Batko, T.; Carlsen, J.; Claudon, M.; Ewertsen, C.; Garre, C.; Genov, J.; et al. The Use of Handheld Ultrasound Devices—An EFSUMB Position Paper. Ultraschall Med. 2019, 40, 30–39. [Google Scholar] [CrossRef]
  16. European Society of Radiology (ESR). ESR statement on portable ultrasound devices. Insights Imaging 2019, 10, 89. [Google Scholar] [CrossRef]
  17. Merkel, D.; Albarri, E.; Yousefzada, M.; Boehm, M.; Gottwald, M.; Schneider, C. Differences in the B-mode imaging quality of ultrasound devices in the mid-price segment. Med. Ultrason. 2023, 25, 288–295. [Google Scholar] [CrossRef]
  18. Malik, A.N.; Rowland, J.; Haber, B.D.; Thom, S.; Jackson, B.; Volk, B.; Ehrman, R.R. The Use of Handheld Ultrasound Devices in Emergency Medicine. Curr. Emerg. Hosp. Med. Rep. 2021, 9, 73–81. [Google Scholar] [CrossRef]
  19. Jenssen, C.; Ewertsen, C.; Piscaglia, F.; Dietrich, C.F.; Gilja, O.H.; Sidhu, P.S.; Saftoiu, A.; Popescu, A.; Sporea, I.; Cantisani, V.; et al. 50th years anniversary of EFSUMB: Initial roots, maturation, and new shoots. Ultraschall Med. 2022, 43, 227–231. [Google Scholar] [CrossRef]
  20. Lanza, G.M. Ultrasound Imaging: Something Old or Something New? Investig. Radiol. 2020, 55, 573–577. [Google Scholar] [CrossRef]
  21. Harris, S. Ultrasound Equipment—World Market Report-2021 Edition, 4th ed.; Signify Research: Cranfield, UK, 2021. [Google Scholar]
  22. Dietrich, C.F.; Bulla, P.; Dudwiesus, H.; Lo, H.; Hocke, M.; Hoffmann, B.; Horn, R.; Lehmann, B.; Morf, S.; Nuernberg, D.; et al. Perspectives and Challenges of hand-held Ultrasound. Z. Gastroenterol. 2023, 61, 836–851. [Google Scholar] [CrossRef]
  23. Merkel, D.; Stahlheber, H.; Chupina, V.; Schneider, C. Comparison of the quality of B-scan ultrasound in modern high-end devices. Z. Gastroenterol. 2018, 56, 1491–1498. [Google Scholar] [CrossRef] [PubMed]
  24. Burleson, S.L.; Swanson, J.F.; Shufflebarger, E.F.; Wallace, D.W.; Heimann, M.A.; Crosby, J.C.; Pigott, D.C.; Gullett, J.P.; Thompson, M.A.; Greene, C.J. Evaluation of a novel handheld point-of-care ultrasound device in an African emergency department. Ultrasound J. 2020, 12, 53. [Google Scholar] [CrossRef]
  25. Bönhof, J.A.; Seitz, K. Geräteauswahl, Geräteeinstellung und Dokumentation. In Sonografie Kompetent: Von der Indikation zur Interpretation; Seitz, K., Ed.; Georg Thieme Verlag KG Verlag: Stuttgart, Germany, 2016; pp. 27–46. [Google Scholar]
  26. Merkel, D.; Züllich, T.F.; Schneider, C.; Yousefzada, M.; Beer, D.; Ludwig, M.; Weimer, A.; Künzel, J.; Kloeckner, R.; Weimer, J.M. Prospective Comparison of Handheld Ultrasound Devices from Different Manufacturers with Respect to B-Scan Quality and Clinical Significance for Various Abdominal Sonography Questions. Diagnostics 2023, 13, 3622. [Google Scholar] [CrossRef] [PubMed]
  27. Weimer, J.M.; Beer, D.; Schneider, C.; Yousefzada, M.; Gottwald, M.; Züllich, T.F.; Weimer, A.; Jonck, C.; Buggenhagen, H.; Kloeckner, R.; et al. Inter-System Variability of Eight Different Handheld Ultrasound (HHUS) Devices—A Prospective Comparison of B-Scan Quality and Clinical Significance in Intensive Care. Diagnostics 2023, 14, 54. [Google Scholar] [CrossRef]
  28. Le, M.T.; Voigt, L.; Nathanson, R.; Maw, A.M.; Johnson, G.; Dancel, R.; Mathews, B.; Moreira, A.; Sauthoff, H.; Gelabert, C.; et al. Comparison of four handheld point-of-care ultrasound devices by expert users. Ultrasound J. 2022, 14, 27. [Google Scholar] [CrossRef] [PubMed]
  29. Toscano, M.; Szlachetka, K.; Whaley, N.; Thornburg, L.L. Evaluating sensitivity and specificity of handheld point-of-care ultrasound testing for gynecologic pathology: A pilot study for use in low resource settings. BMC Med. Imaging 2020, 20, 121. [Google Scholar] [CrossRef]
  30. Rykkje, A.; Carlsen, J.F.; Nielsen, M.B. Hand-Held Ultrasound Devices Compared with High-End Ultrasound Systems: A Systematic Review. Diagnostics 2019, 9, 61. [Google Scholar] [CrossRef]
  31. Howard, M.C. The Relationship Between the Number of Raters and the Validity of Performance Ratings. Ind. Organ. Psychol. 2016, 9, 361–367. [Google Scholar] [CrossRef]
  32. Judmaier, G.; Seitz, K. How reliable is sonography of the upper abdomen with portable sonographic units? What does the future hold? Ultraschall Med. 2004, 25, 408–410. [Google Scholar] [CrossRef]
  33. Seitz, K.; Vasilakis, D.; Ziegler, M. Efficiency of a Portable B-Scan Ultrasound Device in Comparison to a High-End Machine in Abdominal Ultrasound. Results of a Pilot Study. Ultraschall Med. 2003, 24, 96–100. [Google Scholar] [CrossRef] [PubMed]
  34. Ziegler, C.M.; Seitz, K.; Leicht-Biener, U.; Mauch, M. Detection of therapeutically relevant diagnoses made by sonography of the upper abdomen: Portable versus high-end sonographic units—A prospective study. Ultraschall Med. 2004, 25, 428–432. [Google Scholar] [CrossRef] [PubMed]
  35. Stock, K.F.; Klein, B.; Steubl, D.; Lersch, C.; Heemann, U.; Wagenpfeil, S.; Eyer, F.; Clevert, D.A. Comparison of a pocket-size ultrasound device with a premium ultrasound machine: Diagnostic value and time required in bedside ultrasound examination. Abdom. Imaging 2015, 40, 2861–2866. [Google Scholar] [CrossRef]
  36. Lavi, A.; Tzemah, S.; Hussein, A.; Bishara, I.; Shcherbakov, N.; Zelichenko, G.; Mashiah, A.; Gross, M.; Cherbinski, L.; Neemann, Z. A urologic stethoscope? Urologist performed sonography using a pocket-size ultrasound device in the point-of-care setting. Int. Urol. Nephrol. 2017, 49, 1513–1518. [Google Scholar] [CrossRef]
  37. Rossi, D.; Marciano, F.; Cabassa, P. A multi-criteria methodology for evaluating alternative ultrasound devices. Ergonomics 2019, 62, 1301–1312. [Google Scholar] [CrossRef]
  38. Del Medico, M.; Altieri, A.; Carnevale-Maffe, G.; Formagnana, P.; Casella, F.; Barchiesi, M.; Bergonzi, M.; Vattiato, C.; Casazza, G.; Cogliati, C. Pocket-size ultrasound device in cholelithiasis: Diagnostic accuracy and efficacy of short-term training. Intern. Emerg. Med. 2018, 13, 1121–1126. [Google Scholar] [CrossRef]
  39. Rickes, S.; Treiber, G.; Monkemuller, K.; Peitz, U.; Csepregi, A.; Kahl, S.; Vopel, A.; Wolle, K.; Ebert, M.P.; Klauck, S.; et al. Impact of the operator’s experience on value of high-resolution transabdominal ultrasound in the diagnosis of choledocholithiasis: A prospective comparison using endoscopic retrograde cholangiography as the gold standard. Scand. J. Gastroenterol. 2006, 41, 838–843. [Google Scholar] [CrossRef]
  40. Trenker, C.; Gorg, C.; Jenssen, C.; Klein, S.; Neubauer, A.; Dietrich, C.F. The role of abdominal ultrasound in hematological diseases. Z. Gastroenterol. 2018, 56, 1063–1076. [Google Scholar] [CrossRef]
  41. Karlas, T.; Blank, V.; Trenker, C.; Ignee, A.; Dietrich, C.F. Ultrasound systems for abdominal diagnostics—Current methods, clinical applications and new technologies. Z. Gastroenterol. 2023, 61, 1235–1245. [Google Scholar] [CrossRef] [PubMed]
  42. Long, Z.; Zhou, W.; Tradup, D.J.; Stekel, S.F.; Callstrom, M.R.; Hangiandreou, N.J. Systematic optimization of ultrasound grayscale imaging presets and its application in abdominal scanning. J. Appl. Clin. Med. Phys. 2020, 21, 192–199. [Google Scholar] [CrossRef]
  43. Zander, D.; Hüske, S.; Hoffmann, B.; Cui, X.W.; Dong, Y.; Lim, A.; Jenssen, C.; Löwe, A.; Koch, J.B.H.; Dietrich, C.F. Ultrasound Image Optimization (“Knobology”): B-Mode. Ultrasound Int. Open 2020, 6, E14–E24. [Google Scholar] [CrossRef] [PubMed]
  44. Rosenberg, I.; Schwarzenbach, H.R.; Tuma, J. Knobology (Important Device Settings). Praxis 2012, 101, 1207–1212. [Google Scholar] [CrossRef] [PubMed]
  45. Alasaarela, E.; Koivukangas, J. Evaluation of Image Quality of Ultrasound Scanners in Medical Diagnostics. J. Ultrasound Med. 1990, 9, 23–34. [Google Scholar] [CrossRef] [PubMed]
  46. Mannila, V.; Sipila, O. Phantom-based quality assurance measurements in B-mode ultrasound. Acta Radiol. Short. Rep. 2013, 2, 1–4. [Google Scholar] [CrossRef]
  47. Haller, K.; Kollmann, C. Qualitätssicherung. In Sono-Guide für MTRA/RT; Thieme Verlag: Stuttgart, Germany, 2010; pp. 45–47. [Google Scholar]
  48. Herzog, M.; Arsova, M.; Matthes, K.; Husman, J.; Toppe, D.; Kober, J.; Trittler, T.; Swist, D.; Dorausch, E.M.G.; Urbig, A.; et al. Technical assessment of resolution of handheld ultrasound devices and clinical implications. Ultraschall Med. 2024, 45, 405–411. [Google Scholar] [CrossRef]
  49. European Society of Radiology (ESR). Position statement and best practice recommendations on the imaging use of ultrasound from the European Society of Radiology ultrasound subcommittee. Insights Imaging 2020, 11, 115. [Google Scholar] [CrossRef]
  50. Choi, D.T.; Kum, H.C.; Park, S.; Ohsfeldt, R.L.; Shen, Y.; Parikh, N.D.; Singal, A.G. Hepatocellular Carcinoma Screening Is Associated with Increased Survival of Patients with Cirrhosis. Clin. Gastroenterol. Hepatol. 2019, 17, 976–987.e4. [Google Scholar] [CrossRef]
  51. Köpping-Segerling, C.C.; Pohl, M.; Canbay, A.; Best, J. Hepatozelluläres Karzinom—Screening und Surveillance. Die Gastroenterol. 2022, 17, 387–392. [Google Scholar] [CrossRef]
  52. Weimer, A.M.; Weimer, J.M.; Berthold, S.; Stein, S.; Müller, L.; Buggenhagen, H.; Balser, G.; Stankov, K.; Sgroi, M.; Schmidmaier, G.; et al. Shoulder and Knee Arthroscopy Access Point: Prospective Comparison of Sonographic and Palpatory Detection—Which Method is Better for Novices? Ultrasound Int. Open 2024, 10, a22710098. [Google Scholar] [CrossRef]
  53. Weimer, A.M.; Weimer, J.M.; Jonck, C.; Müller, L.; Stäuber, M.; Chrissostomou, C.D.; Buggenhagen, H.; Klöckner, R.; Pirlich, N.; Künzel, J.; et al. Ultrasound supported identification of the ligamentum conicum in teaching head and neck sonography. Laryngorhinootologie 2024. [Google Scholar] [CrossRef]
  54. Moore, C.L.; Wang, J.; Battisti, A.J.; Chen, A.; Fincke, J.; Wang, A.; Wagner, M.; Raju, B.; Baloescu, C. Interobserver Agreement and Correlation of an Automated Algorithm for B-Line Identification and Quantification with Expert Sonologist Review in a Handheld Ultrasound Device. J. Ultrasound Med. 2022, 41, 2487–2495. [Google Scholar] [CrossRef]
  55. Remus, C.; Zeiger, I.; Merkel, D.; Jenssen, C.; Nurnberg, D. HHUS (Handheld Ultrasound) and AI (Artificial Intelligence)—Data Collection Projekt to Improve VScan Air CL in Abdoinal Examination. Ultrasound Med. Biol. 2022, 48, S76–S77. [Google Scholar] [CrossRef]
  56. Grobelski, J.; Recker, F.; Wilsmann-Theis, D.; Hartung, W.; Karakostas, P.; Brossart, P.; Schäfer, V.S. Establishment and validation of a didactic musculoskeletal ultrasound course for dermatologists using an innovative handheld ultrasound system—The MUDE study (Musculoskeletal Ultrasound in Dermatology). J. Dtsch. Dermatol. Ges. 2021, 19, 1753–1759. [Google Scholar] [CrossRef] [PubMed]
  57. Weimer, J.M.; Rink, M.; Vieth, T.; Lauff, J.; Weimer, A.; Müller, L.; Stäuber, M.; Reder, S.R.; Buggenhagen, H.; Bellhäuser, H.; et al. Development and evaluation of a point-of-care ocular ultrasound curriculum for medical students—A proof-of-concept study. BMC Med. Educ. 2023, 23, 723. [Google Scholar] [CrossRef] [PubMed]
  58. Neubauer, R.; Bauer, C.J.; Dietrich, C.F.; Strizek, B.; Schäfer, V.S.; Recker, F. Evidence-based Ultrasound Education?—A Systematic Literature Review of Undergraduate Ultrasound Training Studies. Ultrasound Int. Open 2024, 10, a22750702. [Google Scholar] [CrossRef]
  59. Dietrich, C.F.; Sirli, R.L.; Barth, G.; Blaivas, M.; Daum, N.; Dong, Y.; Essig, M.; Gschmack, A.M.; Goudie, A.; Hofmann, T.; et al. Student ultrasound education—Current views and controversies. Ultraschall Med. 2024, 45, 389–394. [Google Scholar] [CrossRef]
  60. Weimer, J.; Rolef, P.; Müller, L.; Bellhäuser, H.; Göbel, S.; Buggenhagen, H.; Weimer, A.; Waezsada, E.; Kirchhoff, F.; Weinmann-Menke, J. FoCUS cardiac ultrasound training for undergraduates based on current national guidelines: A prospective, controlled, single-center study on transferability. BMC Med. Educ. 2023, 23, 80. [Google Scholar] [CrossRef]
  61. Weimer, J.; Dionysopoulou, A.; Strelow, K.-U.; Buggenhagen, H.; Weinmann-Menke, J.; Dirks, K.; Weimer, A.; Künzel, J.; Börner, N.; Ludwig, M.; et al. Undergraduate ultrasound training: Prospective comparison of two different peer assisted course models on national standards. BMC Med. Educ. 2023, 23, 513. [Google Scholar] [CrossRef]
  62. Prosch, H.; Radzina, M.; Dietrich, C.F.; Nielsen, M.B.; Baumann, S.; Ewertsen, C.; Jenssen, C.; Kabaalioğlu, A.; Kosiak, W.; Kratzer, W.; et al. Ultrasound Curricula of Student Education in Europe: Summary of the Experience. Ultrasound Int. Open 2020, 6, E25–E33. [Google Scholar] [CrossRef]
  63. Recker, F.; Schäfer, V.S.; Holzgreve, W.; Brossart, P.; Petzinna, S. Development and implementation of a comprehensive ultrasound curriculum for medical students: The Bonn internship point-of-care-ultrasound curriculum (BI-POCUS). Front. Med. 2023, 10, 1072326. [Google Scholar] [CrossRef]
  64. Abrokwa, S.K.; Ruby, L.C.; Heuvelings, C.C.; Bélard, S. Task shifting for point of care ultrasound in primary healthcare in low- and middle-income countries-a systematic review. eClinicalMedicine 2022, 45, 101333. [Google Scholar] [CrossRef] [PubMed]
  65. Weimer, J.M.; Kuhn, E.; Ludwig, M.; Malle, G.L.; Kapipi, G.; Schäfer, V.S.; Sadiq, A.; Henke, O. Effectiveness of an ultrasound basic cancer training program through on-site training and virtual case discussions in rural Tanzania: A proof-of-concept study. Ecancermedicalscience 2024, 18, 1722. [Google Scholar] [CrossRef] [PubMed]
Diagnostics 14 01913 g001
Figure 1. Overview of the devices included in this study: (a): Butterfly iQ+; (b): Clarius C3HD3; (c): D5CL Microvue; (d): Philips Lumify; (e): SonoEye; (f): SonoSite iViz; (g): TE Air; (h): Vscan Air; (i): Youkey Q7.
Figure 1. Overview of the devices included in this study: (a): Butterfly iQ+; (b): Clarius C3HD3; (c): D5CL Microvue; (d): Philips Lumify; (e): SonoEye; (f): SonoSite iViz; (g): TE Air; (h): Vscan Air; (i): Youkey Q7.
Diagnostics 14 01913 g001
Diagnostics 14 01913 g002
Figure 2. Exemplary sonographic images of a liver–kidney section of the same subject with the different devices: (a): Butterfly iQ+; (b): Clarius C3HD3; (c): D5CL Microvue; (d): Philips Lumify; (e): SonoEye; (f): SonoSite iViz; (g): TE Air; (h): Vscan Air; (i): Youkey Q7. Due to the transducer ergonomics and device settings, the images are not exactly identical.
Figure 2. Exemplary sonographic images of a liver–kidney section of the same subject with the different devices: (a): Butterfly iQ+; (b): Clarius C3HD3; (c): D5CL Microvue; (d): Philips Lumify; (e): SonoEye; (f): SonoSite iViz; (g): TE Air; (h): Vscan Air; (i): Youkey Q7. Due to the transducer ergonomics and device settings, the images are not exactly identical.
Diagnostics 14 01913 g002
Diagnostics 14 01913 g003
Figure 3. Results of the evaluation of the HHUS devices in relation to the items surveyed in the main categories of B-mode image quality (a), device handling (b), software (c), and overall grade (d). The order in which the individual HHUS devices are displayed was determined by the grade achieved. The black dot indicates the mean value. X^(* = p < 0.05; ** = p < 0.01). A complete list of the p-values can be found in Supplementary Table S3.
Figure 3. Results of the evaluation of the HHUS devices in relation to the items surveyed in the main categories of B-mode image quality (a), device handling (b), software (c), and overall grade (d). The order in which the individual HHUS devices are displayed was determined by the grade achieved. The black dot indicates the mean value. X^(* = p < 0.05; ** = p < 0.01). A complete list of the p-values can be found in Supplementary Table S3.
Diagnostics 14 01913 g003
Diagnostics 14 01913 g004
Figure 4. Specification of a personal favorite after completion of all tests; the devices most frequently in 1st place (a) and 2nd place (b) are listed. The totals may not add up exactly to 100% due to rounding.
Figure 4. Specification of a personal favorite after completion of all tests; the devices most frequently in 1st place (a) and 2nd place (b) are listed. The totals may not add up exactly to 100% due to rounding.
Diagnostics 14 01913 g004
Diagnostics 14 01913 g005
Figure 5. Inter-category comparison per device with regard to B-mode quality, device handling, and software, cumulative for nine different devices. Graded from 1 (unsatisfactory) to 5 (very good).
Figure 5. Inter-category comparison per device with regard to B-mode quality, device handling, and software, cumulative for nine different devices. Graded from 1 (unsatisfactory) to 5 (very good).
Diagnostics 14 01913 g005
Diagnostics 14 01913 g006
Figure 6. Results of the evaluation of the HHUS devices in relation to the items surveyed in the main categories of B-mode image quality, device handling, and software. (a): Butterfly iQ+; (b): Clarius C3HD3; (c): D5CL Microvue; (d): Philips Lumify; (e): SonoEye; (f): SonoSite iViz; (g): TE Air; (h): Vscan Air; (i): Youkey Q7 (* p < 0.05; ** p < 0.01; **** p < 0.0001).
Figure 6. Results of the evaluation of the HHUS devices in relation to the items surveyed in the main categories of B-mode image quality, device handling, and software. (a): Butterfly iQ+; (b): Clarius C3HD3; (c): D5CL Microvue; (d): Philips Lumify; (e): SonoEye; (f): SonoSite iViz; (g): TE Air; (h): Vscan Air; (i): Youkey Q7 (* p < 0.05; ** p < 0.01; **** p < 0.0001).
Diagnostics 14 01913 g006
Table 1. HHUS devices requested for the study (in alphabetical order).
Table 1. HHUS devices requested for the study (in alphabetical order).
Device NameManufacturerCity/CountryImage TransmissionTransducer
1Alpinion
minisono *		Alpinion Medical SystemsSeoul, Republic of KoreaWiredLinear or convex
2Butterfly iQ+Butterfly Network,
Inc.		Burlington, MA, USAWired“All-in-one 1.75 D-Array”
3Clarius C3HD3Clarius Mobile
Health Corp.		Burnaby, BC, CanadaCordlessLinear or convex
4D5CL MicrovueGuangzhou SonoHealth Medical Technologies Co., Ltd.Guangzhou, ChinaCordlessLinear and convex in one
5iSiniQ 30A *Prunus Medical ShenzhenShenzhen, ChinaCordlessTwo attachable probes
6Kosmos *EchoNous
EchoNous Inc.		Redmond, WA, USAWiredAll-in-one transducer
7mSonics MU 1 *Lang Sheng SozhouGuangdong, ChinaWiredConvex
8Philips LumifyKoninklijke Philips N.V.Amsterdam, The NetherlandsWiredLinear, convex, or broadband convex
9SonoEyeCHISON Medical Technologies Co., Ltd.Wuxi, ChinaWiredLinear, convex, sector, or broadband convex
10SonoSite iVizFUJIFILM CorporationTokyo, JapanCordlessLinear or convex
11Sonostar
Uprobe-C4PL *		Universal Diagnostic SolutionsVista, CA, USACordlessLinear and convex in one
12TE AirShenzhen Mindray Bio-Medical Electronics Co., Ltd.Shenzhen, ChinaCordlessSector
13Vscan AirGeneral ElectricBoston, MA, USACordlessConvex/linear or sector/linear
in one
14Youkey Q7Wuhan Youkey Bio-Medical Electronics Co., Ltd.Wuhan, ChinaCordlessFour different replaceable transducers
The devices marked with an asterisk (*) were not included in the study due to a lack of response from the manufacturer or because no devices were available at the time of the study.
Table 2. Grading of HHUS devices overall, per main item overall, and per subitem shown as average grade (±standard deviation). An additional presentation of the results as median and IQR can be found in Supplementary Table S2.
Table 2. Grading of HHUS devices overall, per main item overall, and per subitem shown as average grade (±standard deviation). An additional presentation of the results as median and IQR can be found in Supplementary Table S2.
ItemButterfly iQ+Clarius
C3HD3		Sono-Site
iViz		Philips
Lumify		D5CL
Microvue		SonoEye
Chison		TE Air
Mind-ray		Vscan
Air GE		Youkey
Q7		p-Value
All items3.6 ± 0.83.7 ± 0.83.0 ± 0.93.7 ± 0.83.7 ± 0.94.1 ± 0.83.3 ± 0.94.1 ± 0.83.7 ± 0.9<0.0001
B-mode
image quality		Overall3.0 ± 0.74.3 ± 0.73.3 ± 0.83.4 ± 0.73.2 ± 0.83.9 ± 0.72.6 ± 0.64.2 ± 0.63.7 ± 0.8<0.0001
Resolution3.0 ± 0.94.3 ± 0.73.1 ± 0.83.5 ± 0.73.2 ± 0.93.9 ± 0.82.5 ± 0.54.3 ± 0.53.7 ± 0.9<0.0001
Contrast3.0 ± 0.74.2 ± 0.73.3 ± 0.63.3 ± 0.63.3 ± 0.94.0 ± 0.72.7 ± 0.74.0 ± 0.73.5 ± 0.9<0.0001
Overall impression3.1 ± 0.74.3 ± 0.63.4 ± 0.93.5 ± 0.73.3 ± 0.83.9 ± 0.72.8 ± 0.84.3 ± 0.73.8 ± 0.7<0.0001
HandlingOverall3.6 ± 0.73.4 ± 0.83.4 ± 0.83.9 ± 0.84.4 ± 0.73.6 ± 0.93.6 ± 0.83.8 ± 0.74.0 ± 0.9<0.0001
Haptics3.5 ± 0.53.3 ± 0.73.4 ± 0.74.2 ± 0.74.6 ± 0.53.2 ± 0.73.6 ± 0.84.0 ± 0.63.9 ± 0.8<0.0001
Weight3.2 ± 0.82.9 ± 0.93.8 ± 0.74.3 ± 0.64.6 ± 0.54.2 ± 0.83.8 ± 0.93.8 ± 0.84.2 ± 0.7<0.0001
Shape3.7 ± 0.73.3 ± 0.73.1 ± 0.93.9 ± 0.74.6 ± 0.53.1 ± 0.83.5 ± 0.93.8 ± 0.83.9 ± 1.00.0005
Connectivity4.0 ± 0.63.9 ± 0.73.5 ± 0.53.3 ± 0.94.0 ± 0.74.0 ± 1.03.6 ± 0.73.7 ± 1.03.9 ± 0.80.23
Overall impression3.5 ± 0.53.7 ± 0.83.1 ± 1.03.8 ± 0.64.0 ± 0.93.8 ± 1.03.4 ± 0.94.0 ± 0.43.8 ± 1.10.21
SoftwareOverall3.8 ± 0.73.7 ± 0.82.6 ± 0.83.6 ± 0.83.4 ± 0.94.4 ± 0.63.4 ± 0.84.2 ± 0.83.6 ± 0.9<0.0001
Presets3.8 ± 0.73.9 ± 0.52.3 ± 0.83.6 ± 0.83.4 ± 0.74.5 ± 0.53.4 ± 0.94.0 ± 0.93.8 ± 0.7<0.0001
Depth3.9 ± 0.83.9 ± 0.72.7 ± 0.83.7 ± 0.72.7 ± 1.04.5 ± 0.53.5 ± 1.04.3 ± 1.13.6 ± 0.9<0.0001
Gain4.1 ± 0.53.9 ± 0.72.8 ± 0.73.7 ± 0.93.4 ± 0.84.3 ± 0.73.7 ± 1.04.4 ± 0.93.4 ± 1.10.0002
Duplex/PRF3.6 ± 1.13.3 ± 0.7-3.3 ± 0.63.4 ± 1.04.3 ± 0.73.2 ± 0.84.3 ± 0.63.5 ± 0.80.0008
Saving3.8 ± 0.63.7 ± 1.23.1 ± 0.93.8 ± 0.63.8 ± 0.74.4 ± 0.83.4 ± 0.94.0 ± 0.73.6 ± 0.90.04
Intuitiveness3.5 ± 0.73.8 ± 0.92.7 ± 0.73.7 ± 1.03.3 ± 0.94.3 ± 0.53.4 ± 0.74.5 ± 0.73.6 ± 0.8<0.0001
Overall impression 3.6 ± 0.73.8 ± 0.82.2 ± 0.93.6 ± 0.93.4 ± 0.74.4 ± 0.53.2 ± 0.64.2 ± 0.93.6 ± 0.9<0.0001
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Merkel, D.; Lueders, C.; Schneider, C.; Yousefzada, M.; Ruppert, J.; Weimer, A.; Herzog, M.; Lorenz, L.A.; Vieth, T.; Buggenhagen, H.; et al. Prospective Comparison of Nine Different Handheld Ultrasound (HHUS) Devices by Ultrasound Experts with Regard to B-Scan Quality, Device Handling and Software in Abdominal Sonography. Diagnostics 2024, 14, 1913. https://doi.org/10.3390/diagnostics14171913

AMA Style

Merkel D, Lueders C, Schneider C, Yousefzada M, Ruppert J, Weimer A, Herzog M, Lorenz LA, Vieth T, Buggenhagen H, et al. Prospective Comparison of Nine Different Handheld Ultrasound (HHUS) Devices by Ultrasound Experts with Regard to B-Scan Quality, Device Handling and Software in Abdominal Sonography. Diagnostics. 2024; 14(17):1913. https://doi.org/10.3390/diagnostics14171913

Chicago/Turabian Style

Merkel, Daniel, Christian Lueders, Christoph Schneider, Masuod Yousefzada, Johannes Ruppert, Andreas Weimer, Moritz Herzog, Liv Annebritt Lorenz, Thomas Vieth, Holger Buggenhagen, and et al. 2024. "Prospective Comparison of Nine Different Handheld Ultrasound (HHUS) Devices by Ultrasound Experts with Regard to B-Scan Quality, Device Handling and Software in Abdominal Sonography" Diagnostics 14, no. 17: 1913. https://doi.org/10.3390/diagnostics14171913

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Article metric data becomes available approximately 24 hours after publication online.
Back to TopTop