Reproducibility and Repeatability in Focus: Evaluating LVEF Measurements with 3D Echocardiography by Medical Technologists
Abstract
:1. Introduction
2. Background
3. Materials and Methods
3.1. Real-Time Three-Dimensional Transthoracic Echocardiogram
3.2. Statistical Analysis
4. Results
5. Discussion
6. Clinical Implications
7. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
Appendix A
Appendix A.1. Unlinked Data
Appendix A.2. Repeatability
Appendix A.3. Reproducibility
Appendix A.4. Reliability
Reliability Outcome | Koo et al. [37] | Cicchetti et al. [38] |
---|---|---|
Poor | <0.50 | <0.40 |
Fair | 0.50–0.75 | 0.40–0.60 |
Good | 0.75–0.90 | 0.60–0.75 |
Excellent | 0.90–1.00 | 0.75–1.00 |
References
- Herrmann, J.; Lenihan, D.; Armenian, S.; Barac, A.; Blaes, A.; Cardinale, D.; Carver, J.; Dent, S.; Ky, B.; Lyon, A.R.; et al. Defining Cardiovascular Toxicities of Cancer Therapies: An International Cardio-Oncology Society (IC-OS) Consensus Statement. Eur. Heart J. 2022, 43, 280–299. [Google Scholar] [CrossRef] [PubMed]
- Armenian, S.H.; Lacchetti, C.; Barac, A.; Carver, J.; Constine, L.S.; Denduluri, N.; Dent, S.; Douglas, P.S.; Durand, J.B.; Ewer, M.; et al. Prevention and Monitoring of Cardiac Dysfunction in Survivors of Adult Cancers: American Society of Clinical Oncology Clinical Practice Guideline. J. Clin. Oncol. 2017, 35, 893–911. [Google Scholar] [CrossRef] [PubMed]
- Plana, J.C.; Galderisi, M.; Barac, A.; Ewer, M.S.; Ky, B.; Scherrer-Crosbie, M.; Ganame, J.; Sebag, I.A.; Agler, D.A.; Badano, L.P.; et al. Expert Consensus for Multimodality Imaging Evaluation of Adult Patients during and after Cancer Therapy: A Report from the American Society of Echocardiography and the European Association of Cardiovascular Imaging. J. Am. Soc. Echocardiogr. 2014, 27, 911–939. [Google Scholar] [CrossRef] [PubMed]
- Perez, I.E.; Taveras Alam, S.; Hernandez, G.A.; Sancassani, R. Cancer Therapy-Related Cardiac Dysfunction: An Overview for The. Clin. Med. Insights. Cardiol. 2019, 13, 13. [Google Scholar] [CrossRef] [PubMed]
- Lyon, A.R.; López-Fernández, T.; Couch, L.S.; Asteggiano, R.; Aznar, M.C.; Bergler-Klein, J.; Boriani, G.; Cardinale, D.; Cordoba, R.; Cosyns, B.; et al. 2022 ESC Guidelines on Cardio-Oncology Developed in Collaboration with the European Hematology Association (EHA), the European Society for Therapeutic Radiology and Oncology (ESTRO) and the International Cardio-Oncology Society (IC-OS). Eur. Heart J. 2022, 43, 4229–4361. [Google Scholar] [CrossRef] [PubMed]
- Thavendiranathan, P.; Grant, A.D.; Negishi, T.; Plana, J.C.; Popović, Z.B.; Marwick, T.H.; Carlos Plana, J.; Popovic, Z.B.; Marwick, T.H.; Cleveland, M.; et al. Reproducibility of Echocardiographic Techniques for Sequential Assessment of Left Ventricular Ejection Fraction and Volumes: Application to Patients Undergoing Cancer Chemotherapy. J. Am. Coll. Cardiol. 2013, 61, 77–84. [Google Scholar] [CrossRef] [PubMed]
- Jenkins, C.; Bricknell, K.; Hanekom, L.; Marwick, T.H. Reproducibility and Accuracy of Echocardiographic Measurements of Left Ventricular Parameters Using Real-Time Three-Dimensional Echocardiography. J. Am. Coll. Cardiol. 2004, 44, 878–886. [Google Scholar] [CrossRef] [PubMed]
- Mor-Avi, V.; Jenkins, C.; Kühl, H.P.; Nesser, H.J.; Marwick, T.; Franke, A.; Ebner, C.; Freed, B.H.; Steringer-Mascherbauer, R.; Pollard, H.; et al. Real-Time 3-Dimensional Echocardiographic Quantification of Left Ventricular Volumes: Multicenter Study for Validation with Magnetic Resonance Imaging and Investigation of Sources of Error. JACC Cardiovasc. Imaging 2008, 1, 413–423. [Google Scholar] [CrossRef]
- Burke, L.M.B.; Bashir, M.R.; Neville, A.M.; Nelson, R.C.; Jaffe, T.A. Current Opinions on Medical Radiation: A Survey of Oncologists Regarding Radiation Exposure and Dose Reduction in Oncology Patients. J. Am. Coll. Radiol. 2014, 11, 490–495. [Google Scholar] [CrossRef]
- Fazel, R.; Krumholz, H.M.; Wang, Y.; Ross, J.S.; Chen, J.; Ting, H.H.; Shah, N.D.; Nasir, K.; Einstein, A.J.; Nallamothu, B.K. Exposure to Low-Dose Ionizing Radiation from Medical Imaging Procedures. N. Engl. J. Med. 2009, 361, 849–857. [Google Scholar] [CrossRef]
- Romero-Farina, G.; Aguadé-Bruix, S. Equilibrium Radionuclide Angiography: Present and Future. J. Nucl. Cardiol. 2021, 28, 1315–1322. [Google Scholar] [CrossRef]
- Hansen, M.N.; Haarmark, C.; Kristensen, B.; Zerahn, B. An Algorithm for Individual Dosage in Cadmium–Zinc–Telluride Spect-Gated Radionuclide Angiography. Diagnostics 2021, 11, 2268. [Google Scholar] [CrossRef] [PubMed]
- Walker, J.; Bhullar, N.; Fallah-Rad, N.; Lytwyn, M.; Golian, M.; Fang, T.; Summers, A.R.; Singal, P.K.; Barac, I.; Kirkpatrick, I.D.; et al. Role of Three-Dimensional Echocardiography in Breast Cancer: Comparison with Two-Dimensional Echocardiography, Multiple-Gated Acquisition Scans, and Cardiac Magnetic Resonance Imaging. J. Clin. Oncol. 2010, 28, 3429–3436. [Google Scholar] [CrossRef]
- Jensen, M.M.; Schmidt, U.; Huang, C.; Zerahn, B. Gated Tomographic Radionuclide Angiography Using Cadmium-Zinc-Telluride Detector Gamma Camera; Comparison to Traditional Gamma Cameras. J. Nucl. Cardiol. 2014, 21, 384–396. [Google Scholar] [CrossRef] [PubMed]
- Lertsburapa, K.; Ahlberg, A.W.; Bateman, T.M.; Katten, D.; Volker, L.; Cullom, S.J.; Heller, G.V. Independent and Incremental Prognostic Value of Left Ventricular Ejection Fraction Determined by Stress Gated Rubidium 82 PET Imaging in Patients with Known or Suspected Coronary Artery Disease. J. Nucl. Cardiol. 2008, 15, 745–753. [Google Scholar] [CrossRef] [PubMed]
- Johnson, L.L.; Verdesca, S.A.; Aude, W.Y.; Xavier, R.C.; Nott, L.T.; Campanella, M.W.; Germano, G. Postischemic Stunning Can Affect Left Ventricular Ejection Fraction and Regional Wall Motion on Post-Stress Gated Sestamibi Tomograms. J. Am. Coll. Cardiol. 1997, 30, 1641–1648. [Google Scholar] [CrossRef] [PubMed]
- Rydberg, J.; Andersen, J.; Haarmark, C.; Zerahn, B. The Influence of Anthropometric and Basic Circulatory Variables on Count Rate in Cadmium-Zinc-Telluride SPECT Gated Radionuclide Angiography. J. Nucl. Cardiol. 2019, 26, 1974–1980. [Google Scholar] [CrossRef]
- Guppy-Coles, K.B.; Prasad, S.B.; Smith, K.C.; Hillier, S.; Lo, A.; Atherton, J.J. Evaluation of Training Nurses to Perform Semi-Automated Three-Dimensional Left Ventricular Ejection Fraction Using a Customised Workstation-Based Training Protocol. J. Clin. Nurs. 2015, 24, 1479–1488. [Google Scholar] [CrossRef]
- Guppy-Coles, K.B.; Prasad, S.B.; Smith, K.C.; Lo, A.; Beard, P.; Ng, A.; Atherton, J.J. Accuracy of Cardiac Nurse Acquired and Measured Three-Dimensional Echocardiographic Left Ventricular Ejection Fraction: Comparison to Echosonographer. Heart. Lung Circ. 2020, 29, 703–709. [Google Scholar] [CrossRef] [PubMed]
- Yang, L.; Georgescu, B.; Zheng, Y.; Meer, P.; Comaniciu, D. 3D Ultrasound Tracking of the Left Ventricle Using One-Step Forward Prediction and Data Fusion of Collaborative Trackers. In Proceedings of the 2008 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Anchorage, AK, USA, 23–28 June 2008. [Google Scholar]
- Carkeet, A. A Review of the Use of Confidence Intervals for Bland-Altman Limits of Agreement in Optometry and Vision Science. Optom. Vis. Sci. 2020, 97, 3–8. [Google Scholar] [CrossRef]
- Lee, K.M.; Lee, J.; Chung, C.Y.; Ahn, S.; Sung, K.H.; Kim, T.W.; Lee, H.J.; Park, M.S. Pitfalls and Important Issues in Testing Reliability Using Intraclass Correlation Coefficients in Orthopaedic Research. Clin. Orthop. Surg. 2012, 4, 149. [Google Scholar] [CrossRef] [PubMed]
- Aune, E.; Bækkevar, M.; Rødevand, O.; Otterstad, J.E. Reference Values for Left Ventricular Volumes with Real-Time 3-Dimensional Echocardiography. Scand. Cardiovasc. J. 2010, 44, 24–30. [Google Scholar] [CrossRef] [PubMed]
- Thavendiranathan, P.; Liu, S.; Verhaert, D.; Calleja, A.; Nitinunu, A.; Van Houten, T.; De Michelis, N.; Simonetti, O.; Rajagopalan, S.; Ryan, T.; et al. Feasibility, Accuracy, and Reproducibility of Real-Time Full-Volume 3D Transthoracic Echocardiography to Measure LV Volumes and Systolic Function: A Fully Automated Endocardial Contouring Algorithm in Sinus Rhythm and Atrial Fibrillation. JACC. Cardiovasc. Imaging 2012, 5, 239–251. [Google Scholar] [CrossRef] [PubMed]
- Ababneh, A.A.; Sciacca, R.R.; Kim, B.; Bergmann, S.R. Normal Limits for Left Ventricular Ejection Fraction and Volumes Estimated with Gated Myocardial Perfusion Imaging in Patients with Normal Exercise Test Results: Influence of Tracer, Gender, and Acquisition Camera. J. Nucl. Cardiol. 2000, 7, 661–668. [Google Scholar] [CrossRef] [PubMed]
- Lang, R.M.; Badano, L.P.; Mor-Avi, V.; Afilalo, J.; Armstrong, A.; Ernande, L.; Flachskampf, F.A.; Foster, E.; Goldstein, S.A.; Kuznetsova, T.; et al. Recommendations for Cardiac Chamber Quantification by Echocardiography in Adults: An Update from the American Society of Echocardiography and the European Association of Cardiovascular Imaging. Eur. Heart J.-Cardiovasc. Imaging 2015, 16, 233–271. [Google Scholar] [CrossRef]
- de Geus-Oei, L.F.; Mavinkurve-Groothuis, A.M.C.; Bellersen, L.; Gotthardt, M.; Oyen, W.J.G.; Kapusta, L.; van Laarhoven, H.W.M. Scintigraphic Techniques for Early Detection of Cancer Treatment-Induced Cardiotoxicity. J. Nucl. Med. Technol. 2013, 41, 170–181. [Google Scholar] [CrossRef] [PubMed]
- Sonaglioni, A.; Nicolosi, G.L.; Granato, A.; Bonanomi, A.; Rigamonti, E.; Lombardo, M. Influence of Chest Wall Conformation on Reproducibility of Main Echocardiographic Indices of Left Ventricular Systolic Function. Minerva Cardiol. Angiol. 2024, 72, 111–124. [Google Scholar] [CrossRef] [PubMed]
- Julious, S.A. Sample Size of 12 per Group Rule of Thumb for a Pilot Study. Pharm. Stat. 2005, 4, 287–291. [Google Scholar] [CrossRef]
- Bartlett, J.W.; Frost, C. Reliability, Repeatability and Reproducibility: Analysis of Measurement Errors in Continuous Variables. Ultrasound Obstet. Gynecol. 2008, 31, 466–475. [Google Scholar] [CrossRef]
- Bland, J.M.; Altman, D.G. Agreement between Methods of Measurement with Multiple Observations per Individual. J. Biopharm. Stat. 2007, 17, 571–582. [Google Scholar] [CrossRef]
- Jones, M.; Dobson, A.; O’brian, S. A Graphical Method for Assessing Agreement with the Mean between Multiple Observers Using Continuous Measures. Int. J. Epidemiol. 2011, 40, 1308–1313. [Google Scholar] [CrossRef] [PubMed]
- Christensen, H.S.; Borgbjerg, J.; Børty, L.; Bøgsted, M. On Jones et Al.’s Method for Extending Bland-Altman Plots to Limits of Agreement with the Mean for Multiple Observers. BMC Med. Res. Methodolody 2020, 20, 304. [Google Scholar] [CrossRef] [PubMed]
- Carstensen, B. Comparing Clinical Measurement Methods; Wiley: Hoboken, NJ, USA, 2010; ISBN 9780470694237. [Google Scholar]
- Choudhary, P.K.; Nagaraja, H.N. Measuring Agreement: Models, Methods and Applications; Wiley Series in Probability and Statistics; Wiley: Hoboken, NJ, USA, 2017; ISBN 9781118078587. [Google Scholar]
- BS5497-1; Precision of Test Methods—Guide for the Determination of Repeatability and Reproducibility for a Standard Test Method. British Standards Institution: London, UK, 1979.
- Koo, T.K.; Li, M.Y. A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research. J. Chiropr. Med. 2016, 15, 155. [Google Scholar] [CrossRef] [PubMed]
- Cicchetti, D.V.; Sparrow, S.A. Developing Criteria for Establishing Interrater Reliability of Specific Items: Applications to Assessment of Adaptive Behavior. Am. J. Ment. Defic. 1981, 86, 127–137. [Google Scholar]
Variable | Men | Women |
---|---|---|
Age | 49 (28–66) | 47 (26–62) |
Body mass index (BMI) | 23 (21–31) | 22 (20–32) |
Heart rate | 58 (45–80) | 55 (46–80) |
LVEF | 58 (35–69) | 60 (24–73) |
End-diastolic volume | 124 (97–193) | 117 (58–184) |
End-systolic volume | 53 (32–92) | 43 (19–84) |
LVEF (outliers excluded) | 58 (50–69) | 61 (49–73) |
Operator, Measurement Session | ICC * (95% Confidence Intervals) |
---|---|
OP1, M1 | 0.91 (0.79–0.97) 1 |
OP1, M2 | 0.80 (0.57–0.93) 1 |
OP2, M1 | 0.55 (0.21–0.82) 1 |
OP2, M2 | 0.75 (0.49–0.91) 1 |
Measurement Session | ICC (95% Confidence Intervals) |
---|---|
M1 | 0.03 (−0.61–0.60) |
M2 | 0.77 (0.39–0.93) 1 |
M1 * | 0.35 (−0.31–0.77) |
M2 * | 0.79 (0.45–0.94) 1 |
Variance Component | Standard Deviation (%,Total) |
---|---|
Within-subject (replicates) | 2.6 (21%) |
Between-subject | 2.7 (24%) |
Between-measurement | 2.8 (26%) |
Inter-observer | 0.9 (2%) |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Nielsen, M.Ø.; Ljoki, A.; Zerahn, B.; Jensen, L.T.; Kristensen, B. Reproducibility and Repeatability in Focus: Evaluating LVEF Measurements with 3D Echocardiography by Medical Technologists. Diagnostics 2024, 14, 1729. https://doi.org/10.3390/diagnostics14161729
Nielsen MØ, Ljoki A, Zerahn B, Jensen LT, Kristensen B. Reproducibility and Repeatability in Focus: Evaluating LVEF Measurements with 3D Echocardiography by Medical Technologists. Diagnostics. 2024; 14(16):1729. https://doi.org/10.3390/diagnostics14161729
Chicago/Turabian StyleNielsen, Marc Østergaard, Arlinda Ljoki, Bo Zerahn, Lars Thorbjørn Jensen, and Bent Kristensen. 2024. "Reproducibility and Repeatability in Focus: Evaluating LVEF Measurements with 3D Echocardiography by Medical Technologists" Diagnostics 14, no. 16: 1729. https://doi.org/10.3390/diagnostics14161729