Deep Learning-Based Medical Images Segmentation of Musculoskeletal Anatomical Structures: A Survey of Bottlenecks and Strategies
Abstract
:1. Introduction
2. Materials and Methods
3. Results
3.1. Musculoskeletal Structures and Medical Imaging
3.2. The Challenges of Database Construction
3.2.1. Strategies for a Small Sample Size
3.2.2. Image Pre-Processing Techniques for Uniform Data Distribution
3.2.3. Training/Validation/Testing Subsets Assignment
3.3. Neural Network Architectures Applied to Musculoskeletal Structures Segmentation
3.4. Network Training/Validation/Testing Process
3.4.1. The Network Learning Process
3.4.2. The Network Performance
3.4.3. Post-Processing Operations
4. Discussion
5. Conclusions
Author Contributions
Funding
Conflicts of Interest
References
- Wang, P.; Vives, M.; Patel, V.M.; Hacihaliloglu, I. Robust Real-Time Bone Surfaces Segmentation from Ultrasound Using a Local Phase Tensor-Guided CNN. Int. J. Comput. Assist. Radiol. Surg. 2020, 15, 1127–1135. [Google Scholar] [CrossRef] [PubMed]
- Alsinan, A.Z.; Patel, V.M.; Hacihaliloglu, I. Automatic Segmentation of Bone Surfaces from Ultrasound Using a Filter-Layer-Guided CNN. Int. J. Comput. Assist. Radiol. Surg. 2019, 14, 775–783. [Google Scholar] [CrossRef] [PubMed]
- Kompella, G.; Antico, M.; Sasazawa, F.; Jeevakala, S.; Ram, K.; Fontanarosa, D.; Pandey, A.K.; Sivaprakasam, M. Segmentation of Femoral Cartilage from Knee Ultrasound Images Using Mask R-CNN. Proc. Annu. Int. Conf. IEEE Eng. Med. Biol. Soc. EMBS 2019, 2019, 966–969. [Google Scholar] [CrossRef] [Green Version]
- Balagopal, A.; Kazemifar, S.; Nguyen, D.; Lin, M.H.; Hannan, R.; Owrangi, A.; Jiang, S. Fully Automated Organ Segmentation in Male Pelvic CT Images. Phys. Med. Biol. 2018, 63, 245015. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Norman, B.; Pedoia, V.; Majumdar, S. Use of 2D U-Net Convolutional Neural Networks for Automated Cartilage and Meniscus Segmentation of Knee MR Imaging Data to Determine Relaxometry and Morphometry. Radiology 2018, 288, 177–185. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Torosdagli, N.; Liberton, D.K.; Verma, P.; Sincan, M.; Lee, J.S.; Bagci, U. Deep Geodesic Learning for Segmentation and Anatomical Landmarking. IEEE Trans. Med. Imaging 2019, 38, 919–931. [Google Scholar] [CrossRef] [PubMed]
- Burton, W.; Myers, C.; Rullkoetter, P. Semi-Supervised Learning for Automatic Segmentation of the Knee from MRI with Convolutional Neural Networks. Comput. Methods Programs Biomed. 2020, 189, 105328. [Google Scholar] [CrossRef]
- lo Giudice, A.; Ronsivalle, V.; Spampinato, C.; Leonardi, R. Fully Automatic Segmentation of the Mandible Based on Convolutional Neural Networks (CNNs). Orthod. Craniofacial Res. 2021, 24, 100–107. [Google Scholar] [CrossRef]
- Lahoud, P.; Diels, S.; Niclaes, L.; van Aelst, S.; Willems, H.; van Gerven, A.; Quirynen, M.; Jacobs, R. Development and Validation of a Novel Artificial Intelligence Driven Tool for Accurate Mandibular Canal Segmentation on CBCT. J. Dent. 2022, 116, 103891. [Google Scholar] [CrossRef]
- Nikan, S.; van Osch, K.; Bartling, M.; Allen, D.G.; Rohani, S.A.; Connors, B.; Agrawal, S.K.; Ladak, H.M. Pwd-3dnet: A Deep Learning-Based Fully-Automated Segmentation of Multiple Structures on Temporal Bone Ct Scans. IEEE Trans. Image Process. 2021, 30, 739–753. [Google Scholar] [CrossRef]
- Zhou, J.; Damasceno, P.F.; Chachad, R.; Cheung, J.R.; Ballatori, A.; Lotz, J.C.; Lazar, A.A.; Link, T.M.; Fields, A.J.; Krug, R. Automatic Vertebral Body Segmentation Based on Deep Learning of Dixon Images for Bone Marrow Fat Fraction Quantification. Front. Endocrinol. 2020, 11, 612. [Google Scholar] [CrossRef] [PubMed]
- Kim, D.H.; Jeong, J.G.; Kim, Y.J.; Kim, K.G.; Jeon, J.Y. Automated Vertebral Segmentation and Measurement of Vertebral Compression Ratio Based on Deep Learning in X-ray Images. J. Digit. Imaging 2021, 34, 853–861. [Google Scholar] [CrossRef] [PubMed]
- Hemke, R.; Buckless, C.G.; Tsao, A.; Wang, B.; Torriani, M. Deep Learning for Automated Segmentation of Pelvic Muscles, Fat, and Bone from CT Studies for Body Composition Assessment. Skeletal Radiol. 2020, 49, 387–395. [Google Scholar] [CrossRef]
- Lee, H.; Troschel, F.M.; Tajmir, S.; Fuchs, G.; Mario, J.; Fintelmann, F.J.; Do, S. Pixel-Level Deep Segmentation: Artificial Intelligence Quantifies Muscle on Computed Tomography for Body Morphometric Analysis. J. Digit. Imaging 2017, 30, 487–498. [Google Scholar] [CrossRef] [PubMed]
- Zopfs, D.; Bousabarah, K.; Lennartz, S.; dos Santos, D.P.; Schlaak, M.; Theurich, S.; Reimer, R.P.; Maintz, D.; Haneder, S.; Große Hokamp, N. Evaluating Body Composition by Combining Quantitative Spectral Detector Computed Tomography and Deep Learning-Based Image Segmentation. Eur. J. Radiol. 2020, 130, 109153. [Google Scholar] [CrossRef] [PubMed]
- Krishnaraj, A.; Barrett, S.; Bregman-Amitai, O.; Cohen-Sfady, M.; Bar, A.; Chettrit, D.; Orlovsky, M.; Elnekave, E. Simulating Dual-Energy X-ray Absorptiometry in CT Using Deep-Learning Segmentation Cascade. J. Am. Coll. Radiol. 2019, 16, 1473–1479. [Google Scholar] [CrossRef]
- Gao, Y.; Zhu, T.; Xu, X. Bone Age Assessment Based on Deep Convolution Neural Network Incorporated with Segmentation. Int. J. Comput. Assist. Radiol. Surg. 2020, 15, 1951–1962. [Google Scholar] [CrossRef]
- Das, P.; Pal, C.; Acharyya, A.; Chakrabarti, A.; Basu, S. Deep Neural Network for Automated Simultaneous Intervertebral Disc (IVDs) Identification and Segmentation of Multi-Modal MR Images. Comput. Methods Programs Biomed. 2021, 205, 106074. [Google Scholar] [CrossRef] [PubMed]
- Flannery, S.W.; Kiapour, A.M.; Edgar, D.J.; Murray, M.M.; Fleming, B.C. Automated Magnetic Resonance Image Segmentation of the Anterior Cruciate Ligament. J. Orthop. Res. 2021, 39, 831–840. [Google Scholar] [CrossRef]
- Nishiyama, D.; Iwasaki, H.; Taniguchi, T.; Fukui, D.; Yamanaka, M.; Harada, T.; Yamada, H. Deep Generative Models for Automated Muscle Segmentation in Computed Tomography Scanning. PLoS ONE 2021, 16, e0257371. [Google Scholar] [CrossRef]
- Hudson, M.; Martin, B.; Hagan, T.; Demuth, H.B. Deep Learning ToolboxTM User’s Guide; The MathWorks Inc.: Natick, MA, USA, 1992. [Google Scholar]
- Alsinan, A.Z.; Patel, V.M.; Hacihaliloglu, I. Bone Shadow Segmentation from Ultrasound Data for Orthopedic Surgery Using GAN. Int. J. Comput. Assist. Radiol. Surg. 2020, 15, 1477–1485. [Google Scholar] [CrossRef] [PubMed]
- Kim, P. MATLAB Deep Learning; Apress: New York, NY, USA, 2017. [Google Scholar]
- Page, M.J.; McKenzie, J.E.; Bossuyt, P.M.; Boutron, I.; Hoffmann, T.C.; Mulrow, C.D.; Shamseer, L.; Tetzlaff, J.M.; Akl, E.A.; Brennan, S.E.; et al. The PRISMA 2020 Statement: An Updated Guideline for Reporting Systematic Reviews. Syst. Rev. 2021, 89, 105906. [Google Scholar] [CrossRef]
- Gaj, S.; Yang, M.; Nakamura, K.; Li, X. Automated Cartilage and Meniscus Segmentation of Knee MRI with Conditional Generative Adversarial Networks. Magn. Reson Med. 2020, 84, 437–449. [Google Scholar] [CrossRef] [PubMed]
- Liu, F.; Zhou, Z.; Jang, H.; Samsonov, A.; Zhao, G.; Kijowski, R. Deep Convolutional Neural Network and 3D Deformable Approach for Tissue Segmentation in Musculoskeletal Magnetic Resonance Imaging. Magn. Reson Med. 2018, 79, 2379–2391. [Google Scholar] [CrossRef] [Green Version]
- Tack, A.; Mukhopadhyay, A.; Zachow, S. Knee Menisci Segmentation Using Convolutional Neural Networks: Data from the Osteoarthritis Initiative. Osteoarthr. Cartil. 2018, 26, 680–688. [Google Scholar] [CrossRef]
- Cheng, R.; Alexandridi, N.A.; Smith, R.M.; Shen, A.; Gandler, W.; McCreedy, E.; McAuliffe, M.J.; Sheehan, F.T. Fully Automated Patellofemoral MRI Segmentation Using Holistically Nested Networks: Implications for Evaluating Patellofemoral Osteoarthritis, Pain, Injury, Pathology, and Adolescent Development. Magn. Reson Med. 2020, 83, 139. [Google Scholar] [CrossRef]
- Zhou, Z.; Zhao, G.; Kijowski, R.; Liu, F. Deep Convolutional Neural Network for Segmentation of Knee Joint Anatomy. Magn. Reson Med. 2018, 80, 2759–2770. [Google Scholar] [CrossRef]
- Zeng, G.; Zheng, G. Deep Learning-Based Automatic Segmentation of the Proximal Femur from MR Images. Adv. Exp. Med. Biol. 2018, 1093, 73–79. [Google Scholar] [CrossRef]
- Byra, M.; Wu, M.; Zhang, X.; Jang, H.; Ma, Y.J.; Chang, E.Y.; Shah, S.; Du, J. Knee Menisci Segmentation and Relaxometry of 3D Ultrashort Echo Time Cones MR Imaging Using Attention U-Net with Transfer Learning. Magn. Reson Med. 2020, 83, 1109–1122. [Google Scholar] [CrossRef] [Green Version]
- Zhu, J.; Bolsterlee, B.; Chow, B.V.Y.; Cai, C.; Herbert, R.D.; Song, Y.; Meijering, E. Deep Learning Methods for Automatic Segmentation of Lower Leg Muscles and Bones from MRI Scans of Children with and without Cerebral Palsy. NMR Biomed. 2021, 34, e4609. [Google Scholar] [CrossRef]
- Cheng, R.; Crouzier, M.; Hug, F.; Tucker, K.; Juneau, P.; McCreedy, E.; Gandler, W.; McAuliffe, M.J.; Sheehan, F.T. Automatic Quadriceps and Patellae Segmentation of MRI with Cascaded U2 -Net and SASSNet Deep Learning Model. Med. Phys. 2022, 49, 443–460. [Google Scholar] [CrossRef] [PubMed]
- Li, Z.; Chen, K.; Liu, P.; Chen, X.; Zheng, G. Entropy and Distance Maps-Guided Segmentation of Articular Cartilage: Data from the Osteoarthritis Initiative. Int. J. Comput. Assist. Radiol. Surg. 2022, 17, 553–560. [Google Scholar] [CrossRef] [PubMed]
- Awan, M.J.; Rahim; Salim, N.; Rehman, A.; Garcia-Zapirain, B. Automated Knee MR Images Segmentation of Anterior Cruciate Ligament Tears. Sensors 2022, 22, 1552. [Google Scholar] [CrossRef] [PubMed]
- Deng, Y.; You, L.; Wang, Y.; Zhou, X. A Coarse-to-Fine Framework for Automated Knee Bone and Cartilage Segmentation Data from the Osteoarthritis Initiative. J Digit Imaging 2021, 34, 833–840. [Google Scholar] [CrossRef]
- Gatti, A.A.; Maly, M.R. Automatic Knee Cartilage and Bone Segmentation Using Multi-Stage Convolutional Neural Networks: Data from the Osteoarthritis Initiative. Magn. Reson. Mater. Phys. Biol. Med. 2021, 34, 859–875. [Google Scholar] [CrossRef]
- Flannery, S.W.; Kiapour, A.M.; Edgar, D.J.; Murray, M.M.; Beveridge, J.E.; Fleming, B.C. A Transfer Learning Approach for Automatic Segmentation of the Surgically Treated Anterior Cruciate Ligament. J. Orthop. Res. 2022, 40, 277–284. [Google Scholar] [CrossRef]
- Xue, Y.P.; Jang, H.; Byra, M.; Cai, Z.Y.; Wu, M.; Chang, E.Y.; Ma, Y.J.; Du, J. Automated Cartilage Segmentation and Quantification Using 3D Ultrashort Echo Time (UTE) Cones MR Imaging with Deep Convolutional Neural Networks. Eur Radiol 2021, 31, 7653–7663. [Google Scholar] [CrossRef]
- Latif, M.H.A.; Faye, I. Automated Tibiofemoral Joint Segmentation Based on Deeply Supervised 2D-3D Ensemble U-Net: Data from the Osteoarthritis Initiative. Artif. Intell. Med. 2021, 122, 102213. [Google Scholar] [CrossRef]
- Kemnitz, J.; Baumgartner, C.F.; Eckstein, F.; Chaudhari, A.; Ruhdorfer, A.; Wirth, W.; Eder, S.K.; Konukoglu, E. Clinical Evaluation of Fully Automated Thigh Muscle and Adipose Tissue Segmentation Using a U-Net Deep Learning Architecture in Context of Osteoarthritic Knee Pain. Magn. Reson. Mater. Phys. Biol. Med. 2020, 33, 483–493. [Google Scholar] [CrossRef] [Green Version]
- Perslev, M.; Pai, A.; Runhaar, J.; Igel, C.; Dam, E.B. Cross-Cohort Automatic Knee MRI Segmentation With Multi-Planar U-Nets. J. Magn. Reson. Imaging 2022, 55, 1650–1663. [Google Scholar] [CrossRef]
- Agosti, A.; Shaqiri, E.; Paoletti, M.; Solazzo, F.; Bergsland, N.; Colelli, G.; Savini, G.; Muzic, S.I.; Santini, F.; Deligianni, X.; et al. Deep Learning for Automatic Segmentation of Thigh and Leg Muscles. Magn. Reson. Mater. Phys. Biol. Med. 2022, 35, 467–483. [Google Scholar] [CrossRef] [PubMed]
- Panfilov, E.; Tiulpin, A.; Nieminen, M.T.; Saarakkala, S.; Casula, V. Deep Learning-Based Segmentation of Knee MRI for Fully Automatic Subregional Morphological Assessment of Cartilage Tissues: Data from the Osteoarthritis Initiative. J. Orthop. Res. 2022, 40, 1113–1124. [Google Scholar] [CrossRef] [PubMed]
- Felfeliyan, B.; Hareendranathan, A.; Kuntze, G.; Jaremko, J.; Ronsky, J. MRI Knee Domain Translation for Unsupervised Segmentation by CycleGAN (Data from Osteoarthritis Initiative (OAI)). In Proceedings of the Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Guadalajara, Mexico, 1–5 November 2021; pp. 4052–4055. [Google Scholar] [CrossRef]
- Ambellan, F.; Tack, A.; Ehlke, M.; Zachow, S. Automated Segmentation of Knee Bone and Cartilage Combining Statistical Shape Knowledge and Convolutional Neural Networks: Data from the Osteoarthritis Initiative. Med. Image Anal. 2019, 52, 109–118. [Google Scholar] [CrossRef] [PubMed]
- Al Chanti, D.; Duque, V.G.; Crouzier, M.; Nordez, A.; Lacourpaille, L.; Mateus, D. IFSS-Net: Interactive Few-Shot Siamese Network for Faster Muscle Segmentation and Propagation in Volumetric Ultrasound. IEEE Trans. Med. Imaging 2021, 40, 2615–2628. [Google Scholar] [CrossRef] [PubMed]
- Zhou, G.Q.; Huo, E.Z.; Yuan, M.; Zhou, P.; Wang, R.L.; Wang, K.N.; Chen, Y.; He, X.P. A Single-Shot Region-Adaptive Network for Myotendinous Junction Segmentation in Muscular Ultrasound Images. IEEE Trans. Ultrason. Ferroelectr. Freq. Control. 2020, 67, 2531–2542. [Google Scholar] [CrossRef] [PubMed]
- Antico, M.; Sasazawa, F.; Dunnhofer, M.; Camps, S.M.; Jaiprakash, A.T.; Pandey, A.K.; Crawford, R.; Carneiro, G.; Fontanarosa, D. Deep Learning-Based Femoral Cartilage Automatic Segmentation in Ultrasound Imaging for Guidance in Robotic Knee Arthroscopy. Ultrasound Med. Biol. 2020, 46, 422–435. [Google Scholar] [CrossRef]
- Chen, F.; Liu, J.; Zhao, Z.; Zhu, M.; Liao, H. Three-Dimensional Feature-Enhanced Network for Automatic Femur Segmentation. IEEE J. Biomed. Health Inf. 2019, 23, 243–252. [Google Scholar] [CrossRef]
- Wang, D.; Li, M.; Ben-Shlomo, N.; Corrales, C.E.; Cheng, Y.; Zhang, T.; Jayender, J. A Novel Dual-Network Architecture for Mixed-Supervised Medical Image Segmentation. Comput. Med. Imaging Graph. 2021, 89, 101841. [Google Scholar] [CrossRef]
- Ju, Z.; Wu, Q.; Yang, W.; Gu, S.; Guo, W.; Wang, J.; Ge, R.; Quan, H.; Liu, J.; Qu, B. Automatic Segmentation of Pelvic Organs-at-Risk Using a Fusion Network Model Based on Limited Training Samples. Acta Oncol. 2020, 59, 933–939. [Google Scholar] [CrossRef]
- Kim, Y.J.; Lee, S.R.; Choi, J.Y.; Kim, K.G. Using Convolutional Neural Network with Taguchi Parametric Optimization for Knee Segmentation from X-ray Images. Biomed. Res. Int. 2021, 2021, 5521009. [Google Scholar] [CrossRef]
- Mu, X.; Cui, Y.; Bian, R.; Long, L.; Zhang, D.; Wang, H.; Shen, Y.; Wu, J.; Zou, G. In-Depth Learning of Automatic Segmentation of Shoulder Joint Magnetic Resonance Images Based on Convolutional Neural Networks. Comput. Methods Programs Biomed. 2021, 211, 106325. [Google Scholar] [CrossRef] [PubMed]
- Medina, G.; Buckless, C.G.; Thomasson, E.; Oh, L.S.; Torriani, M. Deep Learning Method for Segmentation of Rotator Cuff Muscles on MR Images. Skelet. Radiol 2021, 50, 683–692. [Google Scholar] [CrossRef] [PubMed]
- Brui, E.; Efimtcev, A.Y.; Fokin, V.A.; Fernandez, R.; Levchuk, A.G.; Ogier, A.C.; Samsonov, A.A.; Mattei, J.P.; Melchakova, I.V.; Bendahan, D.; et al. Deep Learning-Based Fully Automatic Segmentation of Wrist Cartilage in MR Images. NMR Biomed. 2020, 33, e4320. [Google Scholar] [CrossRef]
- Conze, P.H.; Brochard, S.; Burdin, V.; Sheehan, F.T.; Pons, C. Healthy versus Pathological Learning Transferability in Shoulder Muscle MRI Segmentation Using Deep Convolutional Encoder-Decoders. Comput. Med. Imaging Graph. 2020, 83, 101733. [Google Scholar] [CrossRef]
- Kuok, C.P.; Yang, T.H.; Tsai, B.S.; Jou, I.M.; Horng, M.H.; Su, F.C.; Sun, Y.N. Segmentation of Finger Tendon and Synovial Sheath in Ultrasound Image Using Deep Convolutional Neural Network. Biomed Eng. Online 2020, 19, 1–26. [Google Scholar] [CrossRef] [Green Version]
- Folle, L.; Meinderink, T.; Simon, D.; Liphardt, A.M.; Krönke, G.; Schett, G.; Kleyer, A.; Maier, A. Deep Learning Methods Allow Fully Automated Segmentation of Metacarpal Bones to Quantify Volumetric Bone Mineral Density. Sci. Rep. 2021, 11, 9697. [Google Scholar] [CrossRef]
- Zhao, Z.; Yang, X.; Veeravalli, B.; Zeng, Z. Deeply Supervised Active Learning for Finger Bones Segmentation. Proc. Annu. Int. Conf. IEEE Eng. Med. Biol. Soc. EMBS 2020, 2020, 1620–1623. [Google Scholar] [CrossRef]
- Kuang, X.; Cheung, J.P.Y.; Wu, H.; Dokos, S.; Zhang, T. MRI-SegFlow: A Novel Unsupervised Deep Learning Pipeline Enabling Accurate Vertebral Segmentation of MRI Images. Proc. Annu. Int. Conf. IEEE Eng. Med. Biol. Soc. EMBS 2020, 2020, 1633–1636. [Google Scholar] [CrossRef]
- Li, X.; Dou, Q.; Chen, H.; Fu, C.W.; Qi, X.; Belavý, D.L.; Armbrecht, G.; Felsenberg, D.; Zheng, G.; Heng, P.A. 3D Multi-Scale FCN with Random Modality Voxel Dropout Learning for Intervertebral Disc Localization and Segmentation from Multi-Modality MR Images. Med. Image Anal. 2018, 45, 41–54. [Google Scholar] [CrossRef]
- Pang, S.; Pang, C.; Su, Z.; Lin, L.; Zhao, L.; Chen, Y.; Zhou, Y.; Lu, H.; Feng, Q. DGMSNet: Spine Segmentation for MR Image by a Detection-Guided Mixed-Supervised Segmentation Network. Med. Image Anal. 2022, 75, 102261. [Google Scholar] [CrossRef]
- Zhang, Q.; Du, Y.; Wei, Z.; Liu, H.; Yang, X.; Zhao, D. Spine Medical Image Segmentation Based on Deep Learning. J. Health Eng. 2021, 2021, 1–6. [Google Scholar] [CrossRef] [PubMed]
- Ito, S.; Mine, Y.; Yoshimi, Y.; Takeda, S.; Tanaka, A.; Onishi, A.; Peng, T.Y.; Nakamoto, T.; Nagasaki, T.; Kakimoto, N.; et al. Automated Segmentation of Articular Disc of the Temporomandibular Joint on Magnetic Resonance Images Using Deep Learning. Sci. Rep. 2022, 12, 221. [Google Scholar] [CrossRef] [PubMed]
- Mushtaq, M.; Akram, M.U.; Alghamdi, N.S.; Fatima, J.; Masood, R.F. Localization and Edge-Based Segmentation of Lumbar Spine Vertebrae to Identify the Deformities Using Deep Learning Models. Sensors 2022, 22, 1547. [Google Scholar] [CrossRef]
- Weber, K.A.; Abbott, R.; Bojilov, V.; Smith, A.C.; Wasielewski, M.; Hastie, T.J.; Parrish, T.B.; Mackey, S.; Elliott, J.M. Multi-Muscle Deep Learning Segmentation to Automate the Quantification of Muscle Fat Infiltration in Cervical Spine Conditions. Sci. Rep. 2021, 11, 16567. [Google Scholar] [CrossRef] [PubMed]
- Malinda, V.; Lee, D. Lumbar Vertebrae Synthetic Segmentation in Computed Tomography Images Using Hybrid Deep Generative Adversarial Networks. Proc. Annu. Int. Conf. IEEE Eng. Med. Biol. Soc. EMBS 2020, 2020, 1327–1330. [Google Scholar] [CrossRef]
- Ackermans, L.L.G.C.; Volmer, L.; Wee, L.; Brecheisen, R.; Sánchez-gonzález, P.; Seiffert, A.P.; Gómez, E.J.; Dekker, A.; ten Bosch, J.A.; Damink, S.M.W.O.; et al. Deep Learning Automated Segmentation for Muscle and Adipose Tissue from Abdominal Computed Tomography in Polytrauma Patients. Sensors 2021, 21, 2083. [Google Scholar] [CrossRef]
- Park, H.J.; Shin, Y.; Park, J.; Kim, H.; Lee, I.S.; Seo, D.W.; Huh, J.; Lee, T.Y.; Park, T.; Lee, J.; et al. Development and Validation of a Deep Learning System for Segmentation of Abdominal Muscle and Fat on Computed Tomography. Korean J. Radiol. 2020, 21, 88. [Google Scholar] [CrossRef]
- Graffy, P.M.; Liu, J.; Pickhardt, P.J.; Burns, J.E.; Yao, J.; Summers, R.M. Deep Learning-Based Muscle Segmentation and Quantification at Abdominal CT: Application to a Longitudinal Adult Screening Cohort for Sarcopenia Assessment. Br. J. Radiol. 2019, 92, 20190327. [Google Scholar] [CrossRef]
- Hashimoto, F.; Kakimoto, A.; Ota, N.; Ito, S.; Nishizawa, S. Automated Segmentation of 2D Low-Dose CT Images of the Psoas-Major Muscle Using Deep Convolutional Neural Networks. Radiol. Phys. Technol. 2019, 12, 210–215. [Google Scholar] [CrossRef]
- Bae, H.J.; Hyun, H.; Byeon, Y.; Shin, K.; Cho, Y.; Song, Y.J.; Yi, S.; Kuh, S.U.; Yeom, J.S.; Kim, N. Fully Automated 3D Segmentation and Separation of Multiple Cervical Vertebrae in CT Images Using a 2D Convolutional Neural Network. Comput. Methods Programs Biomed. 2020, 184, 105119. [Google Scholar] [CrossRef]
- Schmidt, D.; Ulén, J.; Enqvist, O.; Persson, E.; Trägårdh, E.; Leander, P.; Edenbrandt, L. Deep Learning Takes the Pain out of Back Breaking Work—Automatic Vertebral Segmentation and Attenuation Measurement for Osteoporosis. Clin. Imaging 2022, 81, 54–59. [Google Scholar] [CrossRef]
- Tao, R.; Liu, W.; Zheng, G. Spine-Transformers: Vertebra Labeling and Segmentation in Arbitrary Field-of-View Spine CTs via 3D Transformers. Med. Image Anal. 2022, 75, 102258. [Google Scholar] [CrossRef] [PubMed]
- Liebl, H.; Schinz, D.; Sekuboyina, A.; Malagutti, L.; Löffler, M.T.; Bayat, A.; el Husseini, M.; Tetteh, G.; Grau, K.; Niederreiter, E.; et al. A Computed Tomography Vertebral Segmentation Dataset with Anatomical Variations and Multi-Vendor Scanner Data. Sci. Data 2021, 8, 284. [Google Scholar] [CrossRef] [PubMed]
- Cheng, P.; Yang, Y.; Yu, H.; He, Y. Automatic Vertebrae Localization and Segmentation in CT with a Two-Stage Dense-U-Net. Sci. Rep. 2021, 11, 22156. [Google Scholar] [CrossRef] [PubMed]
- Nazir, A.; Cheema, M.N.; Sheng, B.; Li, P.; Li, H.; Xue, G.; Qin, J.; Kim, J.; Feng, D.D. ECSU-Net: An Embedded Clustering Sliced U-Net Coupled with Fusing Strategy for Efficient Intervertebral Disc Segmentation and Classification. IEEE Trans. Image Process. 2022, 31, 880–893. [Google Scholar] [CrossRef]
- Rehman, F.; Ali Shah, S.I.; Riaz, M.N.; Gilani, S.O.; R, F. A Region-Based Deep Level Set Formulation for Vertebral Bone Segmentation of Osteoporotic Fractures. J. Digit. Imaging 2020, 33, 191–203. [Google Scholar] [CrossRef]
- Blanc-Durand, P.; Schiratti, J.B.; Schutte, K.; Jehanno, P.; Herent, P.; Pigneur, F.; Lucidarme, O.; Benaceur, Y.; Sadate, A.; Luciani, A.; et al. Abdominal Musculature Segmentation and Surface Prediction from CT Using Deep Learning for Sarcopenia Assessment. Diagn. Interv. Imaging 2020, 101, 789–794. [Google Scholar] [CrossRef]
- Tsai, K.J.; Chang, C.C.; Lo, L.C.; Chiang, J.Y.; Chang, C.S.; Huang, Y.J. Automatic Segmentation of Paravertebral Muscles in Abdominal CT Scan by U-Net: The Application of Data Augmentation Technique to Increase the Jaccard Ratio of Deep Learning. Medicine 2021, 100, e27649. [Google Scholar] [CrossRef]
- McSweeney, D.M.; Henderson, E.G.; van Herk, M.; Weaver, J.; Bromiley, P.A.; Green, A.; McWilliam, A. Transfer Learning for Data-Efficient Abdominal Muscle Segmentation with Convolutional Neural Networks. Med. Phys. 2022, 49, 3107–3120. [Google Scholar] [CrossRef]
- Novikov, A.A.; Major, D.; Wimmer, M.; Lenis, D.; Buhler, K. Deep Sequential Segmentation of Organs in Volumetric Medical Scans. IEEE Trans. Med. Imaging 2019, 38, 1207–1215. [Google Scholar] [CrossRef] [Green Version]
- Suri, A.; Jones, B.C.; Ng, G.; Anabaraonye, N.; Beyrer, P.; Domi, A.; Choi, G.; Tang, S.; Terry, A.; Leichner, T.; et al. A Deep Learning System for Automated, Multi-Modality 2D Segmentation of Vertebral Bodies and Intervertebral Discs. Bone 2021, 149, 115972. [Google Scholar] [CrossRef] [PubMed]
- Shin, Y.R.; Han, K.; Lee, Y.H. Temporal Trends in Cervical Spine Curvature of South Korean Adults Assessed by Deep Learning System Segmentation, 2006-2018. JAMA Netw. Open 2020, 3, 2006–2018. [Google Scholar] [CrossRef] [PubMed]
- Kim, K.C.; Cho, H.C.; Jang, T.J.; Choi, J.M.; Seo, J.K. Automatic Detection and Segmentation of Lumbar Vertebrae from X-ray Images for Compression Fracture Evaluation. Comput. Methods Programs Biomed. 2021, 200, 105833. [Google Scholar] [CrossRef] [PubMed]
- al Arif, S.M.M.R.; Knapp, K.; Slabaugh, G. Fully Automatic Cervical Vertebrae Segmentation Framework for X-ray Images. Comput. Methods Programs Biomed. 2018, 157, 95–111. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Wang, W.; Feng, H.; Bu, Q.; Cui, L.; Xie, Y.; Zhang, A.; Feng, J.; Zhu, Z.; Chen, Z. MDU-Net: A Convolutional Network for Clavicle and Rib Segmentation from a Chest Radiograph. J. Health Eng 2020, 2020, 2785464. [Google Scholar] [CrossRef]
- Nozawa, M.; Ito, H.; Ariji, Y.; Fukuda, M.; Igarashi, C.; Nishiyama, M.; Ogi, N.; Katsumata, A.; Kobayashi, K.; Ariji, E. Automatic Segmentation of the Temporomandibular Joint Disc on Magnetic Resonance Images Using a Deep Learning Technique. Dentomaxillofacial Radiol. 2022, 51, 2–5. [Google Scholar] [CrossRef]
- Li, X.; Gong, Z.; Yin, H.; Zhang, H.; Wang, Z.; Zhuo, L. A 3D Deep Supervised Densely Network for Small Organs of Human Temporal Bone Segmentation in CT Images. Neural Netw. 2020, 124, 75–85. [Google Scholar] [CrossRef]
- Jaskari, J.; Sahlsten, J.; Järnstedt, J.; Mehtonen, H.; Karhu, K.; Sundqvist, O.; Hietanen, A.; Varjonen, V.; Mattila, V.; Kaski, K. Deep Learning Method for Mandibular Canal Segmentation in Dental Cone Beam Computed Tomography Volumes. Sci. Rep. 2020, 10, 5842. [Google Scholar] [CrossRef] [Green Version]
- Wang, J.; Lv, Y.; Wang, J.; Ma, F.; Du, Y.; Fan, X.; Wang, M.; Ke, J. Fully Automated Segmentation in Temporal Bone CT with Neural Network: A Preliminary Assessment Study. BMC Med. Imaging 2021, 21, 166. [Google Scholar] [CrossRef]
- Verhelst, P.J.; Smolders, A.; Beznik, T.; Meewis, J.; Vandemeulebroucke, A.; Shaheen, E.; van Gerven, A.; Willems, H.; Politis, C.; Jacobs, R. Layered Deep Learning for Automatic Mandibular Segmentation in Cone-Beam Computed Tomography. J. Dent. 2021, 114, 103786. [Google Scholar] [CrossRef]
- Wang, X.; Pastewait, M.; Wu, T.H.; Lian, C.; Tejera, B.; Lee, Y.T.; Lin, F.C.; Wang, L.; Shen, D.; Li, S.; et al. 3D Morphometric Quantification of Maxillae and Defects for Patients with Unilateral Cleft Palate via Deep Learning-Based CBCT Image Auto-Segmentation. Orthod Craniofacial Res. 2021, 24, 108–116. [Google Scholar] [CrossRef] [PubMed]
- Le, C.; Deleat-Besson, R.; Prieto, J.; Brosset, S.; Dumont, M.; Zhang, W.; Cevidanes, L.; Bianchi, J.; Ruellas, A.; Gomes, L.; et al. Automatic Segmentation of Mandibular Ramus and Condyles. In Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS, Guadalajara, Mexico, 1–5 November 2021; IEEE: New York, NY, USA; pp. 2952–2955.
- Neves, C.A.; Tran, E.D.; Blevins, N.H.; Hwang, P.H. Deep Learning Automated Segmentation of Middle Skull-Base Structures for Enhanced Navigation. Int. Forum Allergy Rhinol. 2021, 11, 1694–1697. [Google Scholar] [CrossRef]
- Hamwood, J.; Schmutz, B.; Collins, M.J.; Allenby, M.C.; Alonso-Caneiro, D. A Deep Learning Method for Automatic Segmentation of the Bony Orbit in MRI and CT Images. Sci. Rep. 2021, 11, 13693. [Google Scholar] [CrossRef] [PubMed]
- Kats, L.; Vered, M.; Blumer, S.; Kats, E. Neural Network Detection and Segmentation of Mental Foramen in Panoramic Imaging. J. Clin. Pediatr. Dent. 2020, 44, 168–173. [Google Scholar] [CrossRef]
- González Sánchez, J.C.; Magnusson, M.; Sandborg, M.; Carlsson Tedgren, Å.; Malusek, A. Segmentation of Bones in Medical Dual-Energy Computed Tomography Volumes Using the 3D U-Net. Phys. Med. 2020, 69, 241–247. [Google Scholar] [CrossRef] [PubMed]
- Zaman, A.; Park, S.H.; Bang, H.; woo Park, C.; Park, I.; Joung, S. Generative Approach for Data Augmentation for Deep Learning-Based Bone Surface Segmentation from Ultrasound Images. Int. J. Comput. Assist. Radiol. Surg. 2020, 15, 931–941. [Google Scholar] [CrossRef]
- Marzola, F.; van Alfen, N.; Doorduin, J.; Meiburger, K.M. Deep Learning Segmentation of Transverse Musculoskeletal Ultrasound Images for Neuromuscular Disease Assessment. Comput. Biol. Med. 2021, 135, 104623. [Google Scholar] [CrossRef] [PubMed]
- Kamiya, N. Muscle Segmentation for Orthopedic Interventions. Adv. Exp. Med. Biol. 2018, 1093, 81–91. [Google Scholar] [CrossRef] [PubMed]
- Klein, A.; Warszawski, J.; Hillengaß, J.; Maier-Hein, K.H. Automatic Bone Segmentation in Whole-Body CT Images. Int. J. Comput. Assist. Radiol. Surg. 2019, 14, 21–29. [Google Scholar] [CrossRef]
- Hiasa, Y.; Otake, Y.; Takao, M.; Ogawa, T.; Sugano, N.; Sato, Y. Automated Muscle Segmentation from Clinical CT Using Bayesian U-Net for Personalized Musculoskeletal Modeling. IEEE Trans. Med. Imaging 2020, 39, 1030–1040. [Google Scholar] [CrossRef] [Green Version]
- Lee, Y.S.; Hong, N.; Witanto, J.N.; Choi, Y.R.; Park, J.; Decazes, P.; Eude, F.; Kim, C.O.; Chang Kim, H.; Goo, J.M.; et al. Deep Neural Network for Automatic Volumetric Segmentation of Whole-Body CT Images for Body Composition Assessment. Clin. Nutr. 2021, 40, 5038–5046. [Google Scholar] [CrossRef] [PubMed]
- Brown, R.A.; Fetco, D.; Fratila, R.; Fadda, G.; Jiang, S.; Alkhawajah, N.M.; Yeh, E.A.; Banwell, B.; Bar-Or, A.; Arnold, D.L. Deep Learning Segmentation of Orbital Fat to Calibrate Conventional MRI for Longitudinal Studies. Neuroimage 2020, 208, 116442. [Google Scholar] [CrossRef] [PubMed]
- Deniz, C.M.; Xiang, S.; Hallyburton, R.S.; Welbeck, A.; Babb, J.S.; Honig, S.; Cho, K.; Chang, G. Segmentation of the Proximal Femur from MR Images Using Deep Convolutional Neural Networks. Sci. Rep. 2018, 8, 16485. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Dey, N. Uneven Illumination Correction of Digital Images: A Survey of the State-of-the-Art. Optik 2019, 183, 483–495. [Google Scholar] [CrossRef]
- Chaki, J.; Dey, N. A Beginner’s Guide to Image Preprocessing Techniques; CRC Press: Boca Raton, FL, USA, 2018. [Google Scholar]
- Chakraborty, S.; Chatterjee, S.; Ashour, A.S.; Mali, K.; Dey, N. Intelligent Computing in Medical Imaging: A Study. In Advancements in Applied Metaheuristic Computing; IGI Global: Hershey, PA, USA, 2018; pp. 143–163. [Google Scholar] [CrossRef]
- Goodfellow, I.J.; Bengio, Y.; Courville, A. Deep Learning; MIT Press: Cambridge, MA, USA, 2016. [Google Scholar]
- Rampun, A.; López-Linares, K.; Morrow, P.J.; Scotney, B.W.; Wang, H.; Ocaña, I.G.; Maclair, G.; Zwiggelaar, R.; González Ballester, M.A.; Macía, I. Breast Pectoral Muscle Segmentation in Mammograms Using a Modified Holistically-Nested Edge Detection Network. Med. Image Anal. 2019, 57, 1–17. [Google Scholar] [CrossRef]
- Luo, C.; Shi, C.; Li, X.; Gao, D. Cardiac MR Segmentation Based on Sequence Propagation by Deep Learning. PLoS ONE 2020, 15, e0230415. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Ronneberger, O.; Fischer, P.; Brox, T. U-Net: Convolutional Networks for Biomedical Image Segmentation; Springer: Cham, Switzerland, 2015. [Google Scholar]
- Kervadec, H.; Dolz, J.; Tang, M.; Granger, E.; Boykov, Y.; ben Ayed, I. Constrained-CNN Losses for Weakly Supervised Segmentation. Med. Image Anal. 2019, 54, 88–99. [Google Scholar] [CrossRef] [Green Version]
- Tan, L.K.; McLaughlin, R.A.; Lim, E.; Abdul Aziz, Y.F.; Liew, Y.M. Fully Automated Segmentation of the Left Ventricle in Cine Cardiac MRI Using Neural Network Regression. J. Magn. Reson. Imaging 2018, 48, 140–152. [Google Scholar] [CrossRef]
Anatomical Structures | Medical Imaging | References |
---|---|---|
Lower Limb | MRI | [5,7,19,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46] |
US | [3,47,48,49] | |
CT | [4,50,51,52] | |
X-ray | [53] | |
Upper Limb | MRI | [54,55,56,57] |
US | [58,59] | |
X-ray | [17,60] | |
Trunk | MRI | [11,18,61,62,63,64,65,66,67] |
CT | [14,15,16,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83] | |
MRI, CT, X-ray | [84] | |
X-ray | [12,85,86,87,88] | |
Head | MRI | [89] |
CT | [6,8,9,10,90,91,92,93,94,95,96] | |
CT, MRI | [97] | |
X-ray | [98] | |
Pelvis | CT | [13,20,99] |
Whole body | US | [1,2,100,101] |
CT | [102,103,104,105] |
Computational Solution | Type | References |
---|---|---|
Data Augmentation | Affine transformations | [9,13,20,27,29,31,32,37,39,40,41,42,43,50,51,52,53,54,55,57,58,63,64,66,67,70,72,79,80,81,82,84,85,86,87,88,90,91,93,96,97,98,99,101,103,104,107] |
Transfer Learning | - | [8,31,38,39,44,45,48,57,82,88,96] |
Computational Solution | References |
---|---|
Normalization/histogram equalization | [1,12,13,15,17,18,19,25,26,28,31,32,33,39,40,55,59,60,61,63,70,73,80,85,86,90,91,104] |
Intensity-based/dimensional-based filtering | [3,6,12,17,28,54,70,86] |
Network Architecture | Medical Imaging | Reference |
---|---|---|
U-Net | MRI | [5,7,11,19,25,27,31,32,33,34,35,36,37,38,39,40,41,42,44,46,55,57,64,65,89,107] |
US | [49,100,101] | |
CT | [4,9,13,15,16,59,69,71,72,73,76,77,78,79,80,81,83,90,91,93,94,95,99,103,104,105] | |
X-ray | [12,17,53,60,85,86,87,88,98] |
Loss Function | Reference |
---|---|
DICE function or related variants | [4,7,8,10,12,15,19,31,37,38,39,46,49,50,52,55,57,59,67,72,73,80,85,91,94,99,101] |
Cross entropy or variants | [1,2,4,5,9,26,28,29,32,33,36,43,50,51,71,74,87,88,93,97,100,105,107] |
A combination of DICE + cross entropy loss function | [25,35,40,48,82,90,103] |
Performance Indicators | Reference |
---|---|
DSC | [3,4,5,6,7,8,9,10,11,12,13,14,15,18,19,20,25,27,28,29,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,62,63,67,68,69,70,71,73,76,77,78,79,80,82,83,84,86,87,88,89,90,91,92,93,94,96,97,98,99,100,101,103,104,105,107] |
HD | [4,6,10,18,28,34,41,44,47,48,52,58,68,73,86,90,91,96] |
IoU | [2,3,7,9,10,11,18,28,33,34,35,47,48,50,52,57,59,67,68,72,73,77,78,81,85,88,89,93,101,103] |
SD | [4,18,26,29,32,34,37,40,41,44,46,51,73,79,90,91,104,107] |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Bonaldi, L.; Pretto, A.; Pirri, C.; Uccheddu, F.; Fontanella, C.G.; Stecco, C. Deep Learning-Based Medical Images Segmentation of Musculoskeletal Anatomical Structures: A Survey of Bottlenecks and Strategies. Bioengineering 2023, 10, 137. https://doi.org/10.3390/bioengineering10020137
Bonaldi L, Pretto A, Pirri C, Uccheddu F, Fontanella CG, Stecco C. Deep Learning-Based Medical Images Segmentation of Musculoskeletal Anatomical Structures: A Survey of Bottlenecks and Strategies. Bioengineering. 2023; 10(2):137. https://doi.org/10.3390/bioengineering10020137
Chicago/Turabian StyleBonaldi, Lorenza, Andrea Pretto, Carmelo Pirri, Francesca Uccheddu, Chiara Giulia Fontanella, and Carla Stecco. 2023. "Deep Learning-Based Medical Images Segmentation of Musculoskeletal Anatomical Structures: A Survey of Bottlenecks and Strategies" Bioengineering 10, no. 2: 137. https://doi.org/10.3390/bioengineering10020137
APA StyleBonaldi, L., Pretto, A., Pirri, C., Uccheddu, F., Fontanella, C. G., & Stecco, C. (2023). Deep Learning-Based Medical Images Segmentation of Musculoskeletal Anatomical Structures: A Survey of Bottlenecks and Strategies. Bioengineering, 10(2), 137. https://doi.org/10.3390/bioengineering10020137