1. Introduction
Muscle is an important tissue that constitutes part of the human body. It is densely distributed around organs and bones. Its function is to produce contractions and facilitate the body’s movement. According to its distribution and function, the human body’s muscle tissue can be divided into three categories: skeletal muscle, cardiac and smooth muscle. Skeletal muscle is attached to bones and accounts for about 40% of body weight. Its contraction and regulation have a great impact on the human body’s mobility, function and health [
1]. Studies have shown that biological tissues’ mechanical properties and pathological changes are related to morphological changes. Therefore, the morphological information of skeletal muscle has a important reference value for disease diagnosis. Accurately quantifying changes in muscle structure parameters during exercise is of great significance to understanding muscle function and judging muscle state [
2].
As a new method of non-contact sensing of lesions, ultrasound technology began to be used in medicine in the mid-1950s [
3]. Initially, ultrasound technology was used in reproductive system imaging and fetal viability monitoring, such as heart valve motion imaging and abdominal organ imaging [
4]. With the development of ultrasound imaging, it has been widely used in the diagnosis and treatment of many clinical diseases, such as skeletal muscle ultrasound images. Sonomyography (SMG) [
5] is a method used to measure muscle morphologic parameters in ultrasound images. Muscle structure parameters can be easily obtained from the SMG, such as muscle thickness, pennation angle and the length of muscle fibers. The pennation angle reflects the axial torque of aponeuroses and its sub-torques. The changes in length of muscle fibers directly reflect muscle contraction, and the changes in muscle thickness demonstrate the position and state of the muscle. These dynamic or static muscle structure parameters [
6] have important guiding significance for evaluating muscle function changes, muscle rehabilitation training and prosthetic control [
7].
Since ultrasound imaging has been widely used in clinical medicine because of its painless, portable, and inexpensive features, the data interpretation of ultrasound images has always been a pivotal task. Traditionally, the quantitative parameters of skeletal muscle ultrasound images are manually measured or annotated using interactive software, but it is time-consuming, cumbersome and error-prone. Therefore, developing a convenient and accurate method for automatic measurement of muscle ultrasound images poses a new challenge. Recently, several methods of applying image processing technology to measure and quantify muscle ultrasound image parameters have emerged [
8,
9,
10,
11,
12,
13]. Zhao et al. [
8] proposed an automatic linear extraction method based on local radon transformation (LRT) and rotation strategy to detect the pennation angle in ultrasound images. However, this method relies on the choice of edge detector. The measurement of images is often semi-automatic, and is not suitable for images containing different complex noises. Zhou et al. [
12] combined the Gabor wavelet with the Hough transform, and used prior knowledge of the geometric arrangement of fascicles and aponeuroses in ultrasound images to realize the automatic measurement of pennation angle and muscle fiber length. Although this method performs well on images with high speckle noise, it lopsidedly relies on the parameterization of some features. The selection of parameters is empirical, and there are some uncertainties in practical application. Salvi et al. [
13] proposed a method called MUSA to measure the muscle thickness on different ultrasound images of skeletal muscle. This method detects the fascicles in the middle of the aponeuroses through the Hough transform, and performs an iterative heuristic search on fascicles in the region of interest to identify contours of the aponeuroses and calculate muscle thickness. This feature detection method can locate the direction of representative fascicles. However, the speckle noise and the presence of intramuscular blood vessels tend to blur the critical features of muscle fibers, which will increase the difficulty of measurement. In addition, the MUSA method only measures the parameter of muscle thickness, which does not help to fully reflect the muscle state and structure. These image processing methods have initially solved the problem of data interpretation of muscle ultrasound images and performed well in accuracy. However, there are still various shortcomings that need to be further improved.
The advent of deep learning is undoubtedly attractive. It brings a new learning and prediction methods based on the inherent law of data. Deep learning methods have greatly improved the state of art in speech recognition, visual object recognition, object detection, and many other fields such as drug discovery and genomics [
14]. The application of deep learning in various areas has also shown great potential in clinical medical research. Shen et al. [
15] summarized the application of some deep generation models [
16] in the medical field, such as deep belief networks (DBN) and deep Boltzmann machines (DBMs) and proposed that they have been successfully applied to brain disease diagnosis [
17,
18], lesion segmentation [
19,
20], cell segmentation [
21] etc. This intelligent method can automatically acquire image features from ultrasound images through data training, effectively ’learning’ to recognize new images. Cunningham et al. [
22] first used convolutional neural networks to analyze muscle structure. They proposed a model based on deep residual networks(Resnet) and convolutional neural networks (CNN) to measure the direction and curvature of human muscle fibers. After that, the method was further improved. The deconvolution and maximum deconvolution (DCNN) were used to quantify muscle parameters [
23]. Through the comparison of various deep learning methods, the DCNN method was selected due to its remarkable muscle region localization ability. However, the difference of the pennation angle measured by this method is as high as 6 °, and there is still a lack of accuracy. Another automatic measurement method of the pennation angle based on deep learning has been proposed by Zheng et al [
24]. In this method, a reference line is introduced to help detect the direction of muscle fibers, and a deep residual network determines the relative position of the line and muscle fibers. Then the pennation angle is calculated by adjusting the line until it is parallel to the fascicles. The results show that this method can accurately measure the pennation angle, but the measurement parameter is single and requires a large number of datasets to support it. With the development of image segmentation, the proposal of U-Net [
25] makes medical image segmentation and recognition easier. J. Cronin et al. [
26] constructed an ultrasonic image analysis method based on image segmentation using U-Net. The muscle parameters are obtained by calculating the segmentation results. We identify the DL method, which realizes the automatic measurement of muscle parameters. However, this method is cumbersome due to the separate training of two networks for aponeuroses and muscle fibers. When the image has complex noise, the measurement error rate is also high.
In general, there are some problems in the above methods: (1) the measurement method is not fully automatic. It relies on manual selection of filters or adjustment of preprocessing steps; (2) the measurement results are not ideal for ultrasound images with complex noises; (3) it is difficult to obtain all the essential parameters in muscle ultrasound images solely by one method, which is not conducive to a comprehensive analysis of muscle status.
In this study, in order to amend the limitations of above methods, we propose a method with an improved neural network structure based on U-Net and deep residual shrinkage networks (DRSN), called RS-Unet, for accurate segmentation of muscle ultrasound images. Firstly, the manually labeled datasets are input into the model for training, and the trained model is used to segment the muscle ultrasound images to obtain clear and accurate results. On this basis, the segmentation results are further processed, aponeuroses and muscle fibers are linearly fitted, and the intersection points are calculated. Finally, the structure parameters of muscle ultrasound images are accurately measured, including muscle thickness, pennation angle and fascicle length. The method we proposed is a fully automatic muscle parameter detection method without any manual preprocessing steps. Experimental results show that our method produced similar results to the manual method, and has good performance in automatic and accurate measurement of muscle parameters.
The main contributions of this study are as follows:
We analyzed the existing muscle ultrasound image analysis methods, including those using image processing technology and deep learning technology, and summarized their advantages and limitations.
We propose a fully automatic muscle ultrasound image analysis method based on RS-Unet. It can analyze muscle ultrasound images with complex noise more accurately without any manual preprocessing. The parameters of muscle thickness, pennation angle and fascicle length can be easily calculated.
A new network model called RS-Unet is proposed for muscle ultrasound image segmentation. It is stable and robust. In the future, it has the potential to be applied in other ultrasonic image analysis fields.
The remaining parts of this study are as follows. In section two, we introduce the specific research methods. Section three is the experimental study of our proposed method on muscle ultrasound images. Section four is the discussion and comparison of experimental results. Section five summarizes the whole paper.