An Improved Level Set Algorithm Based on Prior Information for Left Ventricular MRI Segmentation

Abstract

This paper proposes a new level set algorithm for left ventricular segmentation based on prior information. First, the improved U-Net network is used for coarse segmentation to obtain pixel-level prior position information. Then, the segmentation result is used as the initial contour of level set for fine segmentation. In the process of curve evolution, based on the shape of the left ventricle, we improve the energy function of the level set and add shape constraints to solve the “burr” and “sag” problems during curve evolution. The proposed algorithm was successfully evaluated on the MICCAI 2009: the mean dice score of the epicardium and endocardium are 92.95% and 94.43%. It is proved that the improved level set algorithm obtains better segmentation results than the original algorithm.

1. Introduction

Uremic cardiomyopathy is the most common complication also the cause of death with chronic kidney disease and left ventricular hypertrophy is the most significant pathological feature of uremic cardiomyopathy [1]. Therefore, it is of great significance for the prevention and treatment of uremic diseases to segment left ventricle from medical images and analyze its pathology scientifically, objectively and quantitatively. Heart and other soft tissue images have low contrast with the background and high noise [2], so segmentation of left ventricle has always been a difficult problem in the field of image segmentation. In recent years, image segmentation technology based on deep convolution neural network is widely used in various medical image segmentation, such as MRI, CT and X-ray. Comelli et al. [3] had proved that using deep learning to assist medical segmentation can not only improve the accuracy of diagnosis result, but also improve the management of patients towards personalized risk strategies. However, fully convolution network (FCN), U-Net and other segmentation models of deep convolution neural network are pixel level segmentation [4]. For some medical images with sub-pixel segmentation accuracy, there are still some shortcomings. The level set algorithm based on curve evolution and contour fitting can achieve sub-pixel segmentation effect and the segmentation result is more accurate. However, the level set algorithm needs to set the initial contour artificially. Because the contour of tissues and organs in medical images is fuzzy, it is difficult to achieve pixel level accuracy by human calibration of the initial contour, which is easy to cause evolution error. Moreover, for the sequential medical images, each layer of tissue section is to calibrate the initial contour again, which undoubtedly increases the workload of doctors. Therefore, this paper proposes a level set algorithm based on prior information to segment the left ventricle. We use a convolution neural network to extract the deep information of the image and provide pixel level initial contour for the region to be segmented. Then, we use a level set algorithm based on prior shape constraint to segment the left ventricle in detail and obtain sub-pixel level segmentation results. The experimental results on MICCAI 2009 dataset show that our segmentation algorithm is better than other segmentation algorithms.

2. Related Works

Since Osher et al. [5] proposed the level set algorithm, it has made great achievements in the field of medical image segmentation. The level set algorithm mainly considers how to fuse the image information into the construction of energy functional, so as to segment the image effectively. The Mumford-Shah (MS) [6] model was proposed by Mumford and Shah, which aimed at image segmentation by minimizing the energy function. Because of the smooth term and regular term in the model, the segmentation result was smoother and more accurate, but the solution of the model was more complex. Chan and Vese simplify the solution process of the MS model by improving the smoothing term, which was the famous CV model [7]. Although CV model solved the defects of MS model, it needed to re-initialize the contour in the iterative process of CV model. Li et al. [8] proposed a level set model without reinitialization, which further improved the wide application of level set algorithm in medical image segmentation. In order to further solve the problem of incorrect moving of contour line and error segmentation result caused by weak boundary and uneven gray level in medical image, a series of excellent models were proposed to make level set have higher anti-noise performance in medical image segmentation. In recent years, with the improvement of computational power, convolutional neural network had been gradually applied to medical image segmentation. FCN was first proposed for image segmentation, FCN also achieved good segmentation results in medical images [9]. Subsequently, U-Net innovatively proposed up sampling and feature fusion technology, which had been widely recognized in medical image segmentation [10]. On the basis of U-Net, many had proposed the improved segmentation network [11,12,13].

Although these neural networks can obtain more accurate segmentation results, due to the limited number of specific data sets and the segmentation results were not smooth enough, as we can see from Figure 1, neural networks still cannot obtain sub-pixel segmentation results in actual medical image segmentation. Recently, some scholars had tried to combine level set algorithm with neural network to get better segmentation results. Kim et al. [14] proposed that the energy function of level set was directly used as the loss function of neural network. This method effectively used the contour information of the target area to be segmented and made up for the shortcomings of neural network. Hatamizadeh et al. [15] also used neural network to learn the energy function of level set. The difference is that they added distance regularization change to initialize the level set contour, so they obtained better segmentation results. Based on this idea, Kim [16] proposed a semi supervised segmentation method, using the energy function of the level set as the loss function and using some unlabeled data for training, which also obtained good segmentation results. Chen et al. [17] used the length regular term and area regular term in the level set as constraints to modify the segmentation results of neural network, but only took the length and area of the target region as constraints, without considering the single integrity of the target region, it is easy to cause error segmentation. The above method of applying level set energy function to convolutional neural network is still the result of pixel level segmentation in essence, which cannot reach sub-pixel level. Recently, Comelli [18] added the classification results of machine learning as constraints to the energy function of level set to segment medical images and achieved good results. Thus, the combination of neural network and level set algorithm is not only feasible, but also may get better results.

3. Proposed Method for Left Ventricular Segmentation

In this paper, we propose a level set algorithm based on location and shape prior information to segment left ventricular endocardium and epicardium. It is mainly divided into two steps (as shown in Figure 2): (1) First, we train an improved U-Net to segment the left ventricular endocardium and epicardium in this step, the U-Net can segment the approximate position of them. We name the trained network a priori network and the output result a priori position map. (2) Using the prior network trained in (1), the unlabeled heart image is roughly segmented to obtain the approximate location of the left ventricle. Through the output prior position map, we can calculate the initial contour coordinates needed for the level set evolution; in order to solve the problems of uneven gray level and fuzzy boundary of medical image, we weighted the prior position map and the original image to enhance the gradient information of the target region, so as to facilitate the curve evolution. Based on the prior condition that the left ventricle is approximately circular, we added a shape constraint to the energy function of the level set to drive the curve to fit into a circle, which solves the problems of “burr” and “sag” in the level set. Next, we will introduce our algorithm in detail.

3.1. The Segmentation Network Provides the Initial Contour

There are two ways of using deep convolution neural network to provide prior position information for level set algorithm. (1) Using the object detection network [19,20,21], the coordinate information of the upper left corner and the lower right corner of the detected target is given, so as to obtain the initial contour required by the level set algorithm; (2) using the segmentation network to segment the approximate location area of the target and then on the basis of this area, the initial contour of the level set algorithm is obtained and then the curve evolution is carried out. We find that the initial positioning error of target detection network is larger than that of human calibration. The results are shown in the following Figure 3.

After analysis, it can be concluded that the essence of object detection is to use network learning parameters to fit the artificially labeled object region. Because this kind of fitting has error information, so the learned position information also has error. For the segmentation of left ventricle which requires high accuracy, the initial position cannot reach the pixel level accuracy because of the error of the ground truth manually labeled, so it is more difficult to locate the initial contour by using the network to fit the position information manually labeled. Therefore, we exclude the use of detection network to provide initial contour for level set evolution.

For the output of segmentation network, the output is pixel level classification [9]. We can get the approximate location region of the target at the pixel level from the output of the segmentation network and then use some processing methods to easily obtain the initial contour at the pixel level for the evolution of the level set. In this paper, the output of the segmentation network is used as the prior position information, which is used to initialize the level set contour for curve evolution segmentation of left ventricular endocardium and epicardium. Next, we will introduce the details of our improved segmentation network and initial contour post-processing algorithm.

We can see from Figure 4a, U-Net is named because its network structure presents a “U” shape. On the left side of the network is the encoder, which is used to down sample the image; on the right side is the decoder, which is used to up sample the image to recover the original image size. The encoder has four sub modules, each sub module contains two convolution layers, followed by a maximum pooling layer for down sampling. The decoder also includes four sub modules and the image resolution is increased by up sampling until it is consistent with the resolution of the input image. In U-Net network, jump connection is used to connect the up-sampling result in decoder with the down-sampling sub module with the same output size in encoder and it is used as the input of the next sub module in decoder.

We set the input image size as 256 × 256. After four down sampling, the final output size of the encoding module is 16 × 16. After four times of up sampling, the image size can be restored to 256 × 256. After the last feature extraction layer, SoftMax is added for classification. Outputs are divided into three categories, which correspond to background, ventricle and myocardial wall, respectively. We use cross entropy as the loss function of the network. In order to deepen the number of network layers and enhance the ability of network to extract image features, we replace the convolution module in the original U-Net with the ResNet block [22], as is shown in Figure 4b. In the jump connection part of the network, we add the attention mechanism based on spatial region information proposed by Oktay et al. [23], which is Figure 4c. This attention mechanism can learn to suppress irrelevant areas and focus on useful salient features during training. At the same time, it can greatly improve the accuracy of segmentation. The final segmentation results are shown in Figure 5. It can be seen from the figure that the results of the improved segmentation network are closer to the ground truth.

By calculating the coordinates of the upper left corner and the lower right corner of the prior location map, we can get the location coordinates of the left ventricular epicardium and endocardium. If the coordinate point information is directly mapped to the original image for level set evolution, its accuracy will be affected. Because of the characteristics of medical image such as uneven gray level and fuzzy boundary of object contour, the level set algorithm based on gradient information evolution will be interfered. Therefore, further processing is needed to reduce the interference area in the original image and enhance the gradient of the target area. In order to enhance the image information of the target area and reduce the interference of the tissue around the target, we add the pixel value of the original image and the result of the network segmentation and then map the coordinate information to the weighted image, so as to get the coordinate information of the left ventricular endocardium and epicardium after processing. The weighting formula is as follows:

$$I_1+\left(1-\alpha \right)I_2=I_3$$

(1)

where $I_1$ is the prior location image segmented by U-Net, $I_2$ is the original image; α is the weighting factor, in order to ensure that the weighted pixel value is between 0 and 255, α $\in$ [0,1]. After testing different α values, as shown in Figure 6, we find that when α ≤ 0.3, the original image is seriously distorted; when α ≥ 0.7, the gray level of the ventricular edge of the superimposed image is uneven; finally, α = 0.5 is selected as the weighting factor.

3.2. Shape Constraint Contour Evolution

In this section, we will introduce the improved level set algorithm with shape constraint. We choose the level set model proposed by Li [8] as our basic model, because it is a level set model without re-initialization, which simplifies the steps of curve evolution. On the basis of this model, we add shape constraints to carry out curve evolution.

$$E\left(\varphi \right)=\mu P\left(\varphi \right)+\upsilon {𝓔}_{g,\lambda ,\nu }\left(\varphi \right)$$

(2)

where $\varphi$ is the signed distance function, $\mu >0$ and $\upsilon \mathrm{are} \mathrm{constants}$. $P\left(\varphi \right)$ is the distance regular term, which forces the level set function to be close to the signed distance function and therefore completely eliminates the need of the costly re-initialization procedure.

$$P\left(\varphi \right)={{\displaystyle \int }}_{\Omega }^0\frac{1}{2}{\left(\left|\nabla \varphi \right|-1\right)}^{2}d\Omega$$

(3)

$\mathrm{where} {𝓔}_{g,\lambda ,\nu }\left(\varphi \right)$ is the energy function to drive the curve evolution, which is defined by

$${𝓔}_{g,\lambda ,\nu }\left(\varphi \right)=\lambda {\mathcal{L}}_g\left(\varphi \right)+\upsilon {\mathcal{F}}_g\left(\varphi \right)$$

(4)

where $\lambda >0$, $\upsilon \mathrm{is} \mathrm{a} \mathrm{constant}$, ${\mathcal{L}}_g\left(\varphi \right)$ and ${\mathcal{F}}_g\left(\varphi \right)$ are respectively defined by

$${\mathcal{F}}_g\left(\varphi \right)=\int gH\left(-\varphi \right)d\Omega$$

(5)

$${\mathcal{L}}_g\left(\varphi \right)=\int g\delta \left(\varphi \right) \left|\nabla \varphi \right|d\Omega$$

(6)

$g$ is the edge detection function defined by

$$g=\frac{1}{1+\left|\nabla G_{\sigma }\ast I\right|}$$

(7)

where $G_{\sigma }$ is the Gaussian kernel with standard deviation $\sigma$.

$H$ is the Heaviside function, which defined by

$$H\left(x\right)=\{\begin{array}{c}1, x\ge 0\\ 0, x<0\end{array}$$

(8)

$\delta$ is the Dirac function, in application, it always defined by

$${\delta }_{ϵ}\left(x\right)=\{\begin{array}{cc}0,& \left|x\right|>ϵ\\ \frac{1}{2ϵ}\left[1+cos\left(\frac{\pi x}{ϵ}\right)\right],& \left|x\right|\le ϵ\end{array}$$

(9)

The above is the energy function of the traditional level set model. When segmenting the left ventricle, we find that the gradient force is too small in the low contrast region and the weak edge, the original energy function will appear “leakage” of the evolution curve, which eventually leads to the segmentation result is not smooth enough and “burr” phenomenon, just as shown in Figure 7. Therefore, based on the prior information that the left ventricular membrane is close to a circle, we propose a shape constraint term, which is used to constrain the curve in the evolution process of level set, make it fit to a circle and reduce the “burr” and “sag” of the curve.

$E_{shape}(·)$ is a priori shape constraint added by our improved algorithm. We know that the shape of the left ventricle membrane is similar to a circle, so we add a circle as the shape constraint term to the level set energy function:

$${\left(x-x_0\right)}^2+{\left(y-y_0\right)}^2=r^2$$

(10)

$$\iint \left[r^2-{\left(x-x_0\right)}^2-{\left(y-y_0\right)}^2\right]\times H(\varnothing )dxdy$$

(11)

where $x_0$ and $y_0$ are the coordinates of the center of the circle and $r$ is the radius of the circle. The center coordinates are expressed as: $x_0=\frac{x_{min}+x_{max}}{2}, y_0=\frac{y_{min}+y_{max}}{2};$ The radius of the circle is expressed as: $r^2=min\left\{{\left(x_0-x\right)}^2, {\left(y_0-y\right)}^2\right\}$; Initial time $x_{min}$ and ${\mathrm{y}}_{min}$ are the coordinates of the upper left corner of the initial contour, $x_{max}$ and ${\mathrm{y}}_{max}$ are the coordinates of the upper right corner of the initial contour. Next, with each curve evolution, the fixed center coordinates and radius of the circle will drive the curve to fit towards the minimum inscribed circle of the prior box. When Equation (11) reaches the minimum value, the fitting curve will be approximately circular. The final expression of energy function of level set is as follow:

$$\begin{array}{c}E\left(\varphi \right)=\mu {{\displaystyle \int }}_{\Omega }^0\frac{1}{2}{\left(\left|\nabla \varphi \right|-1\right)}^{2}d\Omega +\gamma \int gH\left(-\varphi \right)d\Omega +\mathsf{\lambda }\int g\delta \left(\varphi \right) \left|\nabla \varphi \right|d\Omega \\ +\iint \left[r^2-{\left(x-x_0\right)}^2-{\left(y-y_0\right)}^2\right]\times H\left(\varphi \right)dxdy\end{array}$$

(12)

Using the gradient descent flow method to solve the energy function, the curve evolution equation can be obtained as follow:

$$\begin{array}{c}\frac{\partial \varphi }{\partial t}=\mu \left[\nabla \varphi -div\left(\frac{\nabla \varphi }{\left|\nabla \varphi \right|}\right)\right]+\lambda \delta \left(\varphi \right)div\left(g\frac{\nabla \varphi }{\left|\nabla \varphi \right|}\right)+\gamma g\delta \left(\varphi \right)\\ +\delta \left(\varphi \right)\left[r^2-{\left(x-x_0\right)}^2-{\left(y-y_0\right)}^2\right]\end{array}$$

(13)

4. Experiments

4.1. Dataset

The data set used was provided by MICCAI 2009. The image data was randomly selected from the clinical database of Sunnybrook Health Science Center. All images were divided into 20 cardiac phases with time resolution. A total of 6–12 layers of short axis images were collected from the atrioventricular ring to the apex. The thickness dimension was 8–10 mm, the visual field was 320 mm × 320 mm and the matrix was 256 × 256. There were 45 cases in the whole data set, which were divided into three groups on average. There were four types of heart pictures in each group of 15 cases, namely four heart failure with ischemia (HF-I), four heart failure without ischemia (HF-NI), four cardiac hypertrophy (HYP) and three normal (N). In order to obtain a robust result and eliminate over-fitting, we use the k-fold cross validation (k = 5) to train and test our data set.

4.2. Implementation Details

In this paper, the segmentation network architecture, as shown in Figure 4, is used to segment the left ventricle to obtain the prior position information; then, the coordinates of the corresponding points are obtained by processing the segmented area and we use these coordinates to initialize the initial contours of the left ventricular endocardium and epicardium, respectively; in the level set evolution stage, we add a shape constraint term to standardize the level set evolution, which makes it fit the contour curve into a circle as much as possible. The algorithm is implemented in python and pytorch and runs on core i9-9900kf @ 3.6GHz 16GB (Sichuan, China) ram and single NVIDIA GTX 1080ti (Fujian, China) computer. In the training segmentation network stage, the random gradient descent method is selected as the optimization method of the model. The initial learning rate is 0.1 and the decay index is 0.9, which decays every two epochs. A total of 50 epochs are trained and the batch size of each iteration is 8. In the evolution stage of level set, referring to the parameter setting of curve evolution equation in ref. [8], the parameters in the curve evolution equation are λ = 5.0, μ = 0.01, ν = 3.0 and the evolution time interval Δt = 5 (they are only suggested values and can be adjusted according to the actual situation). $x_0$ and $y_0 \mathrm{are}$ determined by the prior position map and $x_0$ and $y_0$ of each image are different, $r$ is determined by the prior box of the prior position map. When the prior box is determined, the center and radius of the fitting circle are determined accordingly.

4.3. Evaluation Method

In order to measure the similarity between the segmentation results of our algorithm and the ground truth, the average perpendicular distance (APD) is used to compare the differences between the contours. The smaller the value is, the closer the contours are. We can see from the Figure 8: hypothesis $A_1$, $A_2$, $A_3$ are the contour points segmented by the algorithm; $M_1$, $M_2$, $M_3$ are the contour points of ground truth. $L_1$ is a fitting line, which determined by $A_1$, $A_2$, $A_3$; line $L_2$ perpendicular to $L$ and go through $A_2$. $L_3$ is a straight line fitted by $M_1$, $M_2$, $M_3$ three points, passing through point $A_2$ make a straight line $L_3$, intersect with D, $A_{2}D$ is the required vertical distance; the average vertical distance calculated by selecting more than one group of contour points is APD.

In order to measure the region segmented by the algorithm, we will extract the region surrounded by the left ventricular endocardium and epicardium and convert it into a binary image. We use some common criteria to measure the differences between our algorithms and other algorithms [24]. They are sensitivity, positive predictive value (PPV), Dice Score (DSC), area overlap error (AOE), relative area difference (RAD).

$$\mathrm{Sensitivity}=\frac{\mathrm{True} \mathrm{Positive}}{\mathrm{True} \mathrm{Positive}+\mathrm{False} \mathrm{Negative}}$$

(14)

$$\mathrm{PPV}=\frac{\mathrm{True} \mathrm{Positive}}{\mathrm{True} \mathrm{Positive}+\mathrm{False} \mathrm{Positive}}$$

(15)

$$\mathrm{DSC}=\frac{2\times \mathrm{True} \mathrm{Positive}}{2\times \mathrm{True} \mathrm{Positive}+\mathrm{False} \mathrm{Positive}+\mathrm{False} \mathrm{Negative}}$$

(16)

$$\mathrm{AOE}=1-\frac{\mathrm{True} \mathrm{Positive}}{\mathrm{True} \mathrm{Positive}+\mathrm{False} \mathrm{Positive}+\mathrm{False} \mathrm{Negative}}$$

(17)

$$\mathrm{RAD}=\frac{\left|\mathrm{False} \mathrm{Negative}-\mathrm{False} \mathrm{Positive}\right|}{2\times \mathrm{True} \mathrm{Positive}+\mathrm{False} \mathrm{Positive}+\mathrm{False} \mathrm{Negative}}$$

(18)

5. Results

From the perspective of segmentation contour (as shown in Figure 9): Figure 9a-1,b-1,c-1,d-1 represent the results of epicardium; Figure 9a-2,b-2,c-2,d-2 represent the results of endocardium; As can be seen from the figure below, compared with the original algorithm, the contour of the evolution curve is smoother after adding a priori condition and the phenomenon of “burr” and “sag” are obviously alleviated. Due to the use of pixel weighting, the gradient information of the image is enhanced, so the evolution problem caused by uneven gray level can be significantly reduced in the segmentation of left ventricular inner wall. In order to prove the effectiveness of the algorithm, we randomly extract some data from the test set, calculate their APD and draw them as

Table 1. APD (mm) index of different algorithms.

Subject	Epicardium		Endocardium
	DRLSE	Ours	DRLSE	Ours
HF-I-02	2.97	1.34	2.77	1.23
HF-I-06	3.08	1.41	2.96	1.31
HYP-03	4.43	1.69	3.99	1.45
HYP-05	4.11	1.98	3.71	1.76
HF-NI-15	2.88	1.32	2.61	1.13
HF-NI-21	2.90	1.24	2.69	1.01
N-01	1.97	1.07	1.29	1.15
N-03	2.12	1.17	2.09	1.21
Mean	3.05	1.40	2.76	1.28

. As can be seen from the table, compared with the APD of endocardium and epicardium of DRLSE, 3.05 mm and 2.76 mm, the APD of the proposed algorithm are 1.40 mm and 1.28 mm and its segmentation contour is closer to the contour of the ground truth, which proves that the improved algorithm is more accurate in contour fitting.

In order to further prove the effectiveness of the algorithm, this paper compares the improved algorithm with several other left ventricular segmentation algorithms and calculates the sensitivity, PPV, DSC, AOE and RAD corresponding to the average value of their segmentation results, as is shown in

Table 2. Evaluation criteria results of left ventricular epicardium.

Subject	Sensitivity (%)		PPV (%)		DSC (%)		AOE (%)		RAD (%)
	Mean	±std	Mean	±std	Mean	±std	Mean	±std	Mean	±std
LBF [25]	75.72	1.95	81.27	2.31	77.97	3.11	36.06	2.59	21.83	1.95
LCV [26]	73.95	1.62	80.35	2.97	77.88	3.38	36.12	4.60	21.92	3.43
U-Net [11]	86.87	1.47	90.84	2.37	87.78	3.53	21.62	5.66	11.71	3.86
Attention U-Net [23]	87.44	1.01	93.30	2.58	90.88	1.96	16.66	2.98	8.62	1.99
Ours	89.59	1.33	95.43	3.21	92.95	2.01	14.77	3.12	7.56	2.11

and

Table 3. Evaluation criteria results of left ventricular endocardium.

Subject	Sensitivity (%)		PPV (%)		DSC (%)		AOE (%)		RAD (%)
	Mean	±std	Mean	±std	Mean	±std	Mean	±std	Mean	±std
LBF [25]	75.08	1.43	80.63	1.99	79.53	1.66	26.95	1.43	19.41	2.01
LCV [26]	74.53	1.51	80.78	2.21	79.22	1.98	27.39	2.02	19.96	2.48
U-Net [11]	88.25	1.42	89.32	2.07	88.92	2.15	16.48	4.27	10.69	3.56
Attention U-Net [23]	90.29	1.33	94.61	2.13	93.01	1.38	12.69	1.95	7.92	2.67
Ours	91.78	1.25	95.37	2.41	94.43	1.59	10.52	2.26	6.96	2.94

. LBF [25] and LCV [26] are level set algorithm; U-Net and attention U-Net are convolutional neural network. Compared with these models (all of these are open source), the proposed algorithm has higher accuracy in segmentation of left ventricular endocardium and epicardium.

6. Conclusions

This paper proposes a level set segmentation algorithm based on prior information. Firstly, a deep learning segmentation network is trained to obtain the prior position information of the left ventricle and the output of the trained network is used as the initial contour for the level set evolution. In order to enhance the gradient information of the ventricle, we use the pixel weighting method to enhance the left ventricle contour in the original image. In the evolution stage of level set, a shape constraint is added to drive the curve to fit to a circle, so as to reduce the “burr” and “sag” problems in the curve evolution process. Finally, the improved algorithm obtains better segmentation results. Because the image operation in this paper is based on 8-bit digital image processing and the original medical image is 16-bit, so there are some errors in the conversion process. Therefore, the follow-up work of this paper will consider operating directly in the 16-bit medical image to further improve the accuracy of segmentation results.