Semi-Supervised Learning Method of U-Net Deep Learning Network for Blood Vessel Segmentation in Retinal Images

Abstract

Blood vessel segmentation methods based on deep neural networks have achieved satisfactory results. However, these methods are usually supervised learning methods, which require large numbers of retinal images with high quality pixel-level ground-truth labels. In practice, the task of labeling these retinal images is very costly, financially and in human effort. To deal with these problems, we propose a semi-supervised learning method which can be used in blood vessel segmentation with limited labeled data. In this method, we use the improved U-Net deep learning network to segment the blood vessel tree. On this basis, we implement the U-Net network-based training dataset updating strategy. A large number of experiments are presented to analyze the segmentation performance of the proposed semi-supervised learning method. The experiment results demonstrate that the proposed methodology is able to avoid the problems of insufficient hand-labels, and achieve satisfactory performance.

1. Introduction

The analysis of retinal images is helpful for detecting many diseases including ophthalmopathy, diabetes and cardiovascular disease [1]. The automatization of this analysis process is able to reduce the costs associated with human effort and avoid the problems caused by manual work. In these retinal image analysis tasks, the automated segmentation of blood vessels is the crucial part. The task of segmenting the blood vessels aims to obtain the complete blood vessel tree in the retinal image. The blood vessel tree can be used to achieve many tasks, including retinal blood vessel quantification [1], retinal image registration [2], retinal mosaic [3] and biometric identification [4].

The automated segmentation of blood vessels is a challenging task in the intelligent analysis of retinal images because the size and shape of blood vessels vary hugely in different locations [5,6]. More and more scholars have proposed improved blood vessel segmentation methods. So far, these methods can be divided into three categories: image-processing based blood vessel segmentation methods, machine-learning based blood vessel segmentation methods and deep learning based blood vessel segmentation methods.

The image-processing based blood vessel segmentation methods usually design many segmentation rules according to the features of blood vessels, and then realize the automated segmentation of the blood vessels according to the designed rules. These methods can be divided into many sub-classes, such as matched filter response, blood vessel tracking, morphological approaches, and vessel model based approaches. The matched filtering approaches calculate the matched filter response (MFR) of the retinal image, and then segment the blood vessel tree by using the corresponding MFR [7,8,9,10,11,12,13,14]. The vessel tracking approaches utilize the vessel local profile information to track and segment the blood vessels [15,16,17,18]. The morphological approaches achieve the segmentation task of the blood vessels by using the image-processing techniques such as curvature evaluation [19,20], Gaussian filter [21], vascular centerlines detection [22] and curvelet transform [23]. The model based approaches segment the blood vessels by using the explicit vessel models such as vessel profile model [24], parametric model [25,26,27,28] and geometric model [29]. In addition, the multi-scale approaches are also used in the automated segmentation of the blood vessels and achieve good results [30,31,32,33]. More detailed description can be found in the literature [1].

Different from the image-processing based methods, the machine-learning based blood vessel segmentation methods do not need the presumed rules. Instead, they turn the segmentation problem into a classification task. These methods first extract the feature of each pixel that often consists of the intensity, color, edge, region and MFRs. Then, these features are used to train the different predictors such as k-Nearest Neighbor (k-NN) [34,35], Gaussian mixture model (GMM) [36,37], support vector machine (SVM) [38,39], ensemble method [40,41], neural network [42], random forest [43,44], Extreme Learning Machine (ELM) [45] and Conditional Random Field (CRF) [46]. These methods, by introducing the discriminative features in vascular representation, are often better than the image-processing based methods. However, it is very difficult to acquire the discriminable features of vessels generally. When the discriminable features are not known, these segmentation methods are not able to extract the blood vessel tree accurately. This limits the promotion of the blood vessel segmentation method.

The deep learning theory has greatly improved on the conventional machine learning methods [47,48]. Recently, these deep learning methods have been successfully applied in the automated segmentation of blood vessels [49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67]. A more detailed description can be found in the literature [68]. In 2015, Ronneberger et al. [69] proposed the U-Net deep network, which achieved very good results in medical image segmentation. In 2018, Soomro et al. [70] used the U-Net network to segment the blood vessels and achieved satisfactory results. Therefore, we used the U-Net deep network as the base network for the blood vessel segmentation in our proposed method. However, these existing methods always need sufficient hand-labeled data to achieve the accurate segmentation of blood vessels. This hand-labeled work is difficult and slow, and requires the medical experts to manually assign each pixel an accurate label. The semi-supervised learning method provides a new way to solve these problems. More and more scholars have proposed the improved semi-supervised learning methods, which have achieved satisfactory results [71]. Inspired by these methods, a new semi-supervised method is proposed for blood vessel segmentation in retinal images. In the proposed method, we applied the U-Net, which is a famous segmentation network for medical images at present, to segment the blood vessels. Then, the dataset updating strategy was introduced to update the given training dataset.

We summarize the contributions of this paper as follows: (1) we used the improved U-Net-based blood vessel segmentation method, which is able to learn the hierarchical latent features of the given retinal image automatically, and avoid the problems caused by extracting features manually. (2) We proposed a novel deep learning based, semi-supervised learning method, which would deal with the problem of the high financial and human effort costs of labeling the retinal images. This proposed semi-supervised learning method provides a new, general, high-performance learning framework for blood vessel segmentation with limited labeled data. (3) We discussed the relationship between the number of hand-labeled retinal images and segmentation accuracy, which was helpful for reducing the costs of labeling the retinal images.

This paper is organized as follows. The prior works and our contributions are introduced in Section 1. In Section 2, we introduce the proposed blood vessel segmentation method, including the parameter learning method and dataset updating strategy. In Section 3, we present the experiment contents including the experimental setup, evaluation metrics, results comparison and convergence analysis. We conclude the paper in Section 4.

2. Proposed Method

2.1. Overview

Deep learning methods have achieved satisfactory performances in medical image segmentation. However, these methods mainly depend on the number and quality of the labeled medical images. If the labeled medical images are insufficient or low-quality, it is difficult to achieve the satisfactory segmentation result. In practice, different from the natural scene image, the task of labeling these medical images is very costly in terms of finance and human effort. In order to solve this problem, we proposed a novel semi-supervised learning method to achieve the segmentation of the blood vessel tree. The main purpose of this method is to achieve satisfactory performance by using the limited labeled retinal image.

The framework of our proposed semi-supervised learning method is shown in Figure 1. It includes two parts: parameter learning and dataset updating. In the parameter learning part, we first used a small number of labeled blood vessel data to train the improved U-Net deep neural network. Then, we used the designed dataset updating strategy to update the old dataset. The above steps would be repeated until the training accuracy met the requirement. In this method, we used a small amount of labeled data and a large number of unlabeled data to achieve the task of blood vessel tree segmentation. In the next section, the detailed information of the proposed method will be introduced.

2.2. Parameter Learning Strategy

Blood vessel segmentation is the key to realizing an automated retinal image analysis system. In order to achieve the satisfactory segmentation results, we often need to achieve complex feature engineering and select the appropriate classifier. This task is very costly in terms of both time and human effort. In order to solve this problem, we used the U-Net deep network to segment the blood vessel tree, which was able to learn the features of the given retinal image automatically, and avoid the problem caused by complex feature engineering.

The U-Net deep network contains numerous unknown parameters. The parameter learning process aims to estimate the unknown parameters by using the given training dataset. The typical U-Net consists of two parts: encoder architecture and decoder architecture. In our paper, we used an improved U-Net to segment the retinal blood vessel tree. The detailed network structure was as follows.

In the encoder part, we extracted the latent features from the given retinal images, and down-sample the resolution by using the pooling technique. The encoder part included two 3 × 3 convolutional layers and one 2 × 2 max pooling layer. Each convolutional layer was followed by the rectified linear unit (ReLU). In the down-sampling step, the number of feature channels were doubled.

In the decoder part, we reconstructed the feature map by using the up-sampling technique, and recovered the detailed information of the object by using the skip connections technique. The decoder part consisted of one 2 × 2 up-sampling layer, two 3 × 3 convolutional layers and one 1 × 1 convolutional layer. The nearest neighbor interpolation technology was used in the 2 × 2 up-sampling layer. Each 3 × 3 convolutional layer was followed by ReLU. The 1 × 1 convolution layer aimed to map each 32-component feature vector into the target category.

In addition, in order to increase the accuracy of the parameter learning method, we used the skip connection and the dropout regularization technique [72] in the training process. The skip connection technology can protect the details of the retinal image, and the dropout technology can avoid the over-fitting problem.

Let $x_i$ represent the i-th input retinal image. The loss function for training the U-Net is the standard binary cross-entropy loss [35]. The formula is defined by

$$L\left(\theta \right)=-\frac{1}{N}{{\displaystyle \sum }}_{i=1}^N(y_{i}log{f}_{\theta }\left(x_i\right)+\left(1-y_i\right)\log \left(1-f_{\theta }\left(x_i\right)\right),$$

(1)

where $f_{\theta }\left(x_i\right)$ is the predicted label of the i-th input image, and $y_i$ is the ground-truth. The U-Net model parameters $\theta$ can be estimated by solving the following optimization problem

$${\theta }^{*}=arg \underset{\theta }{\min }L\left(\theta \right).$$

(2)

The Adaptive Moment Estimation (Adam) optimization algorithm [73] was used to solve this optimization problem.

2.3. Dataset Updating Strategy

The aim of the semi-supervised learning method proposed in this paper is to obtain the satisfactory blood vessel tree by using a small amount of labeled data. In practice, we can obtain a large amount of unlabeled retinal images easily. In order to make full use of these images, we implement a dataset updating strategy, as shown in Figure 1. The dataset updating strategy aims to mark the unlabeled images automatically, and increase the number of the labeled images, which is helpful for estimating the network parameters. The specific implementation process is as follows.

Suppose that the given training dataset $D=\left\{x_n,y_n,x_m^{*}; n=1:N, m=1:M\right\}$ and $D=D_1\cup D_2$. The subset $D_1=\left\{x_n,y_n; n=1:N\right\}$ is the given labeled dataset and the subset $D_2=\left\{x_m^{*}; m=1:M\right\}$ is the given unlabeled dataset. Here, the $x_n$ represents the n-th retinal image and the $y_n$ represents the n-th ground-truth label. The $x_m^{*}$ is the m-th retinal image without ground-truth. For convenient description, we let $f_{\theta }$ represent the U-Net model, and the parameter $\theta$ is estimated by solving Equation (2). Let $y_m^{*}$ represent the predicted label of $x_m^{*}$. Then, we can obtain $y_m^{*}$ by

$$y_m^{*}=f_{\theta }\left(x_m^{*}\right).$$

(3)

Next, we can update the training dataset $D$ by

$$D^{*}=\left\{x_n,y_n,x_m^{*},y_m^{*}; n=1:N, m=1:M\right\}.$$

(4)

The updated dataset ${\mathrm{D}}^{*}$ will be used as input to recompute the U-Net model parameters $\mathsf{\theta }$. To summarize, the pseudo-code of our proposed semi-supervised learning method is shown in the Algorithm 1, as follows:

Algorithm 1 Semi-supervised learning method for the blood vessel tree segmentation

Input: Training dataset $D=\left\{x_n,y_n,x_m^{*}; n=1:N, m=1:M\right\}$, retinal image {$x^{\prime }\}$.

Output: The blood vessel tree image $y^{\prime }$.

STEP 1: Initialization:

● Learning method: epoch, batch size, learning rate, weight decay rate.

● Initial training set D.

● Initial U-Net model.

STEP 2: Update the U-Net model parameters $\mathit{\theta }$ by solving Equation (2).

STEP 3: Predict the pseudo-label ${\mathbf{y}}_{\mathbf{m}}^{*}$ using Equation (3).

STEP 4: Update training dataset ${\mathit{D}}^{*}$ using Equation (4).

STEP 5:

● while a stopping criterion is not met do

●  Update the U-Net model parameters $\theta$ using the updated training dataset $D^{*}$;

●  Return to STEP 3

● end while

STEP 6: Take ${\mathit{x}}^{\prime }$ as input and compute ${\mathit{y}}^{\prime }$ using Equation (3).

3. Experimental Results

3.1. Experimental Setup

3.1.1. Datasets

In experiments, we use the famous public dataset DRIVE, which consists of 40 retinal images and their corresponding ground-truth labels and ground-truth masks. The size of the fundus image is 565 * 584. We use 20 fundus images as the training dataset to train the improved U-Net, and 20 fundus images as the test dataset to evaluate the performance of the trained U-Net. As shown in Figure 2, (a) is a given fundus image, (b) is the ground-truth label, and (c) is the ground-truth mask.

3.1.2. Implementation Details

In the training process of U-Net, we use the fixed learning rate ${\gamma }_1$ of 0.001, exponential decay rate of first-order moment estimation ${\beta }_1$ of 0.9, exponential decay rate of second-order moment estimation ${\beta }_2$ of 0.999 in Adam algorithm. The Adam (Adaptive Moment Estimation) algorithm is used to optimize the network parameters. The training process stops when the iteration number of the epoch is more than 100. Here we use the random initializations for the network parameters. The dropout technique is used in the improved U-Net, and the dropout rate is 0.2.

3.2. Evaluation Metrics

In experiments, we use the accuracy (ACC) to evaluate our proposed method $f_{\theta }$, which has been widely used in much literature. Let $D=\left\{I_n,y_n\right\}$ represent the test dataset and $f_{\theta }$ represent our proposed method. It can be calculated by the following formula

$$ACC\left(f_{\theta };D\right)=\frac{1}{N}{{\displaystyle \sum }}_{i=1}^N\prod (m_i=m_i^t),$$

(5)

where $\prod (m_i=m_i^{\mathrm{t}})$ is the indicator function defined as

$$\prod (m_i=m_i^t)=\{\begin{array}{cc}1,& if m_i=m_i^t\\ 0,& else\end{array},$$

(6)

here $m_i$ is the predicted result and $m_i^{\mathrm{t}}$ is the corresponding ground truth.

3.3. Performance Comparison

3.3.1. Quantitative Performance Comparison

In these experiments, we aimed to analyze the segmentation performance of the proposed method. In order to verify the effectiveness of the proposed semi-supervised method, we compared the semi-supervised method with the supervised method. The ACC comparison results between the supervised method and the semi-supervised method are shown in

Table 1. ACC comparison between the supervised method and the semi-supervised method. (S-S: semi-supervised. S: supervised).

n	Method	1	2	3	4	5	6	7	8	9	10	Mean ACC
4	S-S	0.9553	0.9468	0.9567	0.9527	0.9354	0.9491	0.9535	0.9525	0.9512	0.9612	0.95144
	S	0.9544	0.9468	0.9567	0.9528	0.9367	0.9509	0.9547	0.9548	0.9556	0.9600	0.95234
5	S-S	0.9585	0.9580	0.9476	0.9548	0.9608	0.9563	0.9645	0.9600	0.9559	0.9656	0.95820
	S	0.9574	0.9577	0.9502	0.9560	0.9600	0.9568	0.9636	0.9601	0.9570	0.9623	0.95811
6	S-S	0.9592	0.9564	0.9579	0.9622	0.9626	0.9595	0.9573	0.9652	0.9563	0.9653	0.96019
	S	0.9604	0.9570	0.9578	0.9612	0.9617	0.9588	0.9575	0.9639	0.9567	0.9619	0.95969
7	S-S	0.9653	0.9643	0.9625	0.9643	0.9652	0.9584	0.9665	0.9638	0.9538	0.9655	0.96296
	S	0.9639	0.9643	0.9618	0.9623	0.9623	0.9593	0.9659	0.9633	0.9568	0.9644	0.96243
8	S-S	0.9658	0.9652	0.9651	0.9639	0.9627	0.9644	0.9646	0.9627	0.9651	0.9661	0.96456
	S	0.9639	0.9644	0.9638	0.9634	0.9608	0.9647	0.964	0.9624	0.9648	0.9653	0.96375
9	S-S	0.9658	0.9668	0.9642	0.9655	0.9647	0.9654	0.9655	0.9652	0.9646	0.9652	0.96529
	S	0.9653	0.9659	0.9619	0.9637	0.9636	0.9651	0.9640	0.9649	0.9625	0.9640	0.96409
10	S-S	0.9645	0.9653	0.9653	0.9658	0.9675	0.9651	0.9676	0.9653	0.9659	0.9658	0.96581
	S	0.9653	0.9631	0.9649	0.9642	0.9671	0.9649	0.9664	0.9635	0.9651	0.9659	0.96504
11	S-S	0.9678	0.9678	0.9653	0.9671	0.9631	0.9679	0.9671	0.9661	0.9669	0.9668	0.96660
	S	0.9665	0.9674	0.9663	0.9660	0.9636	0.9667	0.9672	0.9647	0.9661	0.9662	0.96607
12	S-S	0.9673	0.9662	0.9679	0.9667	0.9675	0.9673	0.9679	0.9669	0.9673	0.9677	0.96727
	S	0.9658	0.9654	0.9673	0.9668	0.9669	0.9667	0.9677	0.9665	0.9679	0.9673	0.96683
13	S-S	0.9672	0.9681	0.9669	0.9671	0.9678	0.9680	0.9672	0.9665	0.9679	0.9677	0.96744
	S	0.9656	0.9671	0.9662	0.9675	0.9680	0.9675	0.9668	0.9676	0.9673	0.9671	0.96707
14	S-S	0.9672	0.9685	0.9675	0.9673	0.9685	0.9678	0.9674	0.9672	0.9687	0.9681	0.96782
	S	0.9681	0.9680	0.9670	0.9671	0.9683	0.9675	0.9666	0.9673	0.9681	0.9671	0.96751
15	S-S	0.9685	0.9682	0.9676	0.9669	0.9673	0.9684	0.9678	0.9675	0.9682	0.9680	0.96784
	S	0.9684	0.9675	0.9675	0.9676	0.9672	0.9679	0.9672	0.9683	0.9681	0.9683	0.96779
16	S-S	0.9676	0.9681	0.9683	0.9681	0.9671	0.9677	0.9685	0.9676	0.9685	0.9683	0.96798
	S	0.9671	0.9681	0.9684	0.9670	0.9683	0.9680	0.9683	0.9670	0.9681	0.9679	0.96782
17	S-S	0.9683	0.9681	0.9682	0.9683	0.9684	0.9683	0.9684	0.9686	0.9686	0.9683	0.96835
	S	0.9682	0.9685	0.9685	0.9683	0.9678	0.9688	0.9676	0.9682	0.9690	0.9681	0.96830
18	S-S	0.9685	0.9684	0.9687	0.9685	0.9688	0.9685	0.9688	0.9685	0.9683	0.9687	0.96857
	S	0.9684	0.9679	0.9688	0.9682	0.9690	0.9690	0.9685	0.9686	0.9683	0.9692	0.96859
19	S-S	0.9687	0.9688	0.9680	0.9684	0.9684	0.9686	0.9686	0.9688	0.9685	0.9686	0.96854
	S	0.9686	0.9688	0.9689	0.9684	0.9688	0.9682	0.9693	0.9682	0.9690	0.9688	0.96870

. In this experiment, we randomly divided the public DRIVE training dataset (20 fundus images) into A-group and B-group. The A-group included the n labeled fundus images, and the B-group included the 20-n unlabeled fundus images. As shown in the first column in Table 1, the parameter n = 4: 19. In the experiment of the supervised method, we used the n labeled fundus images (A-group) to learn the deep neural network, and assess the performance of the trained model. In the experiment of the semi-supervised method, we used the n labeled (A-group) and 20-n unlabeled (B-group) fundus images to learn the deep neural network, and then assess the performance of the trained model. In addition, in order to ensure reasonable experimental results, we gave ten experimental results. As shown in Table 1, the first row indicated the experiment number. In the corresponding column, we gave the experimental results for the different numbers of labeled images. In the last column, we also gave the mean ACC results of ten experiments.

From the table, we can observe: (1) with the increase in parameter n, the segmentation accuracy of the supervised and semi-supervised methods increased. This is consistent with the fact that an increase in the numbers of labeled images is helpful to improve the accuracy of parameter estimation of a deep neural network model. (2) When n = 5:17, the accuracy of the semi-supervised method was better than that of the supervised method. These experiment results verified the effectiveness of our proposed semi-supervised method. A satisfactory segmentation result could be obtained by labeling a small number of fundus images.

For easy observation, the box plot of the experiment results obtained by the proposed semi-supervised method is shown in Figure 3. As shown in Figure 3, the characteristics of the experimental results distribution can be observed from the box plot. From top to bottom, the observed statistical characteristics included upper extreme, upper quartile, median, lower quartile and lower extreme. In Figure 3, the Y-axis was the accuracy (ACC) and the X-axis was the number of labeled images (n = 4:19). The circular sign represented outliers and the red line represented the median. The green curve represented the curve of ACC with the number of labeled images.

From the figure, we can observe: (1) with the increase of the number of labeled images, the segmentation accuracy increased obviously. (2) When there were more than ten labeled images, the increase of segmentation accuracy was not obvious. Therefore, we can conclude that satisfactory segmentation accuracy can be obtained by labeling ten fundus images. The proposed semi-supervised method can reduce the cost of labeling the retinal images in terms of both finance and human effort.

In order to further verify the effectiveness of this algorithm, we compared our proposed semi-supervised method with state-of-the-art methods. The ACC and AUC (area under curve) comparison results on the DRIVE dataset are shown in

Table 2. ACC and AUC comparison with state-of-the-art methods on the DRIVE dataset. The symbol * represents the image-processing based method, the symbol + represents the machine learning based method, and the symbol ◊ represents the deep learning based method.

Methodology	Type	Year	ACC	AUC
Human observer			0.9470
Chaudhuri et al. [7]	*	1989	0.8773	0.7878
Zana and Klein [19]	*	2001	0.9377	0.8984
Xiaoyi and Mojon [8]	*	2003	0.9212	0.9114
Abramoff et al. [34]	+	2004	0.9416	0.9294
Staal et al. [35]	+	2004	0.9442	0.952
Mendonca and Campilho [21]	*	2006	0.9452
Soares et al. [36]	+	2006	0.9466	0.9614
Al-Rawi et al. [9]	*	2007	0.9535	0.9435
Espona et al. [25]	*	2007	0.9316
Martinez-Perez et al. [30]	*	2007	0.9344
Perez et al. [31]	*	2007	0.9220
Ricci and Perfetti [38]	+	2007	0.9563	0.9558
Espona et al. [26]	*	2008	0.9352
Zhang et al. [29]	*	2009	0.9610
Cinsdikici and Aydin [10]	*	2009	0.9293	0.9407
Vlachos and Dermatas [32]	*	2009	0.929
Zhang et al. [11]	*	2010	0. 9382
Amin and Yan [12]	*	2010	0.92	0.94
Lam et al. [24]	*	2010	0.9472	0.9614
Xu and Luo [39]	+	2010	0.9328
Lupascu et al. [40]	+	2010	0.9597	0.9561
Fraz et al. [22]	*	2011	0.9430
Miri and Mahloojifar [23]	*	2011	0.9458
Marin et al. [42]	+	2011	0.9452	0.9588
Fraz et al. [41]	+	2012	0.9480	0.9747
Odstrcilik et al. [13]	*	2013	0.9340	0.9519
Cheng et al. [43]	+	2014	0.9474	0.9648
Zhao et al. [28]	*	2014	0.9509
Azzopardi et al. [14]	*	2015	0.9442	0.9614
Zhao et al. [27]	*	2015	0.954	0.862
Roychowdhury et al. [37]	+	2015	0.952	0.962
Zhang et al. [58]	◊	2015	0.940
Maji et al. [59]	◊	2016	0.947
Liskowski et al. [60]	◊	2016	0.951	0.971
Fu et al. [61]	◊	2016	0.952
Yao et al. [62]	◊	2016	0.936
Fu et al. [63]	◊	2016	0.947
Li et al. [6]	◊	2016	0.9527	0.9738
Zhu et al. [45]	+	2017	0.9607
Tan et al. [64]	◊	2017	0.926
Ngo et al. [50]	◊	2017	0.9533	0.9752
Zhao et al. [18]	*	2018	0.9592
Soomro et al. [70]	◊	2018	0.948	0.844
Guo et al. [65]	◊	2018	0.954	0.979
Hu et al. [49]	◊	2018	0.9533	0.9759
Yan et al. [66]	◊	2019	0.954	0.975
Our method (11)	◊	2020	0.9631	0.976

. In the table, we listed 46 state-of-the-art methods, including their type, year, and ACC evaluation metrics. In the second column, the symbol * represented the image-processing based method, the symbol + represented the machine learning based method, and the symbol ◊ represented the deep learning based method. In the second row, we listed the accuracy result obtained by a human observer. In the last row, we listed the accuracy result obtained by our proposed semi-supervised method by using eleven labeled fundus images. The number in brackets was the number of labeled images.

From the table, we can observe: (1) The ACC and AUC results obtained by our proposed semi-supervised method outperformed the results obtained by the human observer and many state-of-the-art methods. (2) The proposed semi-supervised method only used eleven labeled fundus images, which meant that our method could save half of the cost of labeling the fundus images in terms of both finance and human effort. Therefore, we can conclude that the proposed semi-supervised method can obtain a state-of-the-art performance and avoid the problems of insufficient hand-labels.

3.3.2. Qualitative Performance Comparison

To further evaluate the proposed semi-supervised method, Figure 4 shows the qualitative performance comparison between the semi-supervised and the supervised method. The original fundus image is shown in Figure 4a and its ground truth is shown in Figure 4b. The segmentation results of the proposed semi-supervised methods by using 8, 12 and 15 labeled images are shown in Figure 4c–e, and the segmentation results of the supervised methods by using 8, 12 and 15 labeled images are shown in Figure 4g–i. For convenient observation, the local enlarged segmentation maps obtained by the proposed semi-supervised and supervised methods are shown in Figure 4f,j. From Figure 4f,j, we can see that the proposed semi-supervised method can obtain the complete retinal blood vessel tree, and the supervised method cannot obtain the complete retinal blood vessel tree. Therefore, we can conclude that the proposed semi-supervised method can achieve satisfactory segmentation performance.

3.4. Convergence Analysis

This experiment aims to verify the convergence of the proposed semi-supervised method. In this experiment, we carried out ten experiments. Based on the results of these ten experiments, we drew the box plot of the loss curve with time for the proposed semi-supervised method, as shown in Figure 5. The box plot was used to reflect the statistical characteristics of the loss curve. In the figure, the Y-axis was the loss and the X-axis was the number of epochs. Similar to Figure 3, the circular sign represented outliers and the red line represented the median. The green curve represented the loss curve with time. From the figure, we can observe: (1) With the increase of the epoch, the loss of the proposed semi-supervised method decayed rapidly. (2) When the number of epochs was more than ten, the change of loss curve tended to be stable. This verified the convergence of the proposed semi-supervised segmentation method.

4. Conclusions

In this paper, we propose a new deep learning based semi-supervised learning method for blood vessel tree segmentation in the retinal image. The proposed method is able to combine the deep neural network with the semi-supervised learning strategy, which deals with the costly problem of labeling the retinal images, in terms of both finance and human effort. On the DRIVE datasets, the proposed semi-supervised method only uses eleven labeled fundus images and outperforms the results obtained by the human observer and many state-of-the-art methods, which means that our method can save half of the cost of labeling the fundus images in terms of both finance and human effort. In addition, a large number of experimental results show that the proposed semi-supervised method can effectively accomplish the blood vessel segmentation task and has good convergence. In the future, we plan to investigate the use of more improved training methods for semi-supervised learning. In addition, we are also interested in the use of the semantic priors of the blood vessel for further improvement.