1. Introduction
The retina, a crucial light-sensitive layer, plays a pivotal role in our ability to perceive both central and peripheral images. It is nourished and cleansed by an intricate network of retinal blood vessels that are essential for maintaining the health of our visual system. Alterations or damage to these vessels’ structure can herald a range of serious health issues, affecting not just the eyes, but also the cardiovascular system. Diseases such as diabetic retinopathy (DR), macular edema (ME), glaucoma, cataracts, tumors, and hypertension have been linked to such morphological changes [
1,
2,
3]. By segmenting retinal images, medical professionals can identify and study these vascular changes, facilitating the early detection of lesions. This proactive approach is crucial for preventing blindness and mitigating a spectrum of vision-related conditions that stem from retinal abnormalities like DR. However, the endeavor to detect these changes is fraught with challenges. The images often suffer from noise, variations in contrast and lighting, and inconsistencies in the appearance, size, shape, and positioning of the blood vessels, complicating the detection process [
4]. Initially, the examination of retinal fundus images for screening purposes is heavily dependent on the manual selection carried out by specialists. This approach, however, has proved to be exceedingly time consuming and laborious, requiring a high level of expertise and leaving much room for variability based on the individual’s experience. Such limitations have rendered it challenging to implement at a wide scale. Consequently, the development of automated methods for retinal vessel segmentation (RVS) have emerged as a compelling alternative. These automated techniques offer a significant advancement in enhancing the accuracy and efficiency of diagnoses, facilitating early identification and enabling prompt intervention for diseases related to the eye.
The global scientific community has developed a considerable number of RVS algorithms to date [
5]. Hence, the above types of algorithms can generally be categorized into two main groups: unsupervised [
6] and supervised [
7]. These algorithms, which attempt to give an accurate segmentation of the retinal blood vessels, work without using annotations or additional guiding information, and they are solely based on the inherent characteristics of the vessels. The methods have mainly based on the inverse transformations of thresholding [
8] and filtering [
9] together with morphological operations. Accordingly, unsupervised algorithms can be further classified into three types: algorithms based on matched filtering [
10], those dependent on the morphology of blood vessels, and those that completely rely on vessel tracking. However, in order to enhance the detection of the small vessels, an improved Frangi filter combined with a patch noise reduction method was proposed by Contejean et al. [
11] to reduce the effect that occurs due to the bright and diseased areas of the retinal images on the segmentation of the blood vessels. The Frangi filter is prone to smoothing the whole image, which results in the ignoring of low contrast vessels. Graham et al. [
12] tackled this issue from the point of view of tracking vessels by taking into account the small branches and junctions of vessels. They proposed the Bayesian theory and multi-scale inspection-based algorithm. This method, however, overly focuses on branch points and overlooks the positional information of the main vessels. Oliveira et al. [
13] proposed fully convolutional networks (FCNs), using convolutional layers (CL) instead of fully connected layers, and they also used upsampling to restore the image size but the segmentation results were rough and less accurate. Zunair et al. [
14] proposed Sharp U-Net, which improved upon FCNs with a symmetric encoder–decoder U-shaped network and used the skip connections between encoders and decoders at the same level to restore lost spatial information, thereby achieving precise segmentation outputs. Zhou et al. [
15] used dense skip connections and introduced convolution in the skip pathways to reduce the feature discrepancy between the encoder and decoder. In instances where fundus images are subjected to uneven illumination, there is a noticeable decline in the precision of segmentation outcomes. Xinfeng et al. [
16] used densely connected U-Net, allowing the network to utilize richer spatial context information. Oktay et al. [
17] proposed attention U-Net, which introduced a gated attention mechanism to enhance the learning of segmentation target features and ignore unimportant features. However, due to the complex imaging of fundus images, noise interference, and the uneven distribution of small blood vessels, these segmentation methods struggle to accurately segment vascular boundaries, and they lack generalizability and their networks are prone to degradation.
On the other hand, the supervised algorithms were learnt from a huge array of manual annotation samples and, thus, enabled through the automatic extraction of retinal blood vessel information and segmentation. Jainish et al. [
18] introduced a new adaptive histogram equalization and robust distance transformation-based preprocessing technique for retinal images to increase in their segmentation accuracy; however, this preprocessing technique showed a lesser processing ability in low-contrast vessels because of the differential preprocessing conducted in the various regions of an image. U-Net [
19] is a fully symmetric algorithm in encoding and decoding, and it was designed to recover the lost vessel feature information during encoding through channel splicing and fusion; on the other hand, this method directly merges the results of the encoder with the decoder, where, of course, vascular information must suffer from loss. Ccs-unet, proposed by Zhu et al. [
20], incorporates an attention gating mechanism in all skip connections to alleviate the influence of background pixels on segmentation results. However, it only changes the encoding–decoding fusion manner but does not touch the multi-scale nature of vessels, thereby making the model have a lower accuracy in the segmentation of small vessels. Prethija et al. [
21] proposed EAMR-Net, in which the spatial attention and channel attention attend to one another in parallel right at the base of the U-Net for an accurate segmentation of the retinal blood vessels. However, this method does not consider the information loss in the encoding and decoding, and sometimes it may lead to an erroneously segmented small vessels. These strategies capitalize on the unique characteristics of fundus images and vascular patterns to identify segmentation targets, offering a speedier segmentation process and reducing the time required compared to the traditional manual screening approach. Nonetheless, these methods necessitate extensive domain-specific knowledge, intricate mathematical procedures, and the development of manual feature extraction techniques to establish the correlations between features and prognostic outcomes. This reliance significantly undermines their practicality and precision for clinical diagnostic purposes [
22].
To address the challenges delineated, this study introduces the multi-scale position-aware cyclic convolutional network (MPCCN). This innovative framework marks the first integration of a multi-scale input module (MIM) within the encoder pathway, significantly expanding the receptive field while providing precise location metrics. Such enhancements bolster the network’s proficiency in vascular feature detection. Further, MPCCN employs a weighted-shared residual module (WRM), enriching the foundational feature dataset and mitigating the risk of detail loss and network degradation that are commonly associated with consecutive downsampling processes. Moreover, the incorporation of a guided attention module (GAM) within the skip connections adeptly curtails noise and extraneous feature representations, thereby refining the segmentation accuracy of retinal vessels. The principal contributions of this study are delineated as follows.
We propose a novel multi-scale position-aware cyclic convolutional network (MPCCN) that incorporates a MIM within the encoder pathway. To the best of our knowledge, this is the first framework to integrate such a module, significantly expanding the receptive field while providing precise location metrics, which enhances the network’s ability to detect vascular features effectively.
We introduce a weighted-shared residual module (WRM), which enriches the foundational feature dataset. This module mitigates the risk of detail loss and network degradation, a common challenge in consecutive downsampling processes. The WRM ensures the preservation of critical information, thereby maintaining high segmentation accuracy.
We incorporate a guided attention module (GAM) within the skip connections; one that is specifically designed to curtail noise and extraneous feature representations. By refining the segmentation process, the GAM enhances the accuracy of retinal vessel detection, setting a new benchmark for performance in this domain.
4. Experimental Results and Analysis
4.1. Experimental Setup
The experiments in this study were conducted on a high-performance Linux machine running Ubuntu 22.04. The deep neural network models were developed and evaluated on a powerful computer equipped with a 13th generation Intel Core i7 processor, an NVIDIA RTX 3090 GPU with 24 GB of GDDR6X memory, and 32 GB of DDR5 RAM. The most recent stable version of the PyTorch framework, with NVIDIA CUDA support, was utilized for implementing and running the experiments. To ensure consistency, both the training and testing of each model were carried out in the same computing environment using identical parameters throughout the process.
4.2. Datasets
A dataset is a crucial component of medical image processing research and serves as a fundamental basis for evaluating the performance of various algorithms. The integrity and diversity of the dataset directly impact the reliability and generalizability of the research findings. This section provides a detailed description of the commonly used datasets in the study of retinal vessel image segmentation. By analyzing both normal retinal vessels and comparing them with lesion examples from different datasets, we can gain deeper insights into the strengths and limitations of each dataset.
Figure 7 illustrates examples of normal retinal vessels alongside lesion examples from various datasets, highlighting the variations and challenges in RVS. The comprehensive understanding of these datasets is essential for advancing algorithm development and enhancing the accuracy of medical image analysis.
4.2.1. DRIVE
The DRIVE dataset [
56] originates from the Netherlands’ diabetic retinopathy screening program, and it is designed to facilitate research in retinal image analysis. This dataset comprises 40 carefully selected images from a pool of 400 diabetic patients. Each image, with a resolution of 565 × 584 pixels and stored in JPEG format, is captured using a Canon CR3 non-mydriatic 45CCD camera. The images are taken with a 5° field of view (FOV) at a 45° angle, with the captured resolution being 768 × 584 pixels. Notably, the images exhibit a circular FOV with a diameter of 540 pixels. For inclusion in the dataset, the images are cropped and provided with FOV mask images to ensure consistency and ease of use. Among these images, seven display early signs of lesions, while the remaining images do not exhibit significant pathological features. Despite its establishment many years ago, the DRIVE dataset remains one of the most frequently utilized datasets for RVS research. Its continued relevance is a testament to its quality and comprehensiveness, although no new images have been added since its initial release.
4.2.2. STARE
The STARE dataset [
57] was initially composed of 20 high-resolution fundus images, each measuring 700 × 605 pixels and stored in PPM format. Each image underwent meticulous manual segmentation by at least two experienced experts, ensuring a high level of accuracy and reliability. Among these images, 10 exhibit signs of retinal lesions, while the remaining images are free from any pathological indications. One notable limitation of the STARE dataset is the homogeneous nature of the lesion types, which may impact the robustness and generalizability of the models trained exclusively on this dataset. Consequently, it is advisable to use the STARE dataset in conjunction with other datasets to enhance the model performance in practical applications. The dataset has since been expanded to include 40 images for vessel segmentation and 80 images for optic nerve detection, broadening its utility and applicability in various research contexts.
4.2.3. CHASE_DB1
The CHASE_DB1 dataset [
58] is part of the Child Heart and Health Study in England (CHASE), which collected images from multi-ethnic schools. This dataset comprises retinal images of both eyes of students, and they were captured using a handheld Nidek NM-200-D fundus camera with a 30° FOV. It consists of 28 fundus images, each sized 1280 × 960 pixels and stored in TIFF format. The recruitment of volunteers for image collection was designed to include a diverse range of ethnicities, resulting in a dataset characterized by varying background transparency and uneven vessel thickness. This diversity effectively addresses the issue of the homogeneous lesion types found in other datasets.
4.2.4. HRF
The HRF dataset [
59] consists of 45 images, each with a resolution of 3504 × 2336 pixels and stored in JPEG format. The images in this dataset are categorized into three types: healthy patients, diabetic retinopathy patients, and glaucoma patients. Each image has been manually annotated, making it one of the highest-resolution datasets available at the time. Numerous studies have utilized the HRF dataset for validation. A comparison of these common public datasets is shown in
Table 2.
4.3. Data Preprocessing and Transfer Learning
The retinal images used for segmentation typically suffer from low contrast, subtle vascular structures, and significant noise interference, which can affect the accuracy of the segmentation model. To address these challenges, a rigorous data preprocessing pipeline described by Ziaur et al. [
60] was applied to enhance the quality of the images before inputting them into the MPCCN model. Given the relatively small number of training images in the available datasets (e.g., DRIVE, STARE, and CHASE_DB1), data augmentation was applied to artificially expand the dataset size and improve the network’s generalizability. This included random rotations, flips, and scaling of the images. Furthermore, a sliding window approach was employed, where patches of a
size were extracted from each image, and, for higher resolution datasets like HRF, a patch size of
was used. This ensures that both global and local vessel structures are captured during training.
Table 2 summarizes the data distribution and patch extraction details.
Furthermore, to accelerate the convergence and improve the overall performance of the MPCCN, transfer learning was employed. The encoder part of the MPCCN model was initialized with weights that were pre-trained on the ImageNet dataset [
61]. These pre-trained weights provided the model with a strong starting point by leveraging knowledge from millions of natural images, enabling the network to extract robust low-level features such as edges, textures, and shapes. After initializing the encoder with pre-trained weights, the model was fine tuned on RVS datasets. During fine tuning, the weights in the encoder were allowed to update gradually, while the decoder part of the network was trained from scratch. Fine tuning allowed the model to adapt the pre-trained features to the specific task of RVS, resulting in faster convergence and improved segmentation performance.
4.4. Evaluation Metrics
In image segmentation tasks, seven metrics commonly serve as standards for evaluating the quality of model segmentation results:
,
SP,
SE,
F1-score, the
AUC of the ROC curve, and the intersection over union (IoU). The AUC is a useful metric for evaluating the overall performance of a binary classifier, representing the likelihood that the classifier will rank a randomly chosen positive instance higher than a randomly chosen negative one. We also used these metrics as evaluation criteria in this study, and their formulas are presented in the following Equations:
Among them, the true positives (TP) represent the number of correctly classified vessel pixels, indicating instances where the algorithm successfully identifies the vessel pixels as such. The true negatives (TN) represent the number of correctly classified background pixels, showing the algorithm’s ability to recognize non-vessel areas correctly. On the other hand, the false positives (FP) are the number of background pixels incorrectly classified as vessel pixels, which highlight potential over-detection issues in the algorithm’s performance. Conversely, the false negatives (FN) are the number of vessel pixels incorrectly classified as background pixels, demonstrating cases where the algorithm fails to detect actual vessel pixels. Furthermore, the IoU, also known as the Jaccard Index, measures the overlap between the predicted segmentation and the ground truth as the ratio of the intersection to the union of the predicted and actual regions. It is particularly useful for evaluating segmentation quality by considering both false positives and false negatives.
4.5. Training Strategy
The training of the MPCCN model was conducted using a series of carefully selected techniques and hyperparameters to optimize performance on the RVS task. Given the complexity of the retinal vasculature, which features vessels of varying sizes and intricate patterns, it is crucial to employ strategies that would enhance the model’s ability to generalize across different datasets while preventing overfitting. Therefore, the hyperparameters were meticulously tuned to strike a balance between training efficiency and model performance. The learning rate, a critical parameter influencing the convergence of the model, was initialized at and gradually reduced using a cosine annealing schedule. A batch size of 16 was selected based on the available GPU memory and the need to maintain stable gradient updates. Moreover, the optimizer used was Adam, with the default parameters for momentum being and the beta values were . Weight decay was set to to regularize the model and prevent overfitting by penalizing large weights. Additionally, warm-up epochs were implemented, where the learning rate was gradually increased over the first five epochs to stabilize training and avoid divergence. To further prevent overfitting, dropout layers were incorporated within the network, with a dropout rate of 0.3 applied to key layers. This technique randomly deactivates neurons during training, forcing the network to develop redundant representations, which enhances its generalization capability. Additionally, weight regularization was applied to limit the magnitude of the network’s weights, reducing the risk of overfitting.
The dataset was divided into training, validation, and test sets, with the model’s performance evaluated on the validation set after each epoch. The training process was monitored over 50 epochs, with early stopping implemented if no significant improvement in validation loss was observed over 10 consecutive epochs. This strategy prevented unnecessary overtraining and conserved computational resources. Throughout training, the F1-score and IoU metrics were used to assess segmentation performance, with the model’s hyperparameters being fine tuned to maximize these metrics on the validation set.
4.6. Qualitative Analysis
To verify the superiority of MPCCN, it was compared with different RVS algorithms, i.e., RVSA [
46], ARDC-UNet [
62], FAFFN [
63], and TCU-NET [
64]. The same experimental conditions and training strategies were used to perform RVS experiments on the DRIVE and CHASE_DB1 datasets. The experimental results are shown in
Figure 8. From
Figure 8 and
Figure 9, it can be observed that RVSA [
46], as a classic algorithm for RVS, can only segment the general direction and distribution of the vessels. However, it tends to show major vessel breaks and mis-segmentation at vessel intersections.
Building upon the foundation of RVSA [
46], ARDC-UNet [
62] replaces convolution blocks with residual convolutions. This allows for the fine extraction of the main vessel network without overfitting, thus outperforming the traditional U-Net. Nevertheless, TCU-NET [
64] has a low accuracy rate for extracting tiny vessels and tends to produce false positive vessel areas. On the other hand, ARDC-UNet [
62] introduces an attention gates mechanism at the skip connections, which helps to suppress noisy background regions while increasing the focus on vessel areas. This mechanism reduces the impact of background noise while extracting the main network, but mis-segmentation still occurs with tiny vessels.
Furthermore, FAFFN [
63] introduced parallel channel and spatial attention modules at the bottom of U-Net, unifying local and global vessel information to achieve high-precision vessel segmentation. It can accurately segment both the main and tiny vessels, with high accuracy in tiny vessel segmentation. However, mis-segmentation still occurs at vessel edges. In contrast, the segmentation results of MPCCN are consistent with the manually segmented vessel images by experts. Compared to other algorithms, MPCCN maintains connectivity in tiny vessel segmentation, reduces the impact of background noise, and ensures vessel connectivity. Therefore, MPCCN finely preserves vessel structural information, has a certain degree of anti-interference capability, and achieves high-precision RVS.
To further verify the performance advantage of MPCCN, a local region comparison of segmented images was conducted with RVSA, ARDC-UNet, and FAFFN algorithms. The comparison results are shown in
Figure 10. RVSA can segment main vessels but has low segmentation accuracy at vessel intersections and in tiny vessel areas. ARDC-UNet improves the segmentation effect of tiny vessels compared to RVSA, but disconnection often occurs. The FAFFN model can gradually segment the general shape of boundary vessels, but disconnection still occurs. In contrast, MPCCN accurately segments both the main and tiny vessels without disconnection, greatly ensuring the integrity of the segmented retinal vessels. This comparison highlights the strengths of MPCCN as it effectively combines the advantages of the aforementioned methods while minimizing their drawbacks. Through a comparison of the segmentation details of different algorithms, it was demonstrated that replacing traditional convolutions with PCC can retain image contents as much as possible and can effectively highlight key vessel information. Additionally, standard convolution, PCC, and efficient channel attention can expand the receptive field to improve segmentation accuracy, promoting the integration of main and tiny vessels. MIM allows the algorithm to fully utilize encoding information at different levels, helping to eliminate background noise and redundant information in vessel feature maps. These results verify that MPCCN has high accuracy and anti-interference capability in RVS.
4.7. Quantitative Analysis
Our thorough evaluation of the algorithm’s performance on the DRIVE, STARE, and CHASE_DB1 datasets involved a quantitative and qualitative comparison. Through this comparative analysis, we demonstrated our algorithm’s proficiency in segmenting fine-vessel bifurcations and areas within the optic disc, macula, and lesions. Our findings, illustrated in various tables and figures, highlight the algorithm’s superior ability in segmenting vessels in challenging conditions, such as lesions or uneven lighting, with greater accuracy and continuity compared to the reference algorithms. By examining the enlarged details of vessel segmentation, our analysis confirms the algorithm’s enhanced performance, particularly in detecting the end branches of vessels and minimizing the impact of lesion areas and artifacts on segmentation accuracy. To verify the effectiveness and feasibility, comparisons were made between the MPCCN and other high-performing deep learning-based segmentation methods. The comparison results are shown in
Table 3,
Table 4 and
Table 5, with the best metrics highlighted in bold.
Furthermore,
Figure 11 illustrates a comparison of the precision–recall curves and ROC curves for different algorithms on the DRIVE and CHASE_DB1 datasets. These curves provide valuable insights into the performance of the algorithms in question.
4.8. Ablation Study
To further validate the effectiveness of each module in the proposed algorithm, ablation experiments were conducted on the DRIVE and CHASE_DB1 datasets. The experimental results are presented in
Table 6, and they reveal that M1 has already achieved commendable accuracy in RVS, though there remains potential for enhancement.
M2 demonstrated improvements across various metrics compared to M1, indicating that the WRM contributes to increased segmentation precision. Subsequently, M3 exhibited further enhancement over M2, suggesting that the GAM effectively extracts multi-scale vessel information. Additionally, M4 showed a notable improvement in all metrics compared to M3, highlighting that the GAM module mitigates the impact of background noise by integrating encoded information from different levels, thereby achieving a high-precision segmentation of small vessels. The aforementioned results substantiate the effectiveness and rationality of the MPCCN framework. The progressive performance improvements from M1 to M4 indicate the efficacy of the proposed modules in enhancing the U-Net algorithm. Notably, M4 achieved the highest AUC value, signifying that MPCCN excels in RVS. It accurately delineated both primary and small vessels while maintaining the connectivity of segmented vessels.
Furthermore,
Figure 12 presents a qualitative comparison of the segmentation results produced by different modules within MPCCN. Each segmented image included two enlarged local detail views, allowing for a closer examination of specific areas. The segmentation performance of the individual modules in MPCCN varied, particularly when one or two modules were combined to evaluate their effect on the network’s overall performance. Notably, the combined use of all modules yielded significantly better results compared to any individual module, especially in the segmentation of microvessels. This demonstrates that the integration of multiple components enhances the network’s ability to capture fine details. In conclusion, MPCCN was proven to be a robust and highly accurate algorithm for RVS.
4.9. Computational Complexity
In medical clinical applications, there are often insufficient hardware computing resources available to deploy and run high-complexity networks. Therefore, the computational complexity of deep learning networks is an important factor in evaluating their potential for real-world applications. Two key metrics commonly used to assess this complexity are the number of network parameters
Q and floating-point operations (FLOPs) [
72]. These metrics provide insight into both the memory requirements and the computational demands of a network. The number of parameters
Q is defined as follows:
And the number of FLOPs is given by
where
and
represent the number of input and output feature map channels, respectively;
K represents the size of the convolution kernel; and
H and
W represent the height and width of the feature map, respectively.
Table 7 includes a comparison, using both FLOPs and parameter count
Q, of the computational complexity of the proposed MPCCN network with other RVS networks on the STARE dataset. As shown in the table, MPCCN achieved a relatively low FLOPs value of 13.1 M and a
Q of 7.1 M, making it computationally efficient while maintaining high segmentation performance. Specifically, MPCCN achieved an accuracy of 0.9775, a specificity of 0.9725, a sensitivity of 0.9888, an F1-score of 0.9055, and an AUC of 0.9905, all while requiring only 0.31 s per image. Compared to other networks, MPCCN struck an excellent balance between low computational complexity and strong performance. For instance, Genetic U-Net produced a higher FLOPs value of 17.5 M and a parameter count of 9.8 M, requiring more computation time, i.e., 1.35 s, while delivering comparable segmentation results. Similarly, ARDC-UNet had a higher FLOPs and parameter count, but it required a much longer processing time of 2.25 s. In contrast, MPCCN outperformed these networks in terms of computational efficiency and segmentation accuracy, making it highly suitable for resource-constrained clinical applications. This demonstrates that the proposed network can achieve high accuracy without requiring a large number of parameters or excessive computational power, highlighting its practical applicability in real-world medical scenarios.
5. Limitations and Future Work
While the MPCCN model demonstrated significant improvements in RVS, it still faced some limitations that need to be addressed. Firstly, the model’s performance is highly dependent on the diversity and size of the dataset even though data augmentation techniques were applied to overcome the limited availability of annotated retinal images. However, the lack of large-scale and diverse datasets covering different ethnicities, age groups, and retinal conditions may still limit the model’s generalizability. The performance could potentially drop when applied to unseen data from different populations or with more complex pathologies. Secondly, retinal images can vary significantly in their visual appearance due to non-uniform illumination conditions, which can impact the clarity of vascular structures. Images taken under inconsistent lighting or low-quality fundus camera images may require pre-adjustments to enhance visibility before being fed into the network. Addressing non-uniform illumination is critical as uncorrected images can reduce the model’s segmentation accuracy, making this an additional limitation that should be considered.
However, the types of data used in this study were somewhat limited. In addition to the commonly used color fundus images, there are also scanning laser ophthalmoscope (SLO) retinal images and optical coherence tomography (OCT) retinal images. Compared to standard color fundus images, SLO and OCT images contain more information and can more effectively reflect the fine structure, especially OCT images that clearly display the 10 layers of the retina. In short, the combination of MPCCN with multi-modal medical imaging techniques to further improve the accuracy of retinal disease diagnosis should be explored. Fusing information from multiple imaging modalities may provide a more comprehensive understanding of retinal conditions and lead to better diagnostic outcomes.