ECF-Net: Enhanced, Channel-Based, Multi-Scale Feature Fusion Network for COVID-19 Image Segmentation

Abstract

Accurate segmentation of COVID-19 lesion regions in lung CT images aids physicians in analyzing and diagnosing patients’ conditions. However, the varying morphology and blurred contours of these regions make this task complex and challenging. Existing methods utilizing Transformer architecture lack attention to local features, leading to the loss of detailed information in tiny lesion regions. To address these issues, we propose a multi-scale feature fusion network, ECF-Net, based on channel enhancement. Specifically, we leverage the learning capabilities of both CNN and Transformer architectures to design parallel channel extraction blocks in three different ways, effectively capturing diverse lesion features. Additionally, to minimize irrelevant information in the high-dimensional feature space and focus the network on useful and critical information, we develop adaptive feature generation blocks. Lastly, a bidirectional pyramid-structured feature fusion approach is introduced to integrate features at different levels, enhancing the diversity of feature representations and improving segmentation accuracy for lesions of various scales. The proposed method is tested on four COVID-19 datasets, demonstrating mIoU values of 84.36%, 87.15%, 83.73%, and 75.58%, respectively, outperforming several current state-of-the-art methods and exhibiting excellent segmentation performance. These findings provide robust technical support for medical image segmentation in clinical practice.

1. Introduction

Since 2020, the coronavirus disease 2019 (COVID-19) has evolved into a global pandemic. As of 10 April 2020, this novel infectious disease, caused by Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2), has resulted in over 92,000 deaths. While the reverse transcription polymerase chain reaction (RT-PCR) serves as the primary diagnostic method for COVID-19, it is time consuming, prone to misdetection, and can yield false-positive results. In contrast, computed tomography (CT) plays a critical role in COVID-19 diagnosis. However, manually diagnosing a large number of CT scans is highly time consuming and labor intensive.

With the rapid advancement of artificial intelligence technology, medical image segmentation using deep learning has emerged as a prominent research area. The Unet [1] network, based on convolutional neural network (CNN) architecture, stands out as a pioneering model for medical image segmentation. Subsequently, various derivative models, such as DeepLab [2], SegNet [3], UNet++ [4], and RefineNet [5], have been introduced by researchers, all leveraging a CNN as a feature extraction network with encoder–decoder architecture to achieve significant segmentation performance. However, the inherent inductive bias of CNNs, which prioritizes local and translational invariance, limit its ability to effectively capture global information. The Transformer [6], originally proposed for natural language processing, has been adapted for computer vision tasks, notably with the Vision Transformer (ViT) [7] being a prominent example. By incorporating self-attention mechanisms, ViT excels in capturing the global context within images, benefiting segmentation tasks as demonstrated by models like Segmenter [8], SegFormer [9], and Swin-UNet [10]. Despite its capability in terms of modeling long-range dependencies, ViT tends to overlook local detailed features. To address this, recent research has integrated CNNs into Transformer architectures. This approach aims to fuse local and global features, thereby enhancing network representations and improving segmentation accuracy. For instance, TransUNet [11], combines a CNN for extracting low-level features with the Transformer architecture for global interactions, showcasing strong performance in tasks like CT-based multi-organ segmentation. Additionally, MobileViT [12] represents an efficient fusion of a CNN and a ViT, enabling feature extraction with reduced parameters.

While the aforementioned methods have shown promising results in image segmentation, medical CT images for COVID-19 present greater challenges, with complex textures, varying lesion morphologies, size discrepancies, and indistinct boundaries compared to natural scene images. Consequently, a single CNN or Transformer architecture for feature learning fails to adequately capture both local and global contextual information. Moreover, existing approaches that combine a CNN and the Transformer architecture typically employ sequential fusion, namely the CNN for low-level features and the Transformer architecture for high-level features, resulting in shallow networks lacking in global context and deep networks missing local detail. Furthermore, the resolution of feature maps decreases through successive convolutional pooling operations, leading to the loss of local details.

To address these challenges, we propose ECF-Net, a multi-scale feature fusion segmentation network based on channel enhancement. ECF-Net comprises four main components: (1) a channel extraction block utilizing the Swin Transformer, ResNeXt [13], and ResNeXt, with an efficient channel attention mechanism ECA [14] to extract diverse image channels in parallel; (2) a channel fusion block that combines extracted features from different channels to generate a high-dimensional feature space; (3) an adaptive channel generation block designed to filter out irrelevant and redundant information, retaining only essential details; and (4) a bidirectional feature pyramid fusion block that integrates multi-scale features from different stages, enhancing network feature expression to effectively address diverse lesion area characteristics.

This paper introduces several key contributions as follows:

(1) We propose a novel segmentation network, ECF-Net, centered on channel enhancement. This model introduces two primary innovations: first, a parallel strategy employing a CNN, Transformer architecture, and a CNN with channel attention for channel enhancement. Second, the integration of an adaptive channel generation block and a bidirectional feature pyramid fusion block to mitigate noise in the high-dimensional feature space and enhance feature expression by bidirectionally transferring features across different scales, thereby improving segmentation accuracy;

(2) Extensive experiments demonstrate that our proposed method significantly outperforms several currently popular methods used in COVID-19 CT image segmentation tasks.

The paper is structured as follows: Section 2 reviews related work, Section 3 details the network workflow and operational mechanisms of each module, Section 4 presents the comparative and ablation experiments, along with analysis of the experimental results, and Section 5 outlines the research methodology and proposes future directions.

2. Related Work

2.1. CNN-Based Methods

The convolutional neural network (CNN), a classical deep learning architecture, has eliminated the need for manual feature labeling in image segmentation, enabling automatic segmentation. The UNet network, an improvement on the fully convolutional network (FCN) [15], adopts an encoder–decoder structure with skip connections to preserve contextual information. This architecture has been widely adopted in various medical imaging applications, yielding good results [16,17,18].

2.2. Transformer-Based Methods

The self-attention mechanism introduced by Transformers, with its ability to establish long-range dependencies and capture contextual information, has gained popularity in various visual tasks. For instance, TransAttUnet [19] combines self-perceived attention with self-attention to effectively learn non-local interactions between encoder features. UNETR [20] utilizes Transformers to efficiently capture global multi-scale information, while Swin-UNet employs a hierarchical Swin Transformer [21], with a shifted window as both encoder and decoder. The CSWin Transformer [22] introduces a cross-window self-attention mechanism to reduce the computational overhead and enhance performance.

2.3. Methods Based on the Combination of a CNN and a Transformer

To enhance the overall network’s feature representation, combining CNN and Transformer architectures has become a prominent research area in computer vision. For example, CTCNet [23] incorporates two encoder branches: one utilizing ResNet34 to extract spatial and contextual features, and the other employing a Swin Transformer to capture long-range dependencies. These features are then fused through a feature complementation module. The method has demonstrated substantial improvements in multi-organ and cardiac segmentation tasks. TC-Net [24] employs CNN-based encoder and decoder structures to extract local information from medical images, while utilizing a Transformer branch to capture the global context. Additionally, it introduces a Local-Aware and Long-Range Dependency Fusion Strategy (LLCS) and a Dynamic Cyclic Focal Loss (DCFL) to address the class imbalance issue in multi-lesion segmentation. BRAU-Net++ [25] integrates a CNN and a Vision Transformer (ViT), utilizing bi-level routing attention as the core building block to construct a U-shaped encoder–decoder architecture. This design features a hierarchical construction of both the encoder and decoder to capture global semantic information, while mitigating computational complexity. The model demonstrates strong segmentation performance across three medical imaging datasets. Pact-Net [26] proposes an effective fusion of global information extracted by Transformers with local features extracted by CNNs using channel and spatial attention mechanisms, along with a multi-scale approach. TGDAUNet [27] introduces a medical image segmentation network based on a two-branch attentional UNet, combining a Transformer and a GCNN. It employs a polarized self-attention (PSA) module to reduce redundant information caused by multiple scales, thereby improving the coupling with feature information extracted from the Transformer backbone. TSCA-Net [28] utilizes a Transformer-based spatial and channel attention module to extract spatial and channel-related global complementary information from different layers of a U-shaped network. In the decoder, a spatial and channel feature fusion block is designed to integrate the Transformer features’ spatial and channel information. However, relying solely on attention mechanisms for feature fusion may cause the network to focus on salient image regions, while neglecting non-salient regions that contain critical details, thus reducing segmentation accuracy. Early signs of certain lesions may appear in these non-significant regions and ignoring them could lead to misdiagnosis. Furthermore, an over-reliance on salient regions may lead to overfitting, causing the model to perform well on training data, but struggle with new, unseen images. TFCN [29] proposes a Transformer for Fully Convolutional DenseNets (FC-DenseNet), combining the ResLinear-Transformer (RL-Transformer) and Convolutional Linear Attention Block (CLAB). While dense connectivity can enhance performance, the network still heavily relies on convolutional layers, potentially limiting its ability to handle complex nonlinear problems. This can result in an inability to effectively capture subtle boundary, shape, and texture features in lesion regions, thereby affecting the segmentation results. To fully leverage the feature learning capabilities of both CNN and Transformer architectures, we propose a novel ECF-Net for COVID-19 CT image segmentation.

3. Materials and Methods

We propose a novel multi-scale feature fusion segmentation network called ECF-Net, which is based on channel enhancement, as illustrated in Figure 1.

ECF-Net primarily comprises a channel extraction block, a channel fusion block, an adaptive feature extraction block, and a bidirectional feature pyramid fusion (BFP) block. Within these components, the channel extraction block employs a Swin-T block, a ResNeXt block, and a ResNeXt_ECA block in parallel with the extract features in three distinct manners. The Swin-T block effectively captures global image information and contextual details at different scales through a hierarchical approach. Additionally, the local window mechanism captures local features, while reducing the computational complexity, thereby avoiding the high computational overhead associated with traditional global attention mechanisms. The ResNeXt block, a variant of the Residual Network (ResNet), introduces grouped convolutions in residual connections to enhance the network’s expressive power, while maintaining computational efficiency. Given that the Efficient Channel Attention (ECA) mechanism improves the ability to model relationships between channels without significantly increasing the computational complexity, we designed the ResNeXt_ECA block to integrate ECA into the ResNeXt network. This integration efficiently captures dependencies between feature channels, further enhancing the network’s performance, particularly when processing multi-channel features and images with complex structures or details. Therefore, the channel fusion block integrates features from different stages of the extraction process, along the channel dimension, enhancing feature diversity. To address redundancy in the resulting high-dimensional features, we designed the adaptive feature extraction block (depicted in Figure 2), which includes two 1 × 1 ordinary convolutions, a 3 × 3 depth-separable convolution, and a sequence of batch normalization and ReLU activation functions. This design effectively reduces the channel dimensions, filtering out irrelevant or less relevant features. Finally, the bidirectional feature pyramid fusion block bidirectionally merges features of various scales. This enhances ECF-Net’s ability to accurately segment lesions with different scales in COVID-19 CT images.

3.1. Channel Extraction Block

In order to integrate both key local information and the global context effectively, we propose a multi-path parallel channel extraction block that leverages the diverse learning capabilities of the CNN and the Transformer, as illustrated in Figure 3.

3.1.1. Channel Extraction 1

As illustrated in (a), channel extraction 1 employs the Swin Transformer architecture utilizing a shift window. This choice is motivated by the significant computational expense and complexity associated with the traditional Multi-Head Self-Attention (MSA) mechanism in Transformers. In contrast, Swin-T introduces the Window Multi-Head Self-Attention Mechanism (W-MSA), which confines the attention computation within a window rather than across the entire global context, resulting in a notable reduction in the computational load. Furthermore, the Shifted Window Mechanism (SW-MSA) captures a broader range of global context by spanning the windows, while maintaining lower computational complexity. In (b), the Swin-T block is composed of two sub-layers: the Multi-Head Self-Attention Layer (W-MSA or SW-MSA) and the Multilayer Perceptron Layer (MLP). Each sub-layer is preceded by a LayerNorm (LN) layer and is interconnected via residual connections. For an input image $\mathrm{I}\in \mathrm{R}(\mathrm{H}\times \mathrm{W}\times \mathrm{C})$ and a window size, the computational complexity of each window is detailed in Equation (1).

$${\mathsf{\Omega }\left(\mathrm{MSA}\right)=4\mathrm{hwC}}^2{+2\left(\mathrm{hw}\right)}^4\mathrm{C}$$

(1)

The total number of windows is $\mathrm{H}/\mathrm{h}\times \mathrm{W}/\mathrm{w}$, so the computational complexity of the W-MSA is as shown in Equation (2).

$$\begin{array}{ll}\mathsf{\Omega }\left(\mathrm{W}\text{-}\mathrm{MSA}\right)& =(\mathrm{H}/\mathrm{h}\times {\mathrm{W}/\mathrm{w}\left)\right(4\mathrm{hwC}}^2{+2\left(\mathrm{hw}\right)}^4\mathrm{C})\\ & {=4\mathrm{HWC}}^2{+2\left(\mathrm{hw}\right)}^4\mathrm{HWC}\end{array}$$

(2)

To address the limitations of the W-MSA, the SW-MSA facilitates information exchange and fusion among neighboring windows by shifting the window position across each layer. The calculation method for the SW-MSA is depicted in Equation (3).

$$\begin{array}{l}{\hat{\mathrm{z}}}^{\mathrm{l}}=\mathrm{W}\text{-}\mathrm{MSA}(\mathrm{LN}({\mathrm{z}}^{\mathrm{l}-1}))+{\mathrm{z}}^{\mathrm{l}-1},\\ {\mathrm{z}}^{\mathrm{l}}=\mathrm{MLP}(\mathrm{LN}({\hat{\mathrm{z}}}^{\mathrm{l}}))+{\hat{\mathrm{z}}}^{\mathrm{l}},\\ {\hat{\mathrm{z}}}^{\mathrm{l}+1}=\mathrm{SW}\text{-}\mathrm{MSA}(\mathrm{LN}({\mathrm{z}}^{\mathrm{l}}))+{\mathrm{z}}^{\mathrm{l}},\\ {\mathrm{z}}^{\mathrm{l}+1}=\mathrm{MLP}(\mathrm{LN}({\hat{\mathrm{z}}}^{\mathrm{l}+1}))+{\hat{\mathrm{z}}}^{\mathrm{l}+1},\end{array}$$

(3)

where ${\hat{\mathrm{z}}}^{\mathrm{l}}$ and ${\mathrm{z}}^{\mathrm{l}}$ represent the output features after the W-MSA and the MLP in the layer, respectively.

3.1.2. Channel Extraction 2

Since the original ResNet [30] employs standard convolutional operations, each layer’s filters are applied to every input channel. This significantly increases the network parameters and computational load. To address this, we propose ResNeXt as an alternative channel extraction network. ResNeXt enhances feature extraction by incorporating grouped convolutions and parallel paths, while maintaining a manageable computational cost. As illustrated in (c), a feature map with 256 channels is fed into the ResNeXt block, where it is processed through 32 branches. Each branch first performs a 1 × 1 convolution to generate a feature map with 4 channels, followed by 3 × 3 and 1 × 1 convolutions to restore the channel count to 256. Finally, the output features from all paths are added together to create a fused output feature, which is residually connected to the initial input features to produce the final output feature.

3.1.3. Channel Extraction 3

To better capture and emphasize locally useful features and reduce the loss of critical local information after a series of convolutional pooling operations, we propose introducing the Efficient Channel Attention (ECA) mechanism into ResNeXt. This mechanism dynamically adjusts the channel weights, while achieving diversified feature extraction through a multi-path structure, enhancing the validity and importance of features across different paths. Consequently, this improves the network’s overall feature expression capability, leading to better recognition of the salient features in lesion regions. Specifically, as illustrated in (d), for the input feature map $\mathrm{I}\in \mathrm{R}(\mathrm{H}\times \mathrm{W}\times \mathrm{C})$, the spatial information of each channel is aggregated into a scalar using global average pooling (GAP) to indicate the channel’s overall activation degree. The channel descriptor s is then generated, as calculated in Equation (4).

$${\mathrm{s}}_{\mathrm{c}}={\displaystyle \frac{1}{\mathrm{H}\times \mathrm{W}}}{\displaystyle {\sum }_{\mathrm{i}=1}^{\mathrm{H}}{\displaystyle {\sum }_{\mathrm{j}=1}^{\mathrm{W}}{\mathrm{I}}_{\mathrm{i},\mathrm{j},\mathrm{c}}}}$$

(4)

Here, ${\mathrm{s}}_{\mathrm{c}}$ denotes the global average of the c channel, H and W denote the height and width of the feature map, respectively, and c denotes the number of channels.

Then, a one-dimensional convolution operation is performed on the channel descriptor s to generate $\hat{\mathrm{s}}$, which captures the local interactions between channels. The $\hat{\mathrm{s}}$ is then passed through a sigmoid activation function to generate the weight coefficients for each channel, W. Finally, these weight coefficients W are used to adjust the weights of each channel in the input feature map, redistributing the importance of each channel to obtain the output feature map O. The computation method is shown in Equation (5).

$$\begin{array}{l}\mathrm{k}=\mathsf{\phi }\left(\mathrm{C}\right)={⌊{\displaystyle \frac{{\log }_2\left(\mathrm{C}\right)}{\mathsf{\gamma }}}+{\displaystyle \frac{\mathrm{b}}{\mathsf{\gamma }}}⌋}_{\mathrm{odd}}\\ \hat{\mathrm{s}}=\mathrm{Conv}1\mathrm{D}(\mathrm{s},\mathrm{k})\\ \mathrm{w}=\mathsf{\sigma }\left(\hat{\mathrm{s}}\right)\\ \mathrm{O}=\mathrm{w}\times \mathrm{I}\end{array}$$

(5)

Here, k denotes the convolution kernel size, adaptively determined by the number of channels, $\mathsf{\phi }$ denotes the adaptive function, C denotes the number of channels in the input feature map, b and $\mathsf{\gamma }$ are hyperparameters, and ⌊ ⌋ denotes rounding down to an odd number to ensure that the convolution kernel size is odd. Moreover, s and $\hat{\mathrm{s}}$ represent the channel descriptors before and after one-dimensional convolution, respectively. W denotes the channel weight coefficient. The sigmoid activation function is represented by σ. Finally, I and O denote the input feature map and output feature map, respectively.

3.2. Bidirectional Feature Pyramid Fusion Block (BFP)

To enhance the feature expression capability of the network, we propose a bidirectional feature pyramid fusion block. This block achieves comprehensive fusion and enhancement of the feature information by combining feature maps with different scales through top-down and bottom-up paths, as shown in Figure 4.

In the top-down path, the process starts from the high-level low-resolution feature map, gradually increasing the resolution through up-sampling operations, and fuses it with the low-level high-resolution feature map to enhance the global semantic understanding of the low-level feature map. Conversely, the bottom-up path starts from the low-level feature maps, gradually reducing their resolution through downsampling operations, and fuses them with the high-level feature maps. This process utilizes the detailed information of the low-level feature maps to enrich the spatial resolution of the high-level feature maps. This bidirectional fusion mechanism allows each layer of the feature map to merge with neighboring layers, ensuring that each layer in the feature pyramid contains information from different scales. This capability enables the network to better handle features of varying sizes and shapes, improving segmentation accuracy, as well as the network’s robustness and generalization ability.

3.3. BCE Loss

Since our study focuses on COVID-19 CT image segmentation, it addresses a pixel-level binary classification problem, specifically identifying the foreground (pneumonia foci) and the background (normal lung tissue). Therefore, using the binary cross-entropy loss function directly measures the difference between the predicted probabilities of the foreground and background and the true labels. The calculation formula is shown in Equation (6).

$$\mathrm{Loss}={\displaystyle \frac{1}{\mathrm{N}}}{\displaystyle {\sum }_{\mathrm{i}=1}^{\mathrm{N}}(-{[\mathrm{y}}_{\mathrm{i}}{\log (\mathrm{p}}_{\mathrm{i}})+(1-{\mathrm{y}}_{\mathrm{i}}\left)\log \right(1-{\mathrm{p}}_{\mathrm{i}}\left)\right])}$$

(6)

Here, N denotes the number of samples in each batch; y denotes the true label value, which takes the value of 0 or 1; p represents the probability that the network predicts the foreground; 1 − p represents the probability that the network predicts the background; log(p) measures the difference between the probability of predicting the foreground and the true label y; and log(1 − p) measures the difference between the probability of predicting the background and the true label 1 − y.

4. Experiments and Analysis of the Results

4.1. Datasets

We performed validation on three datasets: the COVID-19 CT Segmentation Dataset, the COVID-19 CT Lung and Infection Segmentation Dataset, and MosMedData. The COVID-19 CT Segmentation Dataset consists of two subsets, Dataset_A and Dataset_B. To ensure the fairness of the experiments, the datasets were divided into training and testing sets based on the types of cases. The parameter information of the datasets is presented in

Table 1. The parameter information of the datasets.

Dataset	No. of Cases	Total No. of Samples	No. of Samples in Training Set/Case	No. of Samples in Test Set/Case	Image Size	Image	Mask
Dataset_A	>40	100	80	20	512
Dataset_B	9	373	316/7	57/2	512
MosMed	50	785	626/30	159/20	512
Inf	20	1844	1513/12	331/7	512

Inf: the COVID-19 CT Lung and Infection Segmentation Dataset.

.

Dataset_A contains 100 annotated CT images, with 80 used for training and 20 for testing, while Dataset_B contains 373 images, with 316 used for training and 57 for testing. The COVID-19 CT Lung and Infection Segmentation Dataset includes 20 cases totaling 1844 images, from which 12 cases (1513 images) were used for training and 7 cases (331 images) for testing. MosMedData consists of 50 cases totaling 785 images, with 30 cases (626 images) used for training and 20 cases (159 images) used for testing. The size of the CT images in each of the four datasets is 512 × 512.

4.2. Experimental Environment

Our experimental environment is based on a Windows 10 64-bit operating system (Manufacturer: Microsoft Corporation Country: United States Region: Washington, Redmond) with an Intel(R) Core(TM) i9-10885H processor (Manufacturer: Intel Country: United States Region: Santa Clara, CA, USA), using the deep learning open-source framework PyTorch 1.7.1, and training accelerated by an NVIDIA RTX 3090 GPU (Manufacturer: NVIDIA Corporation Country: United States Region: Santa Clara, CA, USA). We used the AdamW optimizer, with an initial learning rate (LR) set to 1 × 10⁻⁴, a momentum range of (0.9, 0.999), the eps set to 1 × 10⁻⁸, and a weight decay of 0.05. A poly learning rate strategy was employed, decaying by a power of 0.9 after each iteration. We performed 50,000 iterations for all four datasets, with the input images uniformly adjusted to a resolution of 512 × 512.

4.3. Evaluation Metrics

To comprehensively analyze the COVID-19 image segmentation performance, we use nine evaluation metrics: the mean intersection over union (mIoU), sensitivity (SEN), specificity (SPC), the F-measure (F1), Dice’s similarity coefficient (DSC), the Hausdorff distance (HD), the mean absolute error (MAE), the number of floating-point operations (FLOPs), and the number of parameters (Params). These metrics evaluate the segmentation network model’s performance from various perspectives: the mIoU and DSC assess the overlap between the model’s segmentation results and the ground truth labels, SEN and SPC gauge the model’s ability to identify infected and normal regions, F1 combines precision and sensitivity, HD measures the geometric accuracy by calculating the boundary discrepancies, and MAE examines the magnitude of differences between the predicted and true values. Meanwhile, FLOPs and Params evaluate the model’s efficiency and resource requirements in terms of computational load and the number of parameters, respectively.

4.4. Evaluation Results

To verify the effectiveness of the proposed method, we conducted comparative experiments with other state-of-the-art methods on different COVID-19 datasets. The results of the experiments on COVID-19 CT Segmentation Dataset_A are shown in

Table 2. Experimental results for Dataset_A.

Method	FLOPs (G)	Params (M)	mIoU (%)	SEN (%)	SPC (%)	F1 (%)	DSC (%)	HD	MAE (%)
APCNet	205	53.98	68.45	59.09	97.13	60.04	59.68	115.00	0.53
DANet	211	47.45	79.63	69.95	99.04	81.5	76.75	94.09	0.28
DeepLabv3	270	65.74	82.06	75.95	98.95	82.81	80.1	49.49	0.26
FCN	198	47.12	80.86	76.46	98.62	79.87	78.56	51.64	0.28
UNet	23.77	13.40	78.42	72.55	95.52	75.37	73.10	44.63	6.88
GCNet	198	47.25	81.19	77.73	98.56	79.83	79.03	43.62	0.27
PSPNet	179	46.60	76.02	66.26	98.56	75.24	71.60	90.74	0.35
Segformer	7.88	3.71	80.06	72.17	98.92	80.89	77.39	57.49	0.28
Swin-T	236	58.94	78.06	66.34	99.09	80.41	74.49	97.47	0.31
UPerNet	237	64.04	76.26	69.01	98.28	74.02	72.06	92.97	0.36
ViT	442	142	76.88	72.57	98.02	73.37	73.07	77.83	0.36
TransUNet	33.49	110.52	77.83	75.87	94.40	73.87	73.27	50.58	7.32
VOLO	7.66	26.78	80.04	77.23	95.44	75.11	75.15	42.12	6.34
PoolFormer	9.98	57.71	80.29	76.25	95.43	76.16	75.21	41.38	6.26
CSWin	4.65	23.03	80.50	78.51	94.92	75.33	75.48	41.93	6.29
UNeXt	0.44	1.47	73.90	68.93	93.76	69.87	67.51	55.93	8.82
MRL-Net	20.01	85.06	82.75	77.78	96.49	79.47	78.13	35.67	5.29
BiseNetv2	7.64	2.20	79.44	72.63	98.69	79.15	76.57	47.10	0.30
AFFormer	2.93	2.16	81.93	66.01	99.23	73.77	72.13	43.47	2.49
ECF-Net	261	98	84.36	80.38	99.10	84.79	83.08	43.77	0.24

Bold indicates the best results.

, which includes APCNet [31], DANet [32], DeepLabv3, FCN, UNet, GCNet [33], PSPNet [34], Segformer, Swin, UPerNet [35], ViT, TransUNet, VOLO [36], PoolFormer [37], CSWin, UNeXt [38], MRL-Net [39], BiseNetv2 [40], and AFFormer [41], totaling 19 methods. To ensure the fairness of the experiment, all training parameters and the experimental environment were kept the same. For the TransUNet, VOLO, PoolFormer, CSWin, and UNeXt methods, we referred to the experimental data in MRL-Net. As shown in Table 2, our ECF-Net achieves optimal performance in regard to nearly all the metrics, with an mIoU of 84.36% and a DSC of 83.08%, significantly outperforming the other methods. Additionally, the MAE is just 0.24%, indicating a minimal deviation between the predicted and actual values. Compared to CNN-based networks, such as UNet, DeepLabv3, and PSPNet, ECF-Net improves the mIoU by 5.94%, 2.30%, and 8.34%, respectively. This suggests that CNN-only networks, while capable of capturing local and multi-scale features, are limited in their ability to capture global contextual information. These networks typically rely on downsampling and upsampling to expand the receptive fields, which can lead to a loss of spatial resolution. In contrast, our network leverages the Swin Transformer and an efficient channel attention mechanism, enabling better global contextual information capture, while preserving detailed features. When compared to Transformer-based networks, such as Swin-T, ViT, and Segformer, ECF-Net achieves an mIoU improvements of 3.05%, 7.48%, and 4.30%, respectively. This is because Transformer-based networks often struggle with processing fine-grained information. ECF-Net, however, combines the strengths of both CNN and Transformer architectures, allowing for more accurate segmentation of lesion regions at different scales through its channel extraction blocks and multi-scale feature fusion.

In comparison with hybrid networks like TransUNet, PoolFormer, and AFFormer, which combine CNN and Transformer architectures, ECF-Net improves the mIoU by 6.53%, 4.07%, and 2.43%, respectively. This is because these networks typically combine CNN and Transformer features through feature splicing or weighting, which may be insufficient for fine-grained tasks, leading to limited feature fusion. Often, features are processed independently at different stages, causing a potential loss of important details or global contextual information during integration. In contrast, ECF-Net’s channel fusion block adopts a more sophisticated feature fusion approach, integrating extracted features at a deeper level to generate a high-dimensional feature space, thereby retaining more details and global information.

Compared to the dual-branch CNN-based BiseNetv2, ECF-Net improves the mIoU by 4.92%. BiseNetv2 uses a dual-branch structure to separate spatial and semantic information, improving the processing efficiency and segmentation to some extent. However, this separation may lead to inadequate information integration, particularly when dealing with complex multi-scale features. The dual-branch structure may also struggle to fully leverage global information. In contrast, ECF-Net fully integrates spatial and semantic information through its channel enhancement mechanism and multi-scale feature fusion block, adaptively filtering redundant information during feature fusion. The bidirectional feature pyramid fusion block further optimizes multi-scale feature integration, enabling the network to better handle complex image scenes and diverse lesion regions.

Additionally, DeepLabv3’s mIoU is 1.74% and 3.64% higher than that of FCN and UNet, respectively, demonstrating the effectiveness of its atrous convolution and multi-scale atrous spatial pyramid pooling (ASPP) module. These components allow the network to expand the receptive field without losing resolution, efficiently capturing global contextual information through multi-scale feature fusion, thereby enhancing segmentation performance. Swin-T outperforms ViT with a 1.18% higher mIoU and a 1.42 higher DSC, while also reducing the number of parameters and computation by 83.06 M and 206 G, respectively. This improvement in segmentation accuracy, combined with a significant reduction in the parameters and computation, is attributed to Swin-T’s efficient hierarchical design and sliding window attention mechanism, which is why it has been adopted as one of the channel extraction blocks in ECF-Net.

To intuitively demonstrate the differences in segmentation performance between our method and other state-of-the-art methods, five CT images were randomly selected from Dataset_A, and five different networks were tested to visualize the segmented lesion regions, as shown in Figure 5.

As shown in Figure 5, in the first and fourth COVID-19 CT images, the other four methods have missed detections, whereas our method accurately identifies tiny lesions. We also observe that Swin-T has a high overall leakage rate and DeepLabv3 exhibits more misdetections. Although our method has some discrepancies compared to the real values, it maintains a high overall fit to the real values, accurately recognizing and segmenting even tiny and fuzzy-bordered lesions. This indicates that our proposed channel extraction strategy, through three pathways, enables the network to capture both spatial detail information and contextual semantic information. Additionally, the bidirectional feature pyramid fusion block enhances the focus on boundary features, effectively addressing the difficulty of recognizing tiny lesions with fuzzy boundaries.

The experimental results for the COVID-19 CT Segmentation Dataset_B are shown in

Table 3. Experimental results for Dataset_B.

Method	FLOPs (G)	Params (M)	mIoU (%)	SEN (%)	SPC (%)	F1 (%)	DSC (%)	HD	MAE (%)
APCNet	205	53.98	83.60	71.92	99.89	87.24	80.79	48.49	0.57
DANet	211	47.45	77.99	59.04	83.78	83.78	72.41	49.42	0.75
DeepLabv3	270	65.74	76.32	54.99	99.95	82.94	69.66	76.89	0.80
FCN	198	47.12	80.31	63.38	99.94	86.31	76.0	55.10	0.66
UNet	23.77	13.40	76.42	51.96	89.73	65.73	56.28	40.78	0.75
GCNet	198	47.25	83.03	70.49	99.90	87.04	80.0	34.27	0.58
PSPNet	179	46.60	83.25	71.06	99.89	87.10	80.3	47.56	0.58
Segformer	7.88	3.71	85.29	76.06	99.88	88.01	83.11	48.98	0.51
Swin-T	236	58.94	82.09	67.36	99.93	87.41	78.64	38.27	0.61
UPerNet	237	64.04	84.39	72.99	99.90	88.34	81.88	37.45	0.54
ViT	442	142	74.91	55.07	99.85	77.66	67.31	122.96	0.89
TransUNet	33.49	110.52	78.12	57.68	92.03	60.05	55.15	44.92	0.79
VOLO	7.66	26.78	80.15	59.20	86.15	57.83	55.63	33.43	0.71
PoolFormer	9.98	57.71	80.49	66.10	91.17	61.30	60.47	34.66	0.71
CSWin	4.65	23.03	79.98	59.56	86.30	61.66	58.59	37.14	0.65
UNeXt	0.44	1.47	73.96	58.00	90.04	50.67	49.20	40.12	1.08
MRL-Net	20.01	85.06	82.75	61.54	89.68	68.46	62.74	36.28	0.56
BiseNetv2	7.64	2.20	79.58	64.62	99.86	82.84	74.92	64.67	0.72
AFFormer	2.93	2.16	76.32	48.65	99.93	56.82	59.20	50.62	0.75
ECF-Net	261	98	87.15	85.44	99.75	85.65	85.57	29.91	0.48

Bold indicates the best results.

. The mIoU of ECF-Net reaches 87.15%, the Dice value is 85.57%, and the Hausdorff distance (HD) is 29.91%. Compared to Swin-T and ViT, which are based solely on the Transformer architecture, ECF-Net improves the mIoU by 5.06% and 12.24%, respectively, and reduces the HD value by 8.36% and 37.40%, indicating a significant reduction in bias between the predicted and actual boundaries. Compared to full convolutional networks, such as FCN, UNet, and DeepLabv3, ECF-Net improves the mIoU by 6.84%, 10.73%, and 10.83%, respectively. These results demonstrate that our method offers better smoothness and higher segmentation accuracy.

Table 4. Experimental results for Inf.

Method	FLOPs (G)	Params (M)	mIoU (%)	SEN (%)	SPC (%)	F1 (%)	DSC (%)	HD	MAE (%)
APCNet	205	53.98	78.03	63.37	99.81	79.32	72.48	52.11	0.76
DANet	211	47.45	77.80	63.73	99.78	78.3	72.11	49.53	0.78
DeepLabv3	270	65.74	72.99	49.34	99.91	77.48	63.83	81.43	0.88
FCN	198	47.12	56.05	16.61	99.66	33.27	24.18	162.08	0.16
UNet	23.77	13.40	80.73	76.65	97.08	70.29	68.01	35.47	0.82
GCNet	198	47.25	81.04	69.58	99.82	82.46	77.11	49.71	0.65
PSPNet	179	46.60	70.38	42.75	99.95	76.09	58.87	69.34	0.94
Segformer	7.88	3.71	73.20	54.32	99.76	72.25	64.29	70.93	0.95
Swin-T	236	58.94	83.33	73.15	99.85	85.51	80.41	33.64	0.56
UPerNet	237	64.04	74.61	53.69	99.88	78.14	66.74	59.50	0.84
ViT	442	142	70.80	67.31	99.06	56.58	60.18	93.32	0.14
TransUNet	33.49	110.52	80.79	70.39	95.74	70.44	66.42	32.19	0.77
VOLO	7.66	26.78	79.91	96.67	95.26	67.70	68.30	24.29	0.85
PoolFormer	9.98	57.71	78.36	77.81	94.91	63.69	65.92	18.34	0.98
CSWin	4.65	23.03	81.40	79.28	95.08	69.51	69.69	20.19	0.79
UNeXt	0.44	1.47	76.41	67.92	92.21	62.15	58.78	24.02	1.04
MRL-Net	20.01	85.06	83.24	81.82	96.38	72.95	73.30	16.99	0.70
BiseNetv2	7.64	2.20	60.6	24.01	99.89	53.94	36.76	121.65	1.30
AFFormer	2.93	2.16	78.72	67.48	99.85	69.69	70.53	50.50	0.72
ECF-Net	261	98	83.73	77.92	99.84	82.95	80.99	52.35	0.72

Bold indicates the best results.

and

Table 5. Experimental results for MosMed.

Method	FLOPs (G)	Params (M)	mIoU (%)	SEN (%)	SPC (%)	F1 (%)	DSC (%)	HD	MAE (%)
APCNet	205	53.98	70.95	53.01	99.85	64.18	59.48	63.03	0.41
DANet	211	47.45	66.80	38.34	99.92	63.09	50.79	82.20	0.42
DeepLabv3	270	65.74	68.53	48.73	99.82	58.8	54.58	75.92	0.46
FCN	198	47.12	69.93	47.02	99.90	66.19	57.42	76.64	0.40
UNet	23.77	13.40	71.86	70.77	95.76	51.54	51.36	35.49	0.45
GCNet	198	47.25	67.83	40.16	99.93	65.64	53.02	88.70	0.41
PSPNet	179	46.60	66.70	36.62	99.95	65.48	50.54	80.44	0.41
Segformer	7.88	3.71	70.56	45.09	99.94	63.58	58.66	47.62	0.36
Swin-T	236	58.94	73.14	58.65	99.84	67.05	63.63	61.92	0.38
UPerNet	237	64.04	69.42	43.81	99.93	68.05	56.36	75.27	0.39
ViT	442	142	65.30	37.54	99.87	56.3	47.42	101.63	0.48
TransUNet	33.49	110.52	72.97	55.05	91.64	54.99	50.04	36.51	0.32
VOLO	7.66	26.78	71.92	58.73	87.24	44.41	44.64	25.97	0.39
PoolFormer	9.98	57.71	71.36	66.23	88.89	46.08	48.05	30.12	0.44
CSWin	4.65	23.03	72.86	67.20	91.94	50.50	50.80	31.68	0.39
UNeXt	0.44	1.47	67.55	50.11	84.64	43.90	39.93	36.39	0.45
MRL-Net	20.01	85.06	74.81	72.48	93.72	57.01	56.67	27.88	0.34
BiseNetv2	7.64	2.20	64.98	38.76	99.84	53.27	46.72	79.25	0.51
AFFormer	2.93	2.16	72.64	51.25	99.89	55.37	52.52	51.65	0.37
ECF-Net	261	98	75.58	71.32	99.78	66.72	68.38	50.85	0.38

Bold indicates the best results.

present the experimental results for the COVID-19 CT Lung and Infection Segmentation Dataset and MosMedData, respectively. Our method achieves the highest mIoU value for both datasets, demonstrating superior segmentation accuracy compared to other state-of-the-art methods. However, the experimental results for MosMedData are not as strong as those for the other three datasets. This is attributed to the fact that the lesions in MosMedData are very small and low resolution, which increases the difficulty of the recognition and impacts the segmentation results. Nonetheless, our method still delivers the best performance metrics relative to several other methods, further demonstrating its strong generalization ability.

4.5. Ablation Study

To thoroughly validate the effectiveness of each component in our proposed method, we conducted a series of ablation experiments on the COVID-19 CT Segmentation Dataset_B. These experiments involved varying the channel extraction blocks, including and excluding feature generation blocks, incorporating and omitting bidirectional feature pyramid fusion blocks, and testing different loss functions.

4.5.1. Ablation Studies Involving Channel Extraction Blocks

From

Table 6. Ablation of different extraction blocks.

Method	mIoU (%)	SEN (%)	SPC (%)	F1 (%)	DSC (%)	HD	MAE (%)
1/wo	82.09	67.36	99.93	87.41	78.64	38.27	0.61
1+2/w	84.91	79.79	99.81	84.43	82.62	78.96	0.74
1+3/w	85.28	78.44	99.70	86.18	83.11	52.03	0.72
1+2+3/w	87.15	85.44	99.75	85.65	85.57	29.91	0.48

1: extract1; 2: extract2; 3: extract3; wo: without; w: with. Bold indicates the best results

, it is evident that the mIoU value for Extract1, which uses only the Swin-T architecture for channel extraction, is 82.09%. This is 5.06% lower than the mIoU achieved by our proposed method, which employs three channel extraction blocks in parallel. The mIoU values for extract1 + extract2 and extract1 + extract3, which utilize two channel extraction blocks in parallel, are 84.91% and 85.28%, respectively. These are lower than our method’s mIoU by 2.24% and 1.87%, respectively. This suggests that a single channel extraction block alone cannot adequately capture both global and local information. Although using two channel extraction blocks in parallel addresses the limitations of a single block, extract1 + extract3 achieves a 0.37% higher mIoU than extract1 + extract2. This indicates that the extensive convolutional pooling operations during CNN-based methods may lead to the loss of some local detailed features. Therefore, incorporating the ECA channel attention mechanism helps capture fine-grained features and effectively addresses this issue.

4.5.2. Ablation Studies Involving Adaptive Feature Generation Block

From

Table 7. Ablation with/without adaptive feature generation block.

Method	mIoU (%)	SEN (%)	SPC (%)	F1 (%)	DSC (%)	HD	MAE (%)
wo	86.34	78.22	99.88	88.79	84.50	49.69	0.68
w	87.15	85.44	99.75	85.65	85.57	29.91	0.48

Bold indicates the best results.

, it can be observed that the mIoU and Dice values for the network without the adaptive feature generation block are 86.34% and 84.50%, respectively. These values are 0.81% and 1.07% lower than those for the network with the adaptive feature generation block. This indicates that the addition of this module effectively maps high-dimensional features to a lower-dimensional space, eliminating redundant information and emphasizing key features, thereby improving segmentation accuracy.

4.5.3. Ablation Studies Involving BFP

Table 8. Ablation with/without BFP.

Method	mIoU (%)	SEN (%)	SPC (%)	F1 (%)	DSC (%)	HD	MAE (%)
wo	85.72	75.84	99.89	89.22	83.68	58.73	0.68
w	87.15	85.44	99.75	85.65	85.57	29.91	0.48

Bold indicates the best results.

presents the experimental results for the network with and without the bidirectional feature pyramid fusion block. The introduction of this block results in a 1.43% increase in the mIoU and a 28.82% reduction in the Hausdorff distance (HD). This improvement is attributed to the block’s ability to perform top-down and bottom-up bidirectional feature fusion, effectively integrating feature information across different scales and enhancing feature expression. The experimental results underscore the effectiveness of this approach.

4.5.4. Ablation Studies Involving the Loss Function

A series of ablation experiments were conducted to assess the impact of different loss functions on the network’s training phase, with the results presented in

Table 9. Ablation of different loss functions.

Method	mIoU (%)	SEN (%)	SPC (%)	F1 (%)	DSC (%)	HD	MAE (%)
FocalLoss	86.20	77.53	99.02	88.99	84.31	56.51	0.70
HDLoss	80.07	79.31	99.26	73.81	75.78	201.40	1.48
DiceLoss	86.50	80.74	99.84	87.48	84.83	47.18	0.71
BCELoss	87.15	85.44	99.75	85.65	85.57	29.91	0.48

Bold indicates the best results.

. The mIoU is the lowest with HDLoss, at 80.07%. The mIoU values for FocalLoss and DiceLoss are similar, with a difference of only 0.30%. CrossEntropy Loss, the binary cross-entropy loss function used in this study, yields the highest performance metrics overall, except for the SPC and F1 values, which are slightly lower than those achieved with DiceLoss. This indicates that the CrossEntropy Loss is more effective at optimizing the network and reducing boundary errors.

5. Discussion and Conclusions

In this study, we propose a multi-scale feature fusion network, ECF-Net, based on channel enhancement to accurately segment COVID-19 lesion regions in lung CT images. The experimental results demonstrate that our method exhibits excellent segmentation performance, with mIoU values of 84.36%, 87.15%, 83.73%, and 75.58% across four COVID-19 CT image datasets, validating its effectiveness and robustness. This improvement in segmentation accuracy has significant potential to enhance surgical planning, optimize doses for adjuvant radiation therapy, and automate pathology analysis, thereby improving the quality and efficiency of healthcare services. Despite these promising results, our method has the following shortcomings:

(1) The datasets used in our experiments are publicly available and no actual clinical trials have been conducted, which limits the validation of our method in real-world scenarios;

(2) The network’s parameter count and computational load reach 98 MB and 261 GB, respectively, reflecting its high complexity in terms of both computation and storage requirements. A high parameter count generally allows the model to capture more detailed features, but also increases the storage needs. This can be challenging in clinical environments, where storage resources may be limited. Additionally, the increased computational load affects both training and inference speeds. During training, the higher computational volume extends the training time, necessitating powerful resources and longer processing cycles, which can increase the development costs and affect research timelines. During inference, where real-time processing is essential for many medical applications, the high computational demand may lead to slower processing speeds, potentially undermining the efficiency of real-time applications.

In future work, we will focus on two key aspects for further research and improvement. First, in terms of the datasets, we plan to collect CT image data from various hospitals and devices to comprehensively evaluate the model’s applicability and robustness. Additionally, we intend to conduct tests in real clinical environments to assess the performance of the proposed network model across different healthcare organizations and devices. This approach will help identify and address potential issues the model may encounter during practical applications, such as image noise or quality variations generated by different devices. By resolving these challenges, we aim to reduce the risk of misdiagnosis and omission, thereby enhancing diagnostic accuracy, speeding up diagnosis, and optimizing treatment decisions to minimize adjustments caused by inaccurate diagnoses. Second, regarding network lightweighting, we will explore techniques, such as pruning, quantization, and sparsification, to reduce the number of network parameters and the computational overhead. We will also investigate hardware-based optimization solutions, including improving the inference speed of models using accelerated hardware like GPUs or FPGAs. Furthermore, we plan to utilize parallel computing and optimization algorithms to enhance inference efficiency. Developing lightweight models is crucial for reducing data processing and transmission latency, thus shortening the time from image acquisition to result output. This is especially important in scenarios, such as emergency medicine and high-throughput screening, where improved efficiency and response times are vital.