Domain Discrimination Expert Weighted Network for Multi-Source Carotid Artery Plaque Classification

Abstract

The rupture of unstable plaques is a major cause of acute cardiovascular events. The early assessment of carotid plaques can significantly reduce the cardiovascular risks, so developing evaluation models suitable for data from different centers is of great clinical importance. This study leverages plaque datasets from multiple centers to develop a Weighted multi-source carotid artery plaque Unsupervised Classification Framework (WUCF). The multi-source domain adaptation module of the WUCF focuses on maintaining feature consistency between each independent source and target center, while also integrating a specialized domain discriminator expert. This ensures that the knowledge from each source center is effectively learned and combined for accurate predictions in the target domain. The experimental evaluation of WUCF, using datasets from three centers, has demonstrated the method’s superiority and robustness.

1. Introduction

Cardiovascular and cerebrovascular diseases, such as stroke, have increasingly become one of the leading causes of morbidity and mortality worldwide, and the World Health Organization has declared them a pandemic [1]. The atherosclerosis of carotid artery plaques is one of the primary contributors to cardiovascular diseases (CVD) [2]. Consequently, researchers have introduced the concept of vulnerable plaques, aiming to prevent cardiovascular diseases by identifying high-risk atherosclerotic plaques [3]. Clinically, ultrasound, as a non-invasive diagnostic tool, has become one of the primary methods for the preoperative assessment of carotid plaque due to the superficial location of the carotid arteries [4]. However, ultrasound imaging is significantly influenced by the operator’s technique and the equipment used, with different operators and devices variably affecting the imaging results [5]. Moreover, the lack of a uniform diagnostic gold standard means that the accuracy of diagnoses largely depends on the clinical experience and expertise of physicians, leading to poor universality [6].

Currently, deep learning has become an effective tool for auxiliary diagnosis and has been progressively applied in the field of plaque diagnosis, showing significant effectiveness in handling plaque data from single medical centers [7,8]. However, the ultrasound plaque image analysis in the real world often involves multiple data sources. These sources, primarily from different medical institutions, contain images with unique feature distributions. To date, there already have been a few studies focusing on multicenter auxiliary diagnostics with a single-source center [9], but research on multi-source centers has not yet been conducted. When expanding from a single source to multiple sources, the complexity of data distribution is significantly higher than in a single-source scenario. The first issue is that not only do the data between the source center and target center show feature differences, but there are also variations between the source centers themselves, which results in the decision boundaries of classifiers designed for single-source centers failing to effectively classify the data from multi-source centers. Secondly, the model faces the issue of inconsistent ultrasound image views, meaning that different source centers contain different views. For example, in the processing of carotid artery plaque images, some source centers might provide datasets that include both longitudinal and transverse images of the carotid artery, while others might offer images from only one angle, as shown in Figure 1. Although this diversity in multi-source center situations adds complexity to the classification task, it also provides a potential for model design—by integrating the “knowledge” of different imaging characteristics and views from different source centers, the model can offer a more enriched set of information sources for the target center.

To address the aforementioned challenges, this paper introduces a novel framework called the Weighted Unsupervised Classification Framework for Multi-Source Center Ultrasound Plaque Images (WUCF), which not only acknowledges the heterogeneity of the data across different source centers but also strives to explore how to efficiently integrate these data to enhance and optimize the classification performance in the target domain. Specifically, a new Multi-domain Adaption module with a domain Discrimination Expert (MADE) is designed in the WUCF. Considering that each source center possesses its unique knowledge, the model trains an expert classifier for each source center to better distinguish the domain characteristics of each source. Managing the integration of knowledge from multiple expert classifiers is crucial, as simply merging all source domains and applying a single-source unsupervised domain adaptation method may not be accurate. The MADE introduces a domain discrimination expert that uses coefficients learned by the domain discriminator to blend the predictions of domain experts. The contributions of this work are summarized as follows:

• The WUCF is proposed for the early detection of and risk reduction in acute cardiovascular events caused by unstable plaque ruptures. This model is designed to handle the diverse ultrasound data from various medical centers.
• A Multi-source domain Adaptation module with a domain Discrimination Expert (MADE) is designed to dynamically adjust the weights and evaluate the contributions of different source domain expert classifiers. This approach aims to refine the integration of the source domain features, reduce negative transfer, and enhance the generalization capabilities across diverse domains.
• The extensive experiments conducted on three centers datasets have demonstrated the efficacy of the proposed WUCF.

2. Related Work

Deep learning has increasingly become pivotal in medical image analysis, with particular success in carotid plaque classification within single-center studies [8,10,11,12]. Convolutional Neural Networks (CNNs) are highly proficient at extracting complex features from ultrasound images, significantly boosting the classification accuracy [13]. For example, Shen et al. [14] pioneered NDDR-LCS, a multitask learning approach that leverages CNNs and data from ultrasound reports to enhance the precision of plaque classification. This approach has led to substantial advancements in single-center plaque classification, offering essential diagnostic tools.

However, the models adapted to single-center data often cannot be applied to other datasets without experiencing a decline in performance, thus necessitating robust multicenter carotid plaque classification models [15]. Despite the limited number of studies specifically targeting multicenter plaque classification, significant progress has been observed in the multicenter studies within the various domains [16].

To address these challenges, Domain Adaptation [17] has become critical, such as Domain-Adversarial Neural Networks (DANNs) [18] and Maximum Classifier Discrepancy (MCD) [19]. A DANN employs adversarial training to minimize the domain discrepancies by forcing the model to learn features that are invariant across different domains, thereby enhancing the generalization. On the other hand, MCD utilizes a pair of classifiers in an adversarial setting to align the domain features effectively, focusing on maximizing the discrepancy between the classifier outputs to better adapt to new domains. These techniques extend the applicability of the machine learning models to multicenter data, ensuring that the advancements in single-center studies translate effectively into more complex multicenter environments.

These advancements in single-source domain adaptation pave the way for exploring more complex scenarios involving multiple source domains. As the medical datasets often come from various centers, each with distinct data characteristics, Multi-Source Domain Adaptation (MSDA) becomes essential.

Multi-Source Domain Adaptation (MSDA) represents a significant branch of machine learning that focuses on transferring knowledge from various source domains to a single target domain [20]. The necessity for MSDA arises due to the varying and complex data distributions across domains, which the single-source domain transfer learning models cannot sufficiently address. The approach involves integrating the data from multiple sources, often utilizing strategies that adjust for the differences in both the marginal and conditional probabilities relative to the target domain. This method ensures that the knowledge from diverse domains is effectively utilized to improve the predictive accuracy and model robustness. The significant contributions to this field include a two-stage weighting framework that strategically combines multi-source data [21]. However, this framework may struggle with the scalability and complexity of high-dimensional data, potentially limiting its effectiveness in more intricate scenarios. Moment Matching for Multi-Source Domain Adaptation (M3SDA) [22] is a technique that dynamically aligns feature distributions to bridge domain gaps effectively. Additionally, Guo et al. [23] explored distance-based measures to assess the domain dissimilarities, particularly in natural language processing tasks, enhancing the text classification under domain variation. Further innovations include the Multi-source Sentiment Generative Adversarial Network (MSGAN) [24], which addresses the visual sentiment classification challenges under multi-source scenarios. This method simply averages the prediction results from several classifiers without specifically adjusting the contribution of each classifier. As a result, it does not provide optimal predictions. A privacy-preserving Knowledge Distillation-based Decentralized Domain Adaptation is proposed [25] to facilitate the unsupervised multi-source domain adaptation through the knowledge distillation across the models from different sources, ensuring privacy and data integrity. These advancements underline the critical role of MSDA in leveraging the diverse data sources to address the domain adaptation challenges, significantly improving the effectiveness of the machine learning models across the varied applications. However, the existing methods often face limitations in handling high-dimensional data, computational efficiency, and the applicability across different types of data, particularly in the medical imaging domain.

3. Materials and Methods

3.1. Datasets

The WUCF was used to evaluate the proposed method using three ultrasound datasets collected from the three collaborating centers: A, B, and C.

Center A dataset: This dataset included 759 ultrasound images of 244 plaques from 156 patients. The plaque types were labeled as intima thickening, hypoechoic (30 images), mixed echogenicity (47 images), and hyperechoic (78 images). Hypoechoic and mixed echogenicity plaques (506 images) are classified as high-risk unstable plaques, while hyperechoic plaques (253 images) are classified as low-risk stable plaques.

Center B dataset: This dataset included 310 carotid ultrasound images from 126 patients with carotid atherosclerosis, collected from cooperating medical center B between 2010 and 2017. The images were categorized into four types: intima thickening (53 images), hypoechoic plaques (132 images), hyperechoic plaques (78 images), and mixed echogenicity plaques (47 images). Hypoechoic and mixed echogenicity plaques (178 images) are classified as high-risk unstable plaques, while hyperechoic plaques (132 images) are classified as low-risk stable plaques. B_L is a subset of B, with all the transverse views removed from B, leaving only the longitudinal views.

Center C dataset: The third dataset involved carotid ultrasound videos collected by cooperating institution C between 2020 and 2021, involving 216 patients. The dataset included 302 videos covering 119 hyperechoic plaques, 76 hypoechoic plaques, and 107 mixed echogenicity plaques. Each video contained 120 frames, with a resolution of 740 × 540 pixels per frame, totaling 36,240 ultrasound images. For this study, we extracted static images from the videos, selecting two frames every 30 frames to balance the data volume across the centers. Finally, center C contained 248 stable plaques and 386 unstable plaques, for a total of 634 ultrasound plaque images.

The dataset from all three datasets were annotated by radiologists with years of clinical experience. In this study, stable plaques are considered low risk, while unstable plaques are deemed high risk due to their risk of detachment [26]. All the datasets are summarized in

Table 1. Dataset details.

Centers	Stable		Unstable		Total
A	506	Transverse: 75 Longitudinal: 431	253	Transverse: 33 Longitudinal: 220	759
B	132	Transverse: 19 Longitudinal: 113	178	Transverse: 27 Longitudinal: 151	310
B_L	113	Longitudinal: 113	151	Longitudinal: 151	264
C	248	Longitudinal: 248	386	Longitudinal: 386	634

. For the purposes of this study, the test set comprises the entire dataset of the target center, ensuring that our evaluation reflects the model’s ability to generalize to new, unseen data.

3.2. Methods

Problem definition. In multi-source unsupervised domain adaptation, there are N distinct foundational source centers, with their associated data and labels denoted as ${\left\{\left(X_{Sj},Y_{Sj}\right)\right\}}_{j=1}^N$. Here, $X_{Sj}={\left\{x_i^{Sj}\right\}}_{i=1}^{\left|X_{Sj}\right|}$ represents samples from source domain center j, and $Y_{Sj}={\left\{y_i^{Sj}\right\}}_{i=1}^{\left|Y_{sj}\right|}$ represents the ground-truth labels for the carotid artery plaque ultrasound images. Additionally, the data from the target domain centers are represented as ${\left\{\left(X_{Tj}\right)\right\}}_{j=1}^N$, and notably, the data from the target centers do not carry labels.

The overall structure of the WUCF proposed is illustrated in Figure 2. Data from multiple source centers and target centers are input into the feature extractor $G$. The entire network is divided into a feature extraction module and a multi-source domain adaptation module, where all source domains share the weights of a single feature extraction module.

3.2.1. Feature Extraction Module

In this study, the model focuses on utilizing information from multiple source domains to facilitate the learning of domain-invariant representations during the transfer learning process. To achieve this goal, an efficient feature extractor G is employed, which primarily functions to map data from various source domains and the target domain into a common latent space. The feature extractor used in the WUCF is a traditional ResNet-50 network. ResNet-50 extracts various features such as edges, textures, and more complex patterns through its convolutional layers, capturing the low-level features in the initial layers and progressively the more abstract features in the deeper layers. These features are then transformed into a high-dimensional latent representation, enabling effective feature alignment and domain adaptation. This approach ensures the uniform processing of data from different domains and provides an effective means to capture and leverage the cross-domain commonalities. Thus, the core of feature extractor G lies in its ability to distill the key features from various source domains and transform these features into a unified representation, thereby simplifying the subsequent transfer learning tasks. This phase is represented as Equation (1).

$$feature=G\left(X_i^j\right)$$

(1)

3.2.2. Multi-Source Domain Adaptation Module with a Domain Discrimination Expert

In ultrasound examinations, plaque ultrasound images can be viewed in both transverse and longitudinal cuts, which necessitates special handling to unify the features from the multiple source centers in multi-source scenarios. Furthermore, plaque ultrasounds face challenges such as the imaging differences and low resolution between different centers. Inspired by MFSAN [27], this paper proposes a multi-source domain adaptation module that constructs independent classifier experts for each source domain and uses a domain discriminator to integrate the knowledge from various domain experts. The schematic diagram of the module is shown in Figure 3. This module is divided into three parts: two stages of domain alignment and a final stage of target label weighted prediction. The first stage consists of two fully connected layers (FC), the second stage is composed of expert classifiers corresponding to the centers, and the third stage is formed by a domain discriminator.

(1)

Distribution alignment phase for specific centers

The MADE includes two fully connected layers (FC), where the features from both the source center $i$ and the target center are input into these layers. The distance between them is calculated to further quantify the differences.

To better achieve the feature alignment in the first stage, the MADE employs the Maximum Mean Discrepancy (MMD) [28] distance as an estimate of the feature distance between different centers. MMD is a kernel two-sample test that rejects or accepts the null hypothesis $p=q$ based on the observed samples. The fundamental idea of MMD is that if the generating distributions are the same, all the statistics are identical. Formally, MMD defines the following difference measure. The correction of noise labels is as follows:

$$D_{\mathcal{H}}\left(p,q\right)\triangleq \parallel {\mathrm{E}}_p\left[\varphi \left(x^s\right)\right]-{\mathrm{E}}_q\left[\varphi \left(x^t\right)\right]{\parallel }_{\mathcal{H}}^2$$

(2)

Here, $\mathcal{H}$ is the reproducing kernel Hilbert space endowed with the feature kernel $k$. The function $\varphi$ represents the feature map that projects the original samples into the reproducing kernel Hilbert space.

$${\widehat{D}}_{\mathcal{H}}\left(p,q\right)=\Vert \frac{1}{n_s}{\displaystyle \sum }_{x_i\in {\mathcal{D}}_s}\varphi \left(x_i\right)-\frac{1}{n_t}{\displaystyle \sum }_{x_j\in {\mathcal{D}}_t}\varphi \left(x_j\right){\Vert }_{\mathcal{H}}^2$$

(3)

where ${\widehat{D}}_{\mathcal{H}}\left(p,q\right)$. ishe unbiased estimate of $D_{\mathcal{H}}\left(p,q\right)$, using Equation (3) to estimate the discrepancy between each source domain and target domain. The MMD loss is reformulated as shown in Equation (4).

$$\begin{array}{cc}{\mathcal{L}}_{mmd}& =\frac{1}{N}{\displaystyle \sum _{j=1}^N}\widehat{D}\left(H_j\left(F\left(X_{sj}\right)\right),H_j\left(F\left(X_t\right)\right)\right)\end{array}$$

(4)

Each specific feature extractor can learn a domain-invariant representation for a particular source domain center and the target domain by minimizing Equation (4).

(2)

Training of expert classifiers for specific centers

$C$ is a multi-output network composed of expert classifiers for $N$ specific centers, denoted as ${\left\{C_j\right\}}_{j=1}^N$. Each predictor $C_j$ is a softmax classifier that receives domain-invariant features processed by the feature extractor $F{C}_j\left(G\left(X_i^{sj}\right)\right)$ specific to the j-th source domain. The softmax classifier is a commonly used method that outputs the probabilities associated with each category, facilitating effective classification. Additionally, to optimize the performance of the predictors and enhance the classification accuracy, each classifier employs a classification loss function based on cross-entropy. This is particularly effective in multi-class problems, as it measures the discrepancy between the predicted probability distribution and the true label distribution, thereby guiding the network to adjust the parameters to improve the classification accuracy. The specific loss function is as follows:

$$\begin{array}{cc}{\mathcal{L}}_{cls}& ={\displaystyle \sum _{j=1}^N}{\mathrm{E}}_{x\sim X_{sj}}J\left(C_j\left(F{C}_j\left(G\left(x_i^{sj}\right)\right)\right),{\mathrm{y}}_i^{sj}\right)\end{array}$$

(5)

(3)

Domain discrimination expert

To more effectively utilize the domain knowledge from multiple source centers, the MADE introduces a domain discriminator expert to integrate the knowledge across the various sources. Specifically, the MADE employs the domain discriminator expert to identify which source domain center is most relevant to the input data, and combines this outcome with the outputs from the previous stage’s expert source domain classifiers to generate the final prediction. The architecture of the expert discriminator, as shown in Figure 4, consists of a fully connected layer, a BatchNorm layer, and a ReLU function. The BatchNorm layer helps reduce the shift in the distribution of the input data, which aids in alleviating the internal covariate shift, thereby making the network training process more stable.

Unlike traditional methods, the MADE does not force multiple source domains to exhibit the same feature representations in latent space; instead, it encourages each source domain to maintain its feature independence. This strategy makes the source domain discrimination expert $D$ more efficient in distinguishing between different source domains, allowing for the precise identification and extraction of the unique features from each source domain. This approach expands the coverage of source domain representations and allows the MADE to flexibly adjust its learning strategies when facing target domain data, by extensively covering and meticulously comparing the features with source domains, thus significantly enhancing the classification accuracy.

To train the domain discriminator expert, the model assigns a unique label to each source center and utilizes the cross-entropy loss function to optimize the domain discriminator. This loss function quantifies the discrepancy between the model’s output and the actual labels. Through this method, the domain discriminator can achieve higher discrimination efficiency when processing the features from different source domains. This step can be expressed mathematically as:

$$L_D={\mathbb{E}}_{\left(z,t\right)\sim \mathcal{D}}\left[CE\left(\mathcal{C}\left(FC\left(x\right)\right),t\right)\right]$$

(6)

The final prediction output for the target domain can be represented as:

$$y={{\displaystyle \sum }}_{j=1}^{N}G\left(X_i^t\right)D\left(C_j\left(F{C}_j\left(G\left(X_i^t\right)\right)\right)\right)$$

(7)

3.2.3. Training Steps

Given the distinct nature of the multicenter issues compared to the common fully supervised models, this section details the training process, including the training and testing phases of the model. The training phase consists of two steps:

Step A. The initial stage aims to build a foundational model that is better suited for the multicenter plaque classification. At this point, the model is trained using the plaque data from the source centers, with a particular focus on the source center with the largest dataset. The main goal of this step is to minimize the cross-entropy loss to optimize the preliminary performance of the model, with the specific objectives described as follows:

$$\underset{{\theta }_{FC},{\theta }_G,{\theta }_C}{\min }{\mathcal{L}}_{CE}$$

(8)

Step B. During this phase, the model is trained to include the data from the target center, and the detailed algorithmic process is displayed in Algorithm 1. Initially, a multicenter domain adaptation layer is implemented to reduce the gap between the target and source centers, and the Maximum Mean Discrepancy ${\mathcal{L}}_{MMD}$ is calculated. Subsequently, the classification ${\mathcal{L}}_{CLS}$ is calculated to independently train the expert classifiers corresponding to each source center. Additionally, the model involves training a domain discriminator expert, achieved by computing the ${\mathcal{L}}_D$. The detailed parameter settings for each loss function will be elaborated upon in the experimental section. The goal of this section is shown in Equation (9).

$$\underset{{\theta }_{FC},{\theta }_D,{\theta }_C}{\min }({\mathcal{L}}_{CLS}+\gamma {\mathcal{L}}_{MMD}+\alpha {\mathcal{L}}_D)$$

(9)

Algorithm 1. The proposed WUCF

Input: Data from source center 1 $\left\{x_{s1},y_{s1}\right\}$ and source center 2 $\left\{x_{s2},y_{s2}\right\}$; target center data $\left\{x_t\right\}$ and the number of training epochs $T$; feature extractor G; expert classifiers $\left\{C_1,C_2\right\}$; multicenter domain adaptation layers $\left\{{FC}_1,{FC}_2\right\}$; domain discriminator expert D. 1: for t = $1\to T$ do 2:   Extract features from multiple source domain centers and target domain images $G\left(x_{s1}\right), G\left(x_{s2}\right), \mathrm{and} G\left(x_t\right)$ using the feature extractor G. 3:   Pass the common latent representations $f_{x_{s1}}$ from the source center samples and $f_t$ from the target center samples to the domain-specific feature extractor $F{C}_1$. This process obtains the domain-specific representations for the source samples. Then, compute the ${\mathcal{L}}_{MMD}$ using Equation (4) to measure the domain alignment. 4:   Pass the common latent representations $f_{x_{s2}}$ from the source center samples and $f_t$ from the target center samples to the domain-specific feature extractor $F{C}_2$. This process obtains the domain-specific representations for the source samples. Then, compute the ${\mathcal{L}}_{MMD}$ using Equation (4) to measure the domain alignment. 5:   Pass $f_{x_{s1}}$ and $f_{x_{s2}}$ through expert classifiers $C_1,C_2$, respectively, and compute ${\mathcal{L}}_{cls}$ using Equation (5). 6:   Pass $f_{x_{s1}}$ and $f_{x_{s2}}$ through expert classifier D, respectively, and compute ${\mathcal{L}}_D$ using Equation (6). 5:   Minimize the loss and update the parameters of $C_1,C_2,G,D,F{C}_1,F{C}_2$ to optimize the model. 14: end for Output: The label prediction y

In regard to the testing phase of the model, the schematic diagram is shown in Figure 2, and the testing algorithm process is presented in Algorithm 2.

The plaque ultrasound images from the target center first pass through the feature extractor $G$ to extract the deep features. The obtained feature $f_t$ is then processed along three independent paths. Two of the paths involve passing $f_t$ through the fully connected layers $F{C}_1$ and $F{C}_2$ trained from two different source centers (for simplicity, the multiple fully connected layers within each center are described here as a whole; in the actual training, they are still two layers of fully connected layers), whose output features are then passed to two expert classifiers $C_1$ and $C_2$. These classifiers are trained to make highly accurate classification predictions for their respective source center data.

The third path includes a well-trained domain discrimination expert $D$, which assesses the probability $p_1$ and $p_2$ that each sample belongs to source center 1 or source center 2. When making predictions, the model does not simply choose the source center with the highest probability or average them, but weights the outputs of the two expert classifiers according to the probability assessment of the domain discrimination expert. This means that the output of each classifier is weighted by the probability of its corresponding source center being chosen as the correct source, and the weighted sum is then taken to obtain the final prediction $y$.

Algorithm 2. Testing the WUCF

Input: Target center data $\left\{x_t\right\}$; feature extractor G; expert classifiers $\left\{C_1,C_2\right\}$; multicenter domain adaptation layers $\left\{F{C}_1,F{C}_2\right\}$; domain discriminator expert D. 1: Extracting target domain image features $f_t=G\left(x_t\right)$ through G. 2: The features are obtained through the fully connected layers trained by two source centers $f_{t1}=F{C}_{\mathrm{s}1}\left(f_t\right), f_{t2}=F{C}_{\mathrm{s}2}\left(f_t\right).$ 3: The outputs of the two classifiers are obtained as $o_1=C_1\left(f_{t1}\right),o_2=C_2\left(f_{t2}\right).$ 4: The prediction outputs (probabilities) $p_1$ and $p_2$ are obtained from the domain expert discriminator D as $p_1,p_2=D\left(f_t\right)$. 5: The prediction output is calculated as the weighted sum of the two classification results $y=p_1\cdot o_1+p_2\cdot o_2$.

Output: The label prediction y

4. Results

4.1. Implementation Details

This study used accuracy, regression, and precision to evaluate the performance of the MUCF. These three metrics are defined as follows:

$$Accuracy=\frac{TP+TN}{TP+TN+FP+FN}$$

(10)

$$Recall=\frac{TP}{TP+FN}$$

(11)

$$Precision=\frac{TP}{TP+FP}$$

(12)

where TP, TN, FP, and FN represent the true positive, true negative, false-positive, and false-negative classifications, respectively.

The experiments presented in this paper are based on ResNet-50 and pre-training parameters from ImageNet [29]. The learning rate of the model is 0.002. According to the experimental results, $\alpha$. = 0.5, $\mathsf{\gamma }$ = 0.5. Additionally, several data preprocessing steps were applied. These steps included random horizontal and vertical flipping, random rotations, and normalization using standard mean and standard deviation values. Each image was resized to 227 × 227 for model training. The input size of the model was N × 3 × 227 × 227, where N is the batch size, 3 is the number of channels, and 227 represents the width and height of the image. We used stochastic gradient descent (SGD) with a momentum of 0.9, a weight decay of 10⁻⁴, and a batch size of 32. The number of iterations was set to 4000. All the experiments in this study were implemented using Pytorch-1.12.1 and conducted on a Linux server (Dell, Beijing, China), which included two Intel^® Xeon^® Gold 6226R CPUs @ 2.90 GHz (Intel Corporation, Santa Clara, CA, USA) and two GeForce GTX 3090 Ti GPUs (NVIDIA Corporation, Santa Clara, CA, USA).

4.2. Experimental Results

4.2.1. Model Comparison

This section compares the WUCF model with other methods, categorized into three types:

• Source-only methods: the source domain data are used for fully supervised training, with direct application to the target center data without any domain adaptation, using Resnet-50 as the base model.
• Source-combine methods: this involves combining source centers into a single source for single-source domain adaptation training—a DANN [18], which uses adversarial training to help models generalize across different domains by learning domain-invariant features and MCD [19], which leverages adversarial techniques between multiple classifiers to align domain features, thereby enhancing the generalization across different domains.
• Multi-source methods: MFSAN [27] aligns specific domain distributions and classifiers within a multi-source domain environment and averages the final classifier outputs to produce the ultimate prediction; MDAN [27] extends the adversarial learning strategies from addressing single-source center issues (DANN) to multi-source center domain adaptation problems; M3SDA [22] is a domain adaptation method that reduces the distribution discrepancies by matching the statistical moments in the feature spaces between multiple source domains and the target domain, with a particular emphasis on matching higher-order statistical properties to achieve comprehensive domain adaptation effects; AMTLDC [30] proposes a CNN-based feature extraction network, followed by the construction of classifiers and domain discriminators.

From

Table 2. Comparison of the model with other methods.

Standards	Methods	AC-B			AB-C			BC-A
		Acc	Pre	Rec	Acc	Pre	Rec	Acc	Pre	Rec
Source-only *	Resnet-50	65.47	66.41	65.71	68.82	69.47	68.82	60.27	59.35	62.41
Source-combine	DANN [18]	70.31	65.82	68.73	71.75	70.19	69.08	72.26	68.57	66.36
	MCD [19]	72.88	70.35	72.26	74.86	73.41	76.38	75.65	72.13	75.25
Multi-source	MFSAN [32]	75.70	78.21	77.73	78.30	85.21	83.97	70.83	69.00	70.65
	MDAN [29]	72.72	72.16	74.90	79.33	83.11	80.37	67.30	70.98	70.93
	M3SDA [22]	75.32	76.46	77.98	81.62	82.67	82.98	70.26	69.82	70.45
	AMTLDC [30]	74.37	79.33	78.54	80.20	84.33	81.27	70.19	70.20	71.78
	WUCF	76.77	80.58	79.29	82.26	86.00	84.15	71.28	73.70	73.32

* Single-center fully supervised, direct testing in target center without domain adaptation, choosing optimal results.

, it is clear that various multi-source, multicenter domain adaptation methods outperform the unadapted baseline ResNet-50 model, demonstrating the necessity of employing multi-source domain adaptations for the issue of multicenter carotid artery plaque ultrasound classification. Additionally, the results from the multi-source methods are superior to those from the source-combine methods, indicating that utilizing the information from multiple source centers enhances the model generalization and classification accuracy more effectively than relying on a single source center’s data. Compared with DANNs [31] and MCD [19], two single-source domain adaptation methods, the WUCF also showed improved performance. The major difference between the two types is that the data features are mapped to a common space in DANNs and MCD, whereas the WUCF performs targeted feature mapping, and the domain expert discriminator can more effectively and flexibly integrate this independent “knowledge”.

Furthermore, the WUCF outperformed the other comparative methods across three key performance metrics: accuracy, precision, and recall, confirming its significant advantages in the classification of multi-source center carotid ultrasound images. Analyzing the results of the other methods, MFSAN [29], despite considering the data differences between different source centers, merely averaged the final classifier results and failed to effectively integrate the heterogeneous knowledge from each source center. MDAN uses an adversarial learning strategy aimed at improving the generalizability by extracting more generalized feature representations, but its practical performance might be limited due to the design constraints of the adversarial model and the specific characteristics of the target domain data. Additionally, although M3SDA assumes that all the source domains contribute similarly to the target domain, the actual contributions of different source domains to the target domain can vary significantly, and this oversimplified assumption might affect the method’s adaptability and effectiveness. Lastly, the CNN-based feature extraction model proposed by AMTLDC may be overly complex for handling small medical datasets, resulting in performance that does not meet expectations. Therefore, the WUCF is comparatively more suitable for addressing the classification issues of multi-source center carotid ultrasound images.

4.2.2. Effect of View Inconsistency on the WUCF

To explore the WUCF in addressing the negative transfer caused by the inconsistencies in the viewing angles, this section establishes experiments for validation. The experiments involved specific adjustments to the dataset, namely selecting the AC-B as pairing, as in Section 4.2.1, where the performance of AC-B was moderate, and both centers A and B included both longitudinal and transverse views, while center C contained only longitudinal views. Additionally, data from center B were filtered to retain only longitudinal views, forming the new dataset B-L.

As shown in

Table 3. WUCF performance under inconsistent views compared with other models.

Methods	AC-(B_L)			AC-B
	Acc	Pre	Rec	Acc	Pre	Rec
MFSAN [32]	71.13	72.57	72.86	75.70	78.21	77.73
MDAN [29]	73.30	73.98	70.03	72.72	72.16	74.90
M3SDA [22]	72.15	73.43	71.36	75.32	76.46	77.98
AMTLDC [30]	73.37	74.34	70.63	74.37	79.33	78.54
WUCF	75.38	78.09	78.02	76.77	80.58	79.29

, when the dataset was changed from B to B_L while the source center data contained both longitudinal and transverse views, the performance of all comparative models declined. However, the WUCF managed to maintain more stable performance. The decline in performance is likely due to the reduced dataset size, which impacts the model training. The experiments demonstrate that the WUCF excels in multi-source center domain adaptation tasks, particularly when dealing with datasets that include both longitudinal and transverse views.

4.2.3. Model Convergence Verification

To validate the convergence of the WUCF model in multi-source center tasks, this paper plots the changes in accuracy, precision, and recall rates over the iterations for three experiments (Figure 5) as well as line graphs of classification loss and domain discrimination loss (Figure 6). These line graphs provide a clearer and more intuitive analysis of the model’s performance changes with increasing iterations. As observed in Figure 5, the model reached a relatively stable state after 1000 iterations.

Figure 6 illustrates that the WUCF significantly reduces the loss values and enhances the classification accuracy throughout the training process. Specifically, the classification loss graph demonstrates a gradual improvement in the model’s ability to recognize the categories from different centers, while the domain discrimination loss graph shows the model’s progressively improving capability to distinguish between different data sources. These results indicate that the WUCF model exhibits good convergence across various combinations of data sources, thereby confirming its stability and reliability.

4.2.4. Domain Adaptation Layer Determination

Due to the different effects of multicenter domain adaptation layers with varying numbers on the model, systematic experiments were conducted to determine the optimal layer count. The findings of these experiments are displayed in Figure 7. The experimental data demonstrate that the multicenter domain adaptation layers indeed contributed to better domain adaptation. Although the use of one multicenter domain adaptation layer led to the highest accuracy, this approach does not significantly differ from that with the use of two layers. However, the domain adaptation with two layers led to better results in terms of both precision and recall. This is because the two layer structure can provide more modeling capabilities than the single-layer multicenter domain adaptation layer can, which helps to better capture the subtle differences between different center data. With the superposition of the layers, the depth of the model increases, which leads to model overfitting.

4.2.5. Ablation Study

In the WUCF, the MADE is a critical component in which the domain discrimination expert is able to dynamically adjust the weights for multiple source centers. This section uses ablation studies to further validate its effectiveness. In the experimental setup, the domain discrimination expert was removed, and the outputs of two expert classifiers from the source centers were simply averaged to explore their performance in the absence of the domain expert. The results shown in

Table 4. Ablation study on dataset (AC-B) for the WUCF with and without a domain discriminator.

Domain Discriminator	Acc	Pre	Rec
✕	72.23	75.30	72.65
✓	76.77	80.58	79.29

emphasize that the domain discrimination expert indeed enhanced the model’s performance. This indicates that the presence of a domain expert discriminator indeed significantly enhances the ability to effectively integrate the knowledge from multiple source centers, particularly in the scenarios where the source center views are inconsistent, further validating the necessity of the MADE’s design for effective application in academic and practical settings.

4.2.6. Determination of the Loss Function Coefficient

To determine the values of the parameters preceding the loss function, this section also presents a series of experiments. Following the approach of MFSAN [32], the WUCF sets the parameter $\lambda$ in front of $L_{MMD}$ and $L_D$. to the same value. In order to thoroughly investigate the sensitivity of the parameter λ, values were systematically sampled from the set {0.01, 0.02, 0.05, 0.1, 0.2, 0.5, 1}, and experiments were conducted for the AC⟶B pairing. The results of these experiments are summarized in

Table 5. Determination of loss function coefficient.

$\mathit{\lambda }$	Acc	Pre	Rec
0.01	72.62	69.17	60.18
0.02	73.79	75.89	73.42
0.05	74.02	75.98	76.98
0.10	74.30	76.34	77.32
0.20	75.33	78.76	78.88
0.50	76.77	80.58	79.29
1	75.30	79.56	79.20

, from which it can be observed that as the value of λ changes, the accuracy of the model initially increases and then decreases. This trend reflects the crucial role of λ in tuning the model’s performance and highlights the importance of finding an appropriate balance during the parameter adjustment. According to the experimental results, the coefficient was set to 0.5.

5. Conclusions

To address the significant challenges in the early detection and risk reduction of acute cardiovascular events caused by unstable plaque ruptures, a multi-source, multicenter model for carotid plaque risk classification has been developed. This model is crucial given that carotid plaque ultrasound datasets often come from diverse medical centers, facing substantial challenges. The variations in medical practices and equipment across different centers lead to the discrepancies in data, while the inconsistencies in ultrasound views, such as longitudinal versus transverse scans, introduce potential negative transfer. These issues undermine the model’s generalizability and can significantly diminish the accuracy when applying a model developed for a single center to multicenter data.

This paper presents the innovative Weighted Unsupervised Classification Framework for multi-source center ultrasound plaque images (WUCF). Addressing the inherent challenges in multi-source ultrasound data, the MADE leverages the domain-specific fully connected layers, tailored for each source center, to create a phase of domain feature adaptation that efficiently aggregates the feature knowledge from various centers. Diverging from traditional methods, the MADE integrates a domain discrimination expert, enhancing the efficiency of the knowledge integration from multiple expert classifiers and boosting the overall classification performance. The extensive experiments across datasets from three centers, compared against source-only, source-combine, and multi-source methods, demonstrated the WUCF’s superior performance. Furthermore, the tests contrasting longitudinal and transverse view differences validated the WUCF’s robust classification capabilities amidst the view discrepancies. These results not only confirm the proposed framework’s superiority in managing multi-source data but also offer new perspectives and methodologies for the future research in the complex data environments.

The potential applications of the WUCF are extensive. It can ensure consistent data analysis across different research sites in multicenter clinical studies, enhancing the reliability of the study results. In telemedicine, the WUCF improves the reliability of remote diagnoses by accommodating the varied imaging protocols from different locations. Additionally, it assists clinicians in accurately identifying and classifying carotid plaques, streamlining the diagnostic workflows, and contributing to personalized treatment plans by providing accurate classifications.

Despite the promising results, several limitations need to be addressed. The model faces scalability issues as the number of centers increases, requiring more computational resources and posing challenges for its real-time implementation. Additionally, the quality of unlabeled data is crucial for the WUCF’s performance; poor-quality data with noise or artifacts can significantly degrade the results. The model also has interpretability challenges, making it difficult to understand the decision-making process, which may hinder the clinical acceptance and trust. Furthermore, the WUCF requires re-training for domain adaptation when encountering the data from new centers, limiting its direct applicability to new datasets.

Future research could extend the framework to include more diverse data sources and datasets to further test its adaptability and generalization capabilities. The advancements in domain adaptation techniques, especially using deep learning methods such as generative models or reinforcement learning, could enhance the framework’s ability to handle increasingly varied data sources. Exploring the integration of real-time data analysis capabilities could also provide immediate clinical decision support, transforming the framework into a more dynamic tool in medical diagnostics.