A Novel Cluster Matching-Based Improved Kernel Fisher Criterion for Image Classification in Unsupervised Domain Adaptation

Abstract

Unsupervised domain adaptation (UDA) is a popular approach to reducing distributional discrepancies between labeled source and the unlabeled target domain (TD) in machine learning. However, current UDA approaches often align feature distributions between two domains explicitly without considering the target distribution and intra-domain category information, potentially leading to reduced classifier efficiency when the distribution between training and test sets differs. To address this limitation, we propose a novel approach called Cluster Matching-based Improved Kernel Fisher criterion (CM-IKFC) for object classification in image analysis using machine learning techniques. CM-IKFC generates accurate pseudo-labels for each target sample by considering both domain distributions. Our approach employs K-means clustering to cluster samples in the latent subspace in both domains and then conducts cluster matching in the TD. During the model component training stage, the Improved Kernel Fisher Criterion (IKFC) is presented to extend cluster matching and preserve the semantic structure and class transitions. To further enhance the performance of the Kernel Fisher criterion, we use a normalized parameter, due to the difficulty in solving the characteristic equation that draws inspiration from symmetry theory. The proposed CM-IKFC method minimizes intra-class variability while boosting inter-class variants in all domains. We evaluated our approach on benchmark datasets for UDA tasks and our experimental findings show that CM-IKFC is superior to current state-of-the-art methods.

1. Introduction

In machine learning (ML) applications, such as object detection and image classification, a vast amount of labeled data is necessary to build a model for there to be a good achievement on test data that has the same distribution as the model training data. Several practical applications, such as the diagnosis of medical images [1], require an enormous quantity of labeled data to be collected, and such efforts are time- and money-consuming.

To handle this issue, the ML paradigm known as “Domain Adaptation” (DA) [2,3] transfers information from a source domain (SD) that has already been labeled to a target domain (TD) that has only recently been labeled by assuming that both domains have the same classification task (see Figure 1). We focus on the UDA scenario in this study, where all target samples are taken to be unlabeled [4]. UDA is used for predicting unlabeled data in a particular TD in a similar label space in which various divergences exist among two domains [5,6]. Typically, because of the distribution shifts, various models from the SD to the TD are unsuccessful. These issues are figured out by employing numerous methods that seek to match the specified source and target distribution and are used to perform several tasks in computer vision (CV). UDA is becoming more popular due to its applications in numerous CV fields, and different methods are being used to solve the issue [7,8,9,10]. During the time of the training process, there are various distributions for two domains, and UDA is concerned with situations in which both an unlabeled TD and a labeled SD are accessible [11]. Several methods, including [12,13], suggest matching the two different distributions in a latent subspace to categorize the unlabeled target data. Reference  [12] suggested that, in order to minimize the distribution shift, samples from both domains should be projected onto a new subspace, while also including marginal and conditional distributions. Adaptively weighting the marginal and conditional distributions was suggested by J. Wang et al. [14], based on [12], to further reduce the distribution shift.

The current approaches are classified into four categories [13]: (a) instance-based adaptation, which decreases domain discrepancy by adjusting sample weights in either the SD or both domains [15], (b) feature representation-based adaptation, which creates feature representations to reduce either learning task inaccuracy, domain shift, or both [13], (c) classifier-based adaptation [16], which uses training data from both domains and develops a new model that lowers the generalization error within the TD and (d) hybrid knowledge-based adaptation, which offers several knowledge types, such as joint instance and feature representation-based adaption [12], joint instance and classifier-based adaptation, or joint feature representation and classifier-based adaptation. Regardless of the classifier’s apparent high performance, the majority of the techniques covered above, including the support vector machine and the nearest neighbor classifier, instruct a straightforward classifier on the projected source data, provide each target sample with a pseudo-label, after that, utilise the capabilities of the classifier. The classification procedure, independent of the target data, may lead to a considerable distribution shift.

This study introduces a novel technique, known as the cluster matching-based improved kernel Fisher criterion (CM-IKFC). To reduce the distribution shift, CM-IKFC suggests aligning the conditional and marginal distributions of the SD and TD using cluster matching and the Fisher criteria. Particularly, K-means clustering is utilised to cluster the samples in the latent subspace in both domains [17]. After that, we provide pseudo-labels to the target cluster by comparing the TD cluster centroid with the centroid of the SD class. This way, when giving pseudo-labels, the distributions of both domains are considered. Additionally, we used the Improved Kernel Fisher Criterion (IKFC) to reduce intra-class discrepancy, while increasing inter-class discrepancy, in both domains. This promotes cluster matching and the minimization of distribution shifts. In the training phase, the IKFC is introduced to extend the technique, so that the adapted image class transitions and the semantic structures are preserved. The highlights of our paper’s contributions are listed below:

• The proposed method, CM-IKFC, clusters the data from both domains using the K-Means technique in a learnt subspace.
• The cluster centroid of both the SD and TD should be matched with one another in order to more accurately assign the target sample pseudo-labels. In this manner, while assigning pseudo-labels, the SD and TD distributions are considered.
• In the training phase of cluster matching, the IKFC increases inter-class discrepancy in both domains while decreasing intra-class discrepancy. The KFC is improved by utilizing normalized parameters.
• Based on experimental results on three benchmark datasets, CM-IKFC shows superior performance over state-of-the-art UDA approaches.

The remaining sections of the paper are arranged as follows: In Section 2, a literature review is described; In Section 3, the proposed method is demonstrated; In Section 4, the experimental results, which demonstrate that the proposed technique is both successful and efficient, are discussed; In Section 5, the paper’s conclusion is presented.

2. Related Work

Domain adaptation (DA) employs labelled source data with a distribution different from the TD in an effort to improve target learning. Due to recent developments in CV, several techniques for UDA have been developed that employ deep learning (DL). Li et al. [18] provided a brand-new UDA technique that attempts to produce target image–label pairings on the spot, and, moreover, creates semantic loss conditional on randomly selected labels. In addition, it uses an adversarial training approach in GAN for similar target styles. The labels on the output images must, specifically, match those on the input images. Due to the reliable target-style preparation data used during training, the model performs well in the TD. Our model performs better when dealing with problems having high domain disagreement because it prevents lead distribution alignment between two domains.

Das et al. [19] suggested a form of UDA that takes into account the presence of the TD unlabeled statistics. The method focuses on connections between samples from SD and TD. To find the correlations, the source and target samples are both considered as graphs and paired using a convex criterion. The measures are class-based regularization and first- and second-order similarity among the graphs. Additionally, they created a convex optimization process that was computationally effective, making the suggested strategy generally applicable. Furthermore, it is indeed necessary to test conventional DA datasets for structured data.

Guan et al. [20] proposed “Cross-Domain Minimization with Deep Autoencoder” (CDMDA) for UDA. CDMDA implements a strategy for multitask learning in which, in a single architecture, CORAL-aligned sharable feature representations are utilized to simultaneously train the SD’s label prediction and input reconstruction on the TD. Additionally, the cluster assumption can be supported by attaching a label discriminator to the adversarial training procedure, which causes the projected target label distribution and the category distribution to appear to have different distributions. Several domain adaptations on visual, as well as on non-visual, datasets demonstrated that the developed method consistently outperformed competing UDA methods. These proposed methods have to be applied to several other domain-adaptable tasks, such as semantic segmentation and classification.

Tian et al. [21] suggested a brand-new DA strategy that extensively investigates the TD data distribution structure and, particularly, treats samples that are part of a similar cluster as a whole, rather than as individuals in the TD. The strategy gives the target cluster pseudo-labels through class matching of the centroid. A self-learning technique for local manifolds was also included in the presentation to fully leverage the target data’s different structural information and to adaptively capture the local connectedness the target samples naturally possessed. The objective function was to be solved with a theoretical convergence guarantee by a powerful iterative optimization technique. A more comprehensive design of semi-supervised algorithms needs further investigation.

Some recent UDA techniques employ the pseudo-labeling approach to take advantage of the semantic information of the target samples. Pseudo labels were used by Xie et al. [22] to estimate class centroids for the TD and to compare them to those in the SD. A self-training system, that alternatively executes model training and pseudo label refinement, was proposed by Zou et al. [23]. Recent studies [24,25] have demonstrated the superiority of clustering-based pseudo-labeling approaches and have demonstrated how these approaches may be successfully applied to DA.

The Kernel Fisher discrimination analysis (KFDA) is a nonlinear technique for two-class and multi-class problems with origins in FDA [26]. KFDA works by transforming the low-dimensional sample space into a high-dimensional feature space, where the FDA is then carried out. To make the computation simpler, the kernel matrix was replaced by its submatrix by  [27]. Liu et al. suggested a new KFDA criterion to maximize the uniformity of class–pair separabilities, which was assessed by the entropy of the normalized class–pair separabilities [28]. One of the theoretical study fields that has received a lot of interest recently is optimal kernel selection. Based on FDA’s quadratic programming formulation, Fung et al. devised an iterative technique [29]. By using second-order cone programming, Khemchandani et al. looked at the issue of locating the data-dependent “optimal” kernel function [30]. In order to solve identification or classification issues, KFDA is increasingly being used in conjunction with strong nonlinear feature extraction abilities, and, as a result, is used extensively and successfully in numerous fields.

In this context, Deng et al. [31] proposed a deep clustering method using a Fisher-like criteria-based loss to align the feature distributions of the SD and TD. However, they only used target clustering as an incremental strategy to increase explicit feature alignment, while our proposed method uses cluster matching to take into account both domains’ distributions and intra-domain category information. Chang et al. [32] proposed a discriminative feature learning method to estimate the inter-class separability in UDA. They calculated inter-class separability using the distances between pairs of class centers, whereas our proposed method utilizes preserved semantic structure and class transitions. Meng et al. [33] proposed a method that utilizes pseudo-labels and iterative clustering to incorporate label structure information in UDA. Their approach aims to improve the accuracy of the pseudo-labels generated for the TD, while our proposed method focuses on generating an accurate pseudo-label in a latent discriminative subspace for each target sample. The majority of modern approaches use a method that explicitly aligns feature distributions between the two domains, while ignoring the target distribution and intra-domain category information. The findings of published works first create a classifier on the labeled domain to generate pseudo-labels for unlabeled samples and then compute the unlabeled class distribution using the pseudo-labels, before aligning the distributions. Unfortunately, if the classifier ignores the unlabeled distribution, it may fail to learn the TD. In the prediction of the unlabeled domain distribution, mislabeled data results in large errors. In order to solve the issue, this research suggests a novel CM-IKFC to produce a precise pseudo-label in a latent discriminative subspace for each target sample, while taking into account both domain distributions.

3. Proposed Method

We start with the basic definitions of the UDA problem before providing an overview and delving into the details of the proposed method.

3.1. Problem Statement

We start with the formal concept of DA [6]. The terminology definitions and the notations used are listed in

Table 1. Notations used for this paper.

Variable	Definition
${\mathcal{D}}_s,{\mathcal{D}}_t$	Source/target domain
$x_{s,i},y_{t,i}$	Source/target sample
$n_s,n_t$	#source/target sample
${\mathbf{L}}_t$	target label matrix
$\mathbf{H}$	projection matrix
$\mathbf{F}$	cluster centroids of the target data
${\mathbf{E}}_{\mathbf{s}}$	class centroids of the source data
$M_w$	within-class scatter matrix
$M_b$	scatter matrix between classes
F	feature space
d	subspace dimension
i	max iteration number
$\mu$	identity matrix
$\delta$	classification parameter
$\alpha$	optimal vector

for clarification.

We focus on UDA for image classification when the SD has enough labeled data ${\mathcal{D}}_s=\{{\mathcal{X}}_s,Y_s\}={\{{\mathbf{x}}_{\mathbf{i}},{\mathbf{y}}_{\mathbf{i}}\}}_{\mathbf{i}=\mathbf{1}}^{{\mathbf{n}}_{\mathbf{s}}}$, and unlabeled data in the TD ${\mathcal{D}}_t=\left\{{\mathcal{X}}_t\right\}={\left\{{\mathbf{x}}_{\mathbf{i}}\right\}}_{\mathbf{i}=\mathbf{1}}^{{\mathbf{n}}_{\mathbf{t}}}$, where ${\mathbf{x}}_{\mathbf{i}}\in {\mathbb{R}}^{dx1}$ denotes the feature vector, ${\mathbf{y}}_{\mathbf{i}}\in \{1,...,K\}$ refers to the sample label, having a total of K classes. It is assumed that the SD and TD features and label spaces are the same. The goal of DA is to locate a mapping h that will maximise the consistency between the mapped spaces of the SD data $h\left({\mathcal{X}}_s\right)$ and the TD data $h\left({\mathcal{X}}_t\right)$. As a result, the model is able to build a more efficient classifier $\mathit{f}(.)$ on the SD using labeled samples to anticipate the labels of the TD, i.e., ${\mathbf{x}}_{\mathbf{t}}\to {\mathbf{y}}_{\mathbf{t}},{\mathbf{y}}_{\mathbf{t}}\in y_t$.

3.2. CM-IKFC

The proposed CM-IKFC focuses on creating a discriminative subspace for reducing distributional changes between two domains. As seen in Figure 2, there are two main components of our model, named the K-means algorithm and IKFC. In CM-IKFC, first, we use the k-means algorithm [17] to cluster the data from the two domains in a learning subspace. After that, we apply pseudo-labels to the target samples by comparing the clustered centroid of the TD with the clustered centroid of the SD. In this method, the distributions of the SD and TD are both considered when deciding on pseudo-labels. Finally, the KFC is improved by utilizing normalized parameters and weighed schemes to rebuild the scatter matrices between different classes and within the same class. As a result, it has the capability to alter the function known as the kernel scatter difference discriminant. CM-IKFC applies the IKFC constraint to increase inter-class discrepancy in both domains, while decreasing intra-class discrepancy. This encourages distribution shift reduction and cluster matching.

3.3. Domain Clustering

The K-means technique is utilised to cluster both domain samples and attach associated labels. The number of clusters is equal to the number of core classes K. The distance means for the K classes in the data set are determined using the distance as the metric and the initial centroid to describe each class. The Euclidean distance is the similarity index for a specific data set X with n multi-dimensional data points and the class K to be partitioned. The clustering objectives reduce the sum of the squares of the different data types [34].

$$\begin{array}{c}\hfill e=\sum _{z=1}^K\sum _{x=1}^i{w}_{ik}{∥(p_x-m_z)∥}^2\end{array}$$

(1)

In this context, if a data point $p_x$ belongs to cluster K, then the value of $w_{ik}$ is set to 1; otherwise, it is set to 0. The variable $m_z$ represents the center of the cluster, i represents the total number of data points, K denotes the cluster number, and $|(p_x-m_z){|}^2$ represents the squared distance between a data point $p_x$ and the cluster center $m_z$.

It is a two-step minimization problem. We start by minimizing e in relation to $w_{ik}$ and treating $m_z$ as fixed. Then, we minimize e w.r.t. $m_z$ and regard $w_{ik}$ as fixed. Technically, we distinguish e with respect to $w_{ik}$ first and then update cluster assignments. After that, we differentiate e concerning $m_z$ and recompute the centroids based on the cluster assignments from the initial phase as follows:

$$\begin{array}{cc}\hfill \frac{\partial e}{\partial w_{ik}}& =\sum _{z=1}^K\sum _{x=1}^i{\parallel p_x-m_z\parallel }^2\hfill \\ \hfill & \Rightarrow w_{ik}=\left\{\begin{array}{cc}1\hfill & \mathrm{if}z=arg\underset{e}{min}{\parallel p_x-m_z\parallel }^2\hfill \\ 0\hfill & \mathrm{otherwise}\hfill \end{array}\right.\hfill \end{array}$$

(2)

It means that the data point $p_x$ should be assigned to the cluster that is the closest to it, based on its sum of squared distance from the centroid of the cluster, and the assignment should be made using the following formula:

$$\begin{array}{cc}\hfill \frac{\partial e}{\partial m_z}=2\sum _{x=1}^i{w}_{ik}\left(p_x-m_z\right)& =0\hfill \\ \hfill & \Rightarrow m_z=\frac{{\sum }_{x=1}^i{w}_{ik}{p}_x}{{\sum }_{x=1}^i{w}_{ik}}\hfill \end{array}$$

(3)

This requires the centroids of all of the clusters to be recalculated so that they reflect the new assignments.

Initially, the cluster center is chosen at random, drawn from the sample set. Every sample point is grouped to form a cluster that represents the center point that is nearest to it. Then, all sample points are combined, and the core of each cluster is its own center. The preceding procedures are performed until the cluster’s center point is constant or until the specified number of iterations have been completed. The algorithm output fluctuates depending on the center point selected, causing instabilities. The K value chosen sets the center point, which serves as the algorithm’s target. The results of clustering directly affect aspects such as local or global optimality.

3.4. Cluster Matching

To calculate the UDA’s conditional distribution, each target sample is labeled along with having a pseudo-label. In contrast to traditional methods, to allocate the pseudo-labels, a classifier that has been trained on the SD is employed. This task takes all these domain distributions into account. In such scenarios, it becomes possible to obtain pertinent information regarding the sample distribution structure of the target data. To accomplish this objective, there are several pre-existing clustering algorithms that can be suitable options. Without sacrificing the generalizability of our approach, we chose to employ the well-established K-means algorithm in this study for the purpose of obtaining cluster prototypes. Therefore, the ensuing formula can be expressed as follows  [21]:

$$\Theta \left(\mathbf{H},\mathbf{F},{\mathbf{L}}_t\right)={∥{\mathbf{H}}^{\mathrm{T}}{\mathbf{X}}_t-\mathbf{F}{\mathbf{L}}_t^{\mathrm{T}}∥}_F^2$$

(4)

where $\mathbf{H}\in {\mathbb{R}}^{m\times d}$ represent the projection matrix. $\mathbf{F}\in {\mathbb{R}}^{d\times C}$ denotes the cluster centroids of the target data. ${\mathbf{L}}_t\in {\mathbb{R}}^{n_t\times C}$ is the cluster indicator matrix for the target data. This matrix is defined as ${\left({\mathbf{L}}_t\right)}_{ij}=1$ if the cluster label of ${\mathbf{x}}_{ti}$ is j, and ${\left({\mathbf{L}}_t\right)}_{ij}=0$ otherwise.

Next, we conduct the computation of class centroids for the source data and class centroid matching of the SD and TD. When we have the cluster prototypes of the target data, we may reframe the distribution discrepancy reduction issue as the class centroid matching problem. This is possible once we receive the cluster prototypes. It is important to keep in mind that the class centroids of the source data may be precisely determined by computing the mean value of the sample features that belong to the same class. In this study, we used the straightforward and time-saving technique of searching for the closest neighbor to find a solution to the issue of matching class centroids. To be more specific, we looked for the class centroid that was geographically closest to each target cluster centroid, and we tried to find a way to reduce the total distance between each pair of class centroids. In conclusion, the formula for class centroid matching between two domains reads as follows [21]:

$$\mathrm{\Omega }(\mathbf{H},\mathbf{F})={∥{\mathbf{H}}^{\mathrm{T}}{\mathbf{X}}_s{\mathbf{E}}_s-\mathbf{F}∥}_F^2$$

(5)

where the matrix ${\mathbf{E}}_s\in {\mathbb{R}}^{n_s\times C}$ is a fixed matrix that is utilized to calculate the class centroids of the source data in the transformed space with each element ${\mathbf{E}}_{ij}=1/n_s^j$ if $y_{si}=j$, and ${\mathbf{E}}_{ij}=0$ otherwise.

3.5. KFDA

KFDA is an effective nonlinear FDA technique, in which the kernel function is applied to handle the issues of nonlinear optimization. The KFDA is widely used in several fields, due to its effectiveness. The numerical principle is evaluated as follows.

Suppose $S=\left\{x_1,x_2,\dots .,x_N\right\}$ is the dataset, which consists of K classes in a d-dimensional real space $R^d$, where $N_i$ samples belong to the j th class, $\left(N=N_1+N_2+\cdots +N_k\right)$.

FDA is employed for the purpose of finding the optimal projection vectors w that minimize the within-class scatter among different samples. The vector $w\in R^d$ that optimizes the Fisher discriminant function provides FDA as

$$maxJ\left(w\right)=\frac{w^T{M}_{b}w}{w^T{M}_{w}w}$$

(6)

where $M_w$ represents the within-class scatter matrix and $M_b$ represents the scatter matrix between classes. FDA primarily relies on linear techniques, posing a challenge when attempting to distinguish samples that exhibit nonlinear separability.

The utilization of a kernel trick in the KFDA leads to a notable enhancement in the classification capability of the FDA when dealing with samples that are nonlinearly separable. To accommodate nonlinear scenarios, the function $\phi \left(x\right)$ transforms the samples from a lower-dimensional space into a higher-dimensional feature space. Note that $\varphi \left({\mu }_i^j\right)\left(i=1,2,\dots ,N_j,j=1,2,\dots ,K\right)$ is the i th projection value in the class ${\omega }_j$. $m^{\varphi }$ denotes the mean vector of the entire population, and $m_j^{\varphi }$ represents the mean vector of class ${\omega }_j$. In the feature space F, we define the total scatter matrix $M_t$, the within-class scatter matrix $M_w$, and the between-class scatter matrix $M_b$ as [35]:

$$M_t^{\varphi }=\frac{1}{N}\sum _{i=1}^N\left(\varphi \left({\mu }_i\right)-m^{\varphi }\right){\left(\varphi \left({\mu }_i\right)-m^{\varphi }\right)}^T$$

(7)

$$M_w^{\varphi }=\frac{1}{N}\sum _{j=1}^K\sum _{i=1}^{N_j}\left(\varphi \left({\mu }_i^j\right)-m_j^{\varphi }\right){\left(\varphi \left({\mu }_i^j\right)-m_j^{\varphi }\right)}^T$$

(8)

$$M_b^{\varphi }=\sum _{j=1}^K\frac{N_j}{N}\left(m_j^{\varphi }-m^{\varphi }\right){\left(m_j^{\varphi }-m^{\varphi }\right)}^T$$

(9)

The development of KFDA may be traced back to the following improvement in the kernel Fisher criteria function:

$$maxJ\left(v\right)=\frac{v^T{M}_b^{\varphi }v}{v^T{M}_w^{\varphi }v}$$

(10)

where the various optimal projection vectors are represented by v. Directly calculating the ideal discriminant vector v is not feasible, due to the large dimension of feature space F and the infinite dimension. Applying the kernel technique is one approach to resolve this issue, as is shown below [36]:

$$K\left({\mu }_i,x_j\right)=\left(\varphi \left({\mu }_i\right),\varphi \left(x_j\right)\right)$$

(11)

Any solution v must exist inside the feature space F, as stated by the notion of reproducing a kernel [26], which spans $\varphi \left(x_1\right),\varphi \left(x\right),\dots ,\varphi \left(x_N\right)$ as follows:

$$v=\sum _{i=1}^N{\alpha }_i\varphi \left({\mu }_i\right)$$

(12)

By projecting any test samples into w in F, the following equation is obtained:

$$\begin{array}{cc}\hfill v^T\varphi \left(x\right)=& \sum _{i=1}^N{\alpha }_i\left(\varphi \left(x\right),\varphi \left({\mu }_i\right)\right)\hfill \\ \hfill =& {\alpha }^T\left(\left(\varphi \left(x\right)\varphi \left(x_1\right)\right),\left(\varphi \left(x\right),\varphi \left(x_2\right)\right),\dots \right.\hfill \\ & \left.\left(\varphi \left(x\right),\varphi \left(x_N\right)\right)\right)\hfill \\ \hfill =& {\alpha }^T\left(K\left(x,x_1\right),K\left(x,x_2\right),\dots ,K\left(x,x_N\right)\right).\hfill \end{array}$$

(13)

The kernel between-class scatter matrix $H_y$ and within-class scatter matrix $H_u$ in F may be defined as follows [37]:

$$H_y=\sum _{i=1}^K\frac{N_i}{N}\left({\mu }_i-{\mu }_0\right){\left({\mu }_i-{\mu }_0\right)}^T,$$

(14)

$$H_u=\frac{1}{N}\sum _{i=1}^K\sum _{j=1}^{N_i}\left({\xi }_{x_j}-{\mu }_i\right){\left({\xi }_{x_j}-{\mu }_i\right)}^T,$$

(15)

$${\mu }_0=\left(\frac{1}{N}\sum _{i=1}^{N}K\left(x_1,{\mu }_i\right),\frac{1}{N}\sum _{i=1}^{N}K\left(x_2,{\mu }_i\right),\dots ,\right.{\left.\frac{1}{N}\sum _{i=1}^{N}K\left(x_N,{\mu }_i\right)\right)}^T,$$

(16)

$${\mu }_i=\left(\frac{1}{N_i}\sum _{j=1}^{N_i}K\left(x_1,x_j^i\right)\right),\dots ,\frac{1}{N_i}\sum _{j=1}^{N_i}K\left(x_N,x_j^i\right),$$

(17)

$${\xi }_{x_j}={\left(k\left(x_1,x_j\right),k\left(x_2,x_j\right),\dots ,k\left(x_n,x_j\right)\right)}^T$$

(18)

We know that $H_u$, $H_u$, $H_t$ are in a semi-definite symmetrical matrix. The fisher criterion function in the feature space F is defined as [38]:

$$maxJ\left(v\right)=\frac{v^T{M}_b^{\varphi }v}{v^T{M}_w^{\varphi }v}=\frac{{\alpha }^T{H}_y\alpha }{{\alpha }^T{H}_u\alpha }=J\left(\alpha \right)$$

(19)

In accordance with the characteristics of the generalized Rayleigh quotient, the optimum solution vector w may be found by increasing the value of the criteria function in (19) until it is equal to the solution of the generalized characteristic equation in the following manner:

$$H_y\alpha =\lambda H_u\alpha$$

(20)

3.6. IKFC

The scatter difference discriminant function [39] was constructed in this study in response to the ill-posed issue of the discriminant criteria function (19):

$$maxJ\left(v\right)=v^{\mathrm{T}}{H}_y^{\varphi }v-v^{\mathrm{T}}{H}_u^{\varphi }v$$

(21)

The within-class scatter matrix singular issue is substantially resolved by this approach.

The basic concept of the weighted kernel maximum scatter difference discriminant analysis is that a class is designated as a margin class if it is significantly distant from the center. In this instance, the margin class is distinguished from other classes using the best discriminant vectors produced by maximizing the kernel scatter difference criteria function, since the class variance is greatest in this direction. The class with a greater distance from the center plays a significant part in the process of maximizing the kernel scatter difference criteria function. These projection vectors not only cannot separate classes, other than the margin class, but also cause neighboring classes in the feature space to overlap. The between-class scatter matrix is redefined in response to this issue in the manner described below [38]:

$$H_y^{\varphi }=\sum _{i=1}^K\frac{N_i}{N}\varpi \left(d_i\right)\left(\varphi \left({\mu }_i\right)-\varphi \left({\mu }_0\right)\right){\left(\varphi \left({\mu }_i\right)-\varphi \left({\mu }_0\right)\right)}^{\mathrm{T}}$$

(22)

where $d_i=\sqrt{{\left(\varphi \left({\mu }_i\right)-\varphi \left({\mu }_0\right)\right)}^{\mathrm{T}}\left(\varphi \left({\mu }_i\right)-\varphi \left({\mu }_0\right)\right)}$; $d_i$ is the distance between ith class and center. $\varpi (·)$ represents the weighted function. In order to reduce the impact of margin classes, we employed a strategy where the larger value of $∥{\mu }_i-{\mu }_0∥$ was assigned a smaller weight. To achieve this, we defined a weighted function $\varpi \left(d_i\right)=d_i^{-3}$. Additionally, we assigned a weight $\alpha =1/\left(1+{\sum }_{i=1}^K{\sum }_{j=1}^{N_i}{d}_j^i\right)$ to the within-class scatter matrix. Here, $d_j^i=\sqrt{{\left(\varphi \left({\xi }_{x_j}\right)-{\left({\mu }_0\right)}^{\varphi }\right)}^{\mathrm{T}}\left(\varphi \left({\xi }_{x_j}\right)-{\left({\mu }_0\right)}^{\varphi }\right)}$. By employing these methods, we obtained the following result.

$$H_u^{\varphi }=\frac{a}{N}\sum _{i=1}^K\sum _{j=1}^{N_i}\left({\xi }_{x_j}^{\varphi }-{\left({\mu }_i\right)}^{\varphi }\right){\left({\xi }_{x_j}^{\varphi }-{\left({\mu }_i\right)}^{\varphi }\right)}^{\mathrm{T}}$$

(23)

The weight’s objective was to bring all of the training samples belonging to a certain class closer together to the class’s center, and so further reducing the amount of large overlapping occurring across classes. The following is a definition of the weighted kernel scatter difference discriminant function in the feature space:

$$maxJ\left(v^{*}\right)=v^{*T}\left(H_y^{\varphi }-H_u^{\varphi }\right)v^{*}$$

(24)

$$v^{*}=\sum _{k=1}^N{\alpha }_k^{*}\varphi \left(x_k\right)=X_{\varphi }{\alpha }^{*}$$

(25)

The column vector ${\alpha }^{*}$ is of dimension N. By applying the Regeneration Kernel Theory and Equations (14)–(22), we obtain the following result:

$$J\left({\alpha }^{*}\right)=v^{*T}\left(H_y^{\varphi }-H_u^{\varphi }\right)v^{*}={\alpha }^{*T}\left(K_B-K_W\right){\alpha }^{*}$$

(26)

$$K_B=\sum _{i=1}^K\frac{N_i}{N}\varpi \left(d_i^{*}\right)\left(u_i-u_0\right){\left(u_i-u_0\right)}^{\mathrm{T}}$$

(27)

$$K_W=\frac{a}{N}\sum _{i=1}^K\sum _{j=1}^{N_i}\left(\phi \left({\xi }_{x_j}\right)-u_i\right){\left(\phi \left({\xi }_{x_j}\right)-u_i\right)}^{\mathrm{T}}$$

(28)

$$K_T=K_B+K_W$$

(29)

In the given context, $d_i^{*}=\sqrt{{\left(u_i-u_0\right)}^T\left(u_i-u_0\right)}.$$K_T,K_W$ and $K_B$ are referred to as the weighted kernel total scatter matrix, weighted kernel within-class scatter matrix, and weighted kernel between-class scatter matrix, respectively. It is important to note that these matrices are semi-definite and symmetrical, with dimensions of $N\times N$. Moving on, we can infer that Equation (22) can be equivalently expressed in the following forms:

$$\left\{\begin{array}{c}maxJ\left({\alpha }^{*}\right)={\alpha }^{*T}\left(K_B-K_W\right){\alpha }^{*}\hfill \\ {\alpha }^{*T}{\alpha }^{*}=1\hfill \end{array}\right.$$

(30)

The expression $\frac{{\alpha }^{*T}\left(K_B-K_W\right){\alpha }^{*}}{{\alpha }^{*T}{\alpha }^{*}}$ can be rewritten as $\frac{{\alpha }^{*T}\left(K_B-K_W\right){\alpha }^{*}}{{\alpha }^{*T}I{\alpha }^{*}}$, where $\mathbf{I}$ represents a unit matrix. Using the Expansion Rayleigh Quotient, we derive the following information.

$$\left(K_B-K_W\right){\alpha }^{*}=\lambda {\alpha }^{*}$$

(31)

The optimal solutions ${\alpha }^{*}$ correspond to the solutions of Equation (26). By utilizing ${\alpha }^{*}$ and Equation (26), we derive the nonlinear discriminant vectors in the feature space $\mathrm{F}$. These vectors are denoted as $v^{*}$, which can be represented as $v^{*}=\left(v_1^{*},v_2^{*},\cdots ,v_k^{*}\right)$, where $v_i^{*}={\sum }_{k=1}^{K-1}{\alpha }_k^{*i}\varphi \left(x_k\right),i=$$1,2,\cdots ,N$.

3.7. Selecting a Normalized Parameter

A normalized parameter was used because the characteristic equation is difficult to solve. If $H_y$ represents a non-singular matrix, then $\alpha$ is an optimal vector which is obtained by maximizing Equation (19), which is comparable to the feature vector corresponding to the top m greatest eigenvalues, as described in the article by [40]. Equation (20) may also be written as

$${\left(H_y\right)}^{-1}{H}_u\alpha =\lambda \alpha$$

(32)

where $H_y$ represents the singular matrix. It is not always possible to just use Equation (32). Furthermore, employing a regularized method improves the stability of numerical methods.

$${\left(H_y\right)}_{\delta }=H_y+\delta \mu ,$$

(33)

where, $\delta$ represents a small, positive number, and $\mu$ indicates the identity matrix. Then, Equation (20) can be written as

$${\left(H_y+\delta \mu \right)}^{-1}{H}_u\alpha =\lambda \alpha$$

(34)

While KFDA is employed to solve applied problems, the parameter is determined based on experience or experimental results. In this, a normalized parameter was used and the value $H_y+\delta \mu$ regarded as a function of $\delta$:

$$f\left(\delta \right)=\mid H_y+\delta \mu \mid$$

(35)

where, function f is stable, and the value of the function tends to zero in Equation (36), the $\delta$ is the best classification parameter,

$$\underset{\delta \to 0}{\lim }f\left(\delta \right)⟶0$$

(36)

In Algorithm 1, the whole optimization of CM-IKFC is presented.

Algorithm 1 CM-IKFC

1: Input: source data $\{{\mathcal{X}}_s,Y_s\}$, target data $\left\{{\mathcal{X}}_t\right\}$, initial target label matrix ${\mathbf{L}}_t$, subspace dimensionality d = 100, maximum iteration number i = 10, parameters $\gamma$ = 5, $\alpha$ and $\beta$; 2: Output: Target label matrix ${\mathbf{L}}_t$ 3: while converge condition not satisfied do 4:     Calculate the source class center with a focus on the source features 5:     Configure the source class center to create the target cluster center 6:     Discover the target samples’ pseudo-labels using the K-means technique (Section 3.4) 7:     Update $\mathbf{H}$ by Equation (4) 8:     Update ${\alpha }^{*}$ with Equation (26) 9:     Update $f\left(\delta \right)$ with Equation (35) 10: end while 11: Output: target label matrix ${\mathbf{L}}_t$

4. Experiments and Analysis

We evaluated the CM-IKFC method using benchmarks, including Office-31, ImageCLEF, and Office-Home, for object classification. We compared CM-IKFC with state-of-the-art DA methods. To enable an accurate comparison, original or state-of-the-art papers were used to obtain the results. The research findings showed how well the proposed CM-IKFC handled the DA problem.

4.1. Benchmark Datasets

The research used three well-known benchmark datasets: Office-31, ImageCLEF, and Office-Home. Each dataset is briefly described below, and

Table 2. Description of the benchmark datasets.

Dataset	Domain	#Samples	#Classes
Office-31 [41]	DSLR (D) Amazon (A) Webcam (W)	498 2817 795	31
Office-Home [42]	Artistic images (Ar) Clip Art (Cl) Real-World images (Rw) Product images (Pr)	958 15,710 1299 295	65
ImageCLEF [43]	ImageNet (I) Caltech (C) Pascal (P)	4000 3847 2626	31

provides a quick rundown of each dataset’s specifics.

The Office-31 [41], a frequently used evaluation dataset for visual DA, is made up of 4652 images and 31 classes from the following three different domains: images gathered from the (1) Amazon site (Amazon domain), (2) digital SLR photo (DSLR domain), and (3) web camera (Webcam domain) under various conditions, as shown in Figure 3. The dataset’s distribution across domains is shown in Table 2 with 2817 images belonging to the Amazon domain, 795 images to the Webcam domain, and 498 images to the DSLR domain.

The Office–Home [42] poses significant challenges when utilizing it for the examination of the DA model. The 65 categories contain more than 15,500 images of things used every day in homes and offices. The four distinct domains are artistic images (Ar), clip art (Cl), product images (Pr), and real-world images (Rw), as shown in Figure 4. Backgrounds and appearances in these photos are very different. It is more difficult to transfer data between domains because there are many more categories than in Office-31. We examined each technique on each of the 12 adaptation tasks.

The ImageCLEF [43] dataset has 31 categories and 3 domains, namely, Pascal (P), Caltech (C) and ImageNet (I), as shown in Figure 5. We set up six DA tasks: I$\to$C, I→P, P$\to$I, C→I, P→C, C→P.

4.2. Implementation Details

The model was implemented using the PyTorch framework. In every experiment, the projection matrix dimension was always 100. Twenty iterations were the most that could be carried out. We used K-means algorithms for clustering, so the value of K was set according to the classes in the datasets. Specifically, the values of K on the Office-31, Office-Home and ImageCLEF datasets were set as 31, 65, and 31, respectively. Furthermore, we set the parameter $\delta$= 0.2 analytically. The following section analyzes the impact of the trade-off parameter.

4.3. Baseline Methods

To evaluate the effectiveness of our proposed strategy in an experimental setting, we looked at both conventional and state-of-the-art DA methodologies, such as DAN [44], DANN [45], CyCADA [46], JDDA [47], CDAN [48], HAFN [15], SAFN [15], DMP [49], ADDA [50], JAN [51] and rRevGrad+ CAT [31]. We evaluated the method with other domain adaptation techniques. For the purpose of classifying target samples, ResNet-50 [52] simply utilizes the classification model trained on the SD. Multi-mode structures are used by MADA [53] to achieve proper alignment of different data distributions, based on numerous domain discriminators.

4.4. Results and Analysis

In order to demonstrate the efficacy of CM-IKFC, this section presents the classification accuracy achieved on the benchmark datasets. Additionally, we conducted several experiments to investigate how and why our CM-IKFC model successfully addressed domain adaptability. The bold values in the tables represent the highest accuracy achieved for the specific task, indicating superior performance.

The experimental data confirmed the model’s efficacy in this research. The classification results of our model on six UDA tasks from the Office-31 dataset, are shown in

Table 3. Results (%) comparison of proposed and existing methodologies for Office-31.

Method	A→W	D→W	W→D	A→D	D→A	W→A	Avg
ResNet-50 [52]	68.4	96.7	99.3	68.9	62.5	60.7	76.1
ADDA [50]	86.2	96.2	98.4	77.8	69.5	68.9	82.9
JAN [51]	85.4	97.2	99.8	84.7	68.8	70.0	84.3
HAFN [15]	83.4	98.3	99.7	84.4	69.4	68.5	83.9
JDDA [47]	82.6	95.2	99.7	79.8	57.4	66.7	80.0
CyCADA [46]	82.2	94.6	99.7	78.7	60.5	67.8	80.6
MADA [53]	80.7	97.4	99.6	87.8	70.3	66.4	85.2
rRevGrad+ CAT [31]	90.0	97.6	100.0	76.4	63.7	62.2	80.1
CM-KFC Base	86.0	93.6	96.2	83.1	70.9	72.3	83.68
CM-IKFC (ours)	91.0	97.3	99.7	89.1	72.9	76	87.67

. The source-only model performed satisfactorily for D⇆W tasks because the domain gap was minimal. When compared to other models, our method’s accuracy improved more significantly for the challenging A⇆W and A⇆D adaptation tasks. For instance, our approach considerably outperformed all the current adaptation techniques on A→D, W→A, and D→A, especially when compared to CyCADA [46], which had an equal number of class-based discriminators. Our suggested CM-IKFC model outperformed other approaches in the majority of domain adaptation tasks and also improved average performance by 2.47%. In the six experiments, CM-IKFC produced the four best results and none of the poorest results. This demonstrates how well K-means cluster matching and the IKFC work with UDA.

In this study, we describe how the use of IKFC enhanced the performance of our proposed approach, CM-IKFC, in comparison to the base KFC. To demonstrate this improvement, we conducted an experiment where we compared the performance of our proposed approach, CM-IKFC, with that of the KFC-based method without the IKFC extension. The results presented in Table 3, demonstrate that the use of the IKFC criterion significantly enhanced performance by increasing inter-class variability and decreasing intra-class variation. As a result, we observed a substantial improvement in the accuracy of pseudo-labels for TD samples. These findings comprehensively explain how the IKFC criterion was integrated into the KFC to improve its effectiveness. Specifically, the inclusion of a normalized parameter enabled the IKFC criterion to solve the characteristic equation more accurately, leading to a significant improvement in the classification performance of our proposed approach.

The experiment’s results for the office–home dataset are presented in

Table 4. Results (%) comparison of proposed and existing methodologies for Office–Home.

Methods	Ar→ Cl	Ar→ Pr	Ar→ Rw	Cl→ Ar	Cl→ Pr	Cl→ Rw	Pr→ Ar	Pr→ Cl	Pr→ Rw	Rw→ Ar	Rw→ Cl	Rw→ Pr	Avg
ResNet-50 [52]	34.9	50	58	37.4	41.9	46.2	38.5	31.2	60.4	53.9	41.2	59.9	46.1
JAN [51]	45.9	61.2	68.9	50.4	59.7	61	45.8	43.4	70.3	63.9	52.4	76.8	58.3
DANN [45]	45.6	59.3	70.1	47	58.5	60.9	46.1	43.7	68.5	63.2	51.8	76.8	57.6
ADDA [50]	43.5	57.2	67.3	45	57.2	59.3	44.3	43.2	67.2	62.8	51.2	73.9	56
CDAN+ E [48]	50.7	70.6	76	57.6	70	70	57.4	50.9	77.3	70.9	56.7	81.6	65.8
HAFN [15]	50.2	70.1	76.6	61.1	68	70.7	59.5	48.4	77.3	69.4	53	80.2	65.4
SAFN [15]	52	71.7	76.3	64.2	79.9	71.9	63.7	51.4	77.1	70.9	57.1	81.5	67.3
CM-IKFC (ours)	52.3	72.1	76	65.3	79.8	72.3	64.8	52.8	77.0	71.3	58.0	80.9	68.5

. Compared to other methods, CM-IKFC outperformed some traditional methods and performed better on 8 of the 12 cross-domain tasks. The average classification accuracy obtained by CM-IKFC was 68.50%, and, compared with the baseline model, it displayed an improvement in accuracy of 1.3% on average, proving that the process of incremental hardening of prediction labels improves the discriminative information therein. Overall, the majority of tasks achieved excellent results.

On the ImageCLEF dataset,

Table 5. Results (%) comparison of proposed and existing methodologies for ImageCLEF.

Method	I → P	P → I	I → C	C → I	C → P	P → C	Avg
ResNet-50 [52]	74.8	83.9	78	65.5	91.2	60.7	80.7
DAN [44]	74.5	82.2	92.8	86.3	69.2	89.9	82.5
DANN [45]	75	86	96.2	87	74.3	91.5	85
JAN [51]	76.8	88	94.7	89.5	74.2	91.7	85.8
HAFN [15]	76.9	89	94.4	89.6	74.9	92.9	86.3
CDAN [48]	76.7	90.6	97	90.5	74.5	93.5	87.1
CDAN+E [48]	77.7	90.7	97.7	91.3	74.2	94.3	87.7
SAFN [15]	78	91.7	96.2	91.1	77	94.7	88.1
DMP* [49]	80.7	92.5	97.2	90.5	77.7	96.2	89.1
CM-IKFC (ours)	81.1	93.3	96.6	92.1	79.2	96.5	89.8

displays the experimental results of this method and the related comparison methods. In domain adaptation tasks, such as I→P, P→I, C→I, and P→C, CM-IKFC achieved the four best accuracies and none of the six domain adaptation trials’ poorest performances. This demonstrates that CM-IKFC produces greater improvements on challenging DA tasks. The proposed model achieved an average accuracy result of 89.8%, an improvement of 0.7% in comparison with the other baseline methods. We found, through our experiments on all three datasets, that good results with fewer classes were achieved, as seen with the average accuracy of Office-31 and ImageCLEF’s being higher than Office–Home.

The proposed method is well-suited for UDA in image classification tasks. However, it may not be applicable to other types of data. The performance of the method can be affected by high-dimensional input data, and it may be necessary to employ feature selection or dimensionality reduction techniques. Additionally, the method requires similar feature distributions between the SD and TD, as large differences may compromise its performance. These limitations should be taken into consideration when applying the method. The information provided is helpful in understanding the restrictions associated with this approach.

4.5. Effectiveness of the Parameter Tuning

Finally, we examined the consequences of the parameter $\delta$. In order to take the parametric sensitivities into account, we chose 4 transfer tasks from each trial. as we tuned the parameter over the range of {0.1, 0.2, 0.3, 0.5, 0.8, 1.0}. In Figure 6, the outcomes are displayed. It is clear that the results of several trials differed from one another. The performance levels in the Office-31 and Office-Home experiments rose gradually, peaked when the trade-off parameter $\delta$ was around 0.2, and then declined as $\delta$ grew. The ImageCLEF experiment showed that it performed best when $\delta$ was around 0.3 and degraded as the parameter increased. This pattern showed that the ideal $\delta$ varied depending on the transfer task; for instance, the ideal $\delta$ for task A-W from the Office-31 experiment was 0.2, although it was around $\delta$ 0.3 for other tasks within the same experiment. We selected 0.2 for Office–Home and Office-31 and 0.3 for the ImageCLEF experiment in our experimental settings because the parameters were suitable for most tasks.

5. Conclusions

The purpose of UDA is to reduce distribution discrepancy when data is being transferred data from a labeled SD to an unlabeled TD. The new CM-IKFC approach proposed creates a proper pseudo-label for every target sample. Specifically, the samples are clustered by utilizing K-Means clustering in both domains. In the TD, the clusters are matched by using cluster matching, and then this is extended in the training phase by suggesting an IKFC. This ensures that the updated images’ semantic architectures and class transitions are preserved. Furthermore, because the characteristic equation is difficult to solve, to improve the KFC, a normalized parameter is utilized. In all domains, this CM-IKFC minimizes intra-class variability while boosting inter-class variants. The results from several experiments showed that, on a variety of image benchmark datasets, CM-IKFC was superior to state-of-the-art UDA methods.