A Survey on Deep Transfer Learning and Beyond

Abstract

Deep transfer learning (DTL), which incorporates new ideas from deep neural networks into transfer learning (TL), has achieved excellent success in computer vision, text classification, behavior recognition, and natural language processing. As a branch of machine learning, DTL applies end-to-end learning to overcome the drawback of traditional machine learning that regards each dataset individually. Although some valuable and impressive general surveys exist on TL, special attention and recent advances in DTL are lacking. In this survey, we first review more than 50 representative approaches of DTL in the last decade and systematically summarize them into four categories. In particular, we further divide each category into subcategories according to models, functions, and operation objects. In addition, we discuss recent advances in TL in other fields and unsupervised TL. Finally, we provide some possible and exciting future research directions.

1. Introduction

Machine learning has been successfully applied in many fields such as face recognition [1,2], autonomous driving [3], and smart healthcare [4]. Machine learning often emphasizes that training and testing data come from the same dataset and share consistent feature distributions. However, the consistency cannot be guaranteed in practical applications. In addition, as the acquired data become larger and more complicated, several problems arise, such as few annotations in datasets, poor computational capability of devices, and model generalization with limited data. For instance, millions of images in image processing may be contained in a dataset [5]. Labeling these images is an expensive and time-consuming task. A large amount of image data and a relatively small number of labels have triggered the contradiction between the large amount of data and the few labels and the contradiction between the large amount of data and the weak computing capability. Transfer learning (TL) has been proven to be efficient in solving the above problems. In addition, many researchers have demonstrated the theoretical viability of TL; see [6,7,8,9,10,11,12,13]. For example, Wang et al. [6] investigated model complexity and learning algorithm stability to derive TL theoretical bounds, Phung et al. [10] developed efficient algorithms of domain-invariant learning, and Wu et al. [12] described these from the perspective of information theory. A large number of TL-related approaches have been proposed, as can be seen in Figure 1.

TL reapplies the learned knowledge on source domains to achieve good performance on different but related target domains [14,15]. Next, we give some definitions of TL. A domain can be represented formally as $\mathcal{D}=\left\{\mathcal{X},P\left(\mathit{X}\right)\right\}$, where $\mathcal{X}$ denotes a feature space and $P\left(\mathit{X}\right)$ denotes a marginal distribution for $\mathit{X}=[{\mathit{x}}^1,{\mathit{x}}^2,\cdots ,{\mathit{x}}^n]\in {\mathbb{R}}^{m\times n}$. For a specific domain $\mathcal{D}=\left\{\mathcal{X},P\left(\mathit{X}\right)\right\}$, a task can be represented formally as $\mathcal{T}=\left\{\mathcal{Y},f\left(·\right)\right\}$, where $\mathcal{Y}$ denotes a label space and $f\left(·\right)$ denotes a decision function. Pan et al. [14] provided a definition of TL: given a source domain ${\mathcal{D}}_{\mathrm{S}}$ and learning task ${\mathcal{T}}_{\mathrm{S}}$, a target domain ${\mathcal{D}}_{\mathrm{T}}$ and learning task ${\mathcal{T}}_{\mathrm{T}}$, TL aims to help improve the learning of the decision function $f\left(·\right)$ in ${\mathcal{D}}_{\mathrm{T}}$ using knowledge in ${\mathcal{D}}_{\mathrm{S}}$ and ${\mathcal{T}}_{\mathrm{S}}$, where ${\mathcal{D}}_{\mathrm{S}}\ne {\mathcal{D}}_{\mathrm{T}}$ or ${\mathcal{T}}_{\mathrm{S}}\ne {\mathcal{T}}_{\mathrm{T}}$. As shown in Figure 2, we assume that Amazon and DSLR (digital single-lens reflex) are the source domain ${\mathcal{D}}_{\mathrm{S}}$ and the target domain ${\mathcal{D}}_{\mathrm{T}}$. We train a classification model to complete the category-level object detection ${\mathcal{T}}_{\mathrm{S}}$ in the Amazon dataset. Transferring the trained parameters (knowledge) in the model to a new model will reduce the training cost and improve ${\mathcal{T}}_{\mathrm{T}}$ in the DSLR dataset.

According to [14,15,16,17], TL approaches can be categorized into four types: instance-based TL, model-based TL, feature-based TL, and relational-based TL. We provide a brief review of the four types as follows. (i) Instance-based TL completes the transfer by assigning different weights to different instances. A meaningful approach is using the ratio of source-domain and target-domain instances as sample weights [18,19,20]. Another method is the kernel mean matching approach [21], which matches the means between the source-domain and the target-domain instances in a reproducing kernel Hilbert space. (ii) Feature-based TL completes the transfer by transforming the features of different domains. One representative is the statistical feature transformation approach [22,23,24], which minimizes the distribution difference between source domains and target domains by statistical techniques. Another notable method is the geometric feature transformation approach [25,26,27], which implicitly aligns feature spaces between source domains and target domains by transforming features. (iii) Model-based TL completes the transfer by building models with shared parameters. These studies are broadly divided into two categories: the knowledge transfer based on shared model components and the regularization knowledge transfer based on a support vector machine (SVM). The former learns target-domain models by sharing source-domain models or hyperparameters [28,29]. The latter prevents overfitting by constraining hyperparameters with regularization terms [30,31]. (iv) Relational-based TL completes the transfer by constructing a logical mapping relationship of source domains and target domains. It is assumed that the logical relationship between source and target domains has a common pattern. Thus, the logical relationship or rules learned in source domains can be transferred to target domains. One popular approach is the first-order Markov logic network [32].

With the development of deep neural networks (DNN) [33,34,35,36], many researchers have suggested integrating deep learning techniques with TL, thereby sharing both the advantages of deep learning and TL. Thus, a huge amount of deep transfer learning (DTL) frameworks have been constructed and have been shown to be promising. It should be noted that there exists a transition from TL to DTL, i.e., incomplete DTL. Specifically, this incompleteness lies in the fact that DNN simply act as feature extractors which are combined with shallow approaches [37,38,39,40,41,42,43]. For example, Donahue et al. [39] proposed the deep convolutional activation feature (DeCAF) for generic visual recognition. This work was to acquire generic features in the source domain by DNN in a fully supervised manner. After learning enough generalization-competent features, a simple linear classifier is used to handle this task in the target domain with no or few labels. Csurka et al. [37] used the maximum mean difference (MMD) to compensate for the differences between domains to improve the extraction performance of DNN. Then, they adopted shallow approaches to complete the classification. Li et al. [38] studied a low-rank parameterized convolutional neural network that extracts common features in source and target domains to accomplish TL. Although these works have achieved relatively good performance due to the fact that DNN can learn excellent transferable representations, the two-step learning process causes the accumulation of errors to affect the final accuracy. As suggested in [44], end-to-end learning approaches can overcome this shortcoming. Therefore, increasing numbers of researchers have explored DNN architectures to construct DTL models. Figure 3 provides an overview of these approaches in chronological order.

This survey provides a comprehensive review of the recent development of DTL. The main contributions of this survey are summarized as follows.

• We introduce over 50 representative approaches of DTL and systematically summarize them into four categories and further subcategories; see Figure 4.
• We present frontier advances in the application of DTL and recent advances in unsupervised TL.
• We provide some potential research directions that can give a good reference for promoting future work in this field.

This paper is structured as follows. Section 2, Section 3, Section 4 and Section 5 summarize each of the four approaches: DTL, including model-based DTL, discrepancy-based DTL, GAN-based DTL, and relational-based DTL; see

Table 1. A brief summary of DTL approaches.

DTL Approaches	Subcategories	Brief Description
Model-Based DTL	Fine-Tuning [43,45,46,47,48,49,50] Self-Training [51,52,53] Transformer Mechanism [54,55,56,57,58]	Share and fine-tune the parameters of deep learning models
Discrepancy-Based DTL	Dual-Stream Architecture [59,60,61,62,63,64,65,66,67,68,69,70,71,72] Operate on Image Features [73,74,75]	Reduce feature discrepancies between source and target domains by DNN
GAN-Based DTL	Feature Extraction [76,77,78,79,80,81] Feature Transformation [82,83,84,85,86,87,88,89,90]	Extract domain invariant features by generative adversarial networks
Relational-Based DTL	Cross-Domain Relationship [91,92] Logical Networks [93,94]	Construct relationship using cross-domain relationship or logical networks

. Section 6 gives extensions and additions of DTL. Section 7 concludes this paper with suggestions for future research. In addition, open-source codes and datasets of DTL approaches are presented in Appendix A.

Notation: In this paper, all spaces are indicated by calligraphic uppercase letters, i.e., $\mathcal{X}$; all matrices are denoted by bold uppercase italic letters, i.e., $\mathit{X}$; all vectors are represented by bold lowercase italic letters, i.e., $\mathit{x}$; and all scalars are defined as lowercase italic letters, i.e., $\lambda$. Moreover, all variable indices, such as i, j, are denoted by italic superscripts, while specific names, such as source domain $\mathrm{S}$, target domain $\mathrm{T}$, label predictor $\mathrm{y}$, domain classifier $\mathrm{d}$, feature extractor $\mathrm{f}$, are upright and denoted in subscripts. Let ${\mathbb{R}}^{m\times n}$ characterize the set of all $m\times n$ matrices. Denote $\mathit{X}=[{\mathit{x}}^1,{\mathit{x}}^2,\cdots ,{\mathit{x}}^n]\in {\mathbb{R}}^{m\times n}$ be the input samples matrix, where m is the number of feature dimension and n is the number of samples.

2. Model-Based DTL

In this section, model-based DTL approaches are summarized into three categories. One is the fine-tuning model. which fine-tunes the parameters of source-domain networks to achieve good performance in target domains [43,45,46,47,48,49,50]. The second is the self-training approach, which is adopted to overcome the limitations of the fine-tuning model in the case of data enhancement and annotation increases [51,52,53]. The third is the transformer-based architecture, which introduces the attention mechanism in the image recognition field [54,55,56,57,58]. We give a brief summary of model-based DTL approaches in

Table 2. A brief summary of model-based DTL approaches.

Model-Based DTL Approaches	Representatives	Brief Description
Fine-Tuning	Yosinski et al. [45], DLID [43], Rozantsev et al. [47]	Reuse the different layer parameters of DNN trained in source domains
Self-Training	He et al. [51], Xie et al. [52], Zoph et al. [53]	Enhance model performance by using predicted pseudo-labels and noise
Transformer Mechanism	BEiT [55], TVT [57], Xu et al. [58]	Share and fine-tune parameters of transformer in target domains

.

2.1. Fine-Tuning

Fine-tuning is one of the earliest attempts at model-based DTL, which can be traced back to the interpolation path approach proposed by DLID: deep learning for domain adaptation by interpolating between domains [43]. Yosinski et al. [45] investigated the transferability of AlexNet in 2014. The researchers divided 1000 classification datasets into two equal parts: A and B. They trained two networks for the two datasets and fine-tuned the first seven layers of the network one by one to investigate the role of different layers in the model transfer process. The experimental results show that “transfer plus fine-tuning” leads to the best performance. The following conclusions are drawn from this experiment: the first three layers of AlexNet are general features that facilitate a transfer, and adding fine-tuning to the neural network can overcome the variability of the data to improve the network performance. Chu et al. [46] reached the same conclusion based on this experiment by considering the effect of dataset bias and the number of target-domain data markers in this experiment. These studies illustrate that the choice of the fine-tuning layer affects the model performance, which provides the basis for subsequent improvements in fine-tuning approaches.

Although the approach of fine-tuning parameters of the first few layers has subsequently been widely used, there was no clear basis for determining sharing layers. Until 2018, Rozantsev et al. [47] proposed a deep domain adaptation approach, which selectively shared and restricted parameters of different layers. The approach introduces a maximum mean difference (MMD) loss function in a dual-stream structure to measure the same layers of neural networks trained simultaneously in the target and source domains. Then, the weights for the restricted layers with large MMD losses are regularized so that the parameters satisfy a certain linear relationship. The objective function optimized in this approach by minimizing the loss function is [47]

$$L\left({\Theta }_{\mathrm{S}},{\Theta }_{\mathrm{T}}\left|{\mathit{X}}_{\mathrm{S}},Y_{\mathrm{S}},{\mathit{X}}_{\mathrm{T}},Y_{\mathrm{T}}\right.\right)=L_{\mathrm{S}}+L_{\mathrm{T}}+L_{\mathrm{W}}+L_{\mathrm{DD}},$$

(1)

where loss functions are

$$\begin{array}{cc}\hfill L_{\mathrm{S}}& =\frac{1}{n_{\mathrm{S}}}\sum _{i=1}^{n_{\mathrm{S}}}c\left({\Theta }_{\mathrm{S}}\left|{\mathit{x}}_{\mathrm{S}}^i,y_{\mathrm{S}}^i\right.\right),\hfill \\ \hfill L_{\mathrm{T}}& =\frac{1}{n_{\mathrm{T}}}\sum _{j=1}^{n_{\mathrm{T}}}c\left({\Theta }_{\mathrm{T}}\left|{\mathit{x}}_{\mathrm{T}}^j,y_{\mathrm{T}}^j\right.\right),\hfill \\ \hfill L_{\mathrm{w}}& ={\lambda }_{\mathrm{w}}\sum _{k\in \Omega }^{}{r}_{\mathrm{w}}\left({\theta }_{\mathrm{S}}^k,{\theta }_{\mathrm{T}}^k\right),\hfill \\ \hfill L_{\mathrm{DD}}& ={\lambda }_{\mathrm{u}}{r}_{\mathrm{u}}\left({\Theta }_{\mathrm{S}},{\Theta }_{\mathrm{T}}\left|{\mathit{X}}_{\mathrm{S}},{\mathit{X}}_{\mathrm{T}}\right.\right).\hfill \end{array}$$

(2)

where ${\Theta }_{\mathrm{S}}=\left\{{\theta }_{\mathrm{S}}^i\right\}$ and ${\Theta }_{\mathrm{T}}=\left\{{\theta }_{\mathrm{T}}^j\right\}$ denote the parameters of all layers in the source and target domains. ${\mathit{X}}_{\mathrm{S}}={\left\{{\mathit{x}}_{\mathrm{S}}^i\right\}}_{i=1}^{n_{\mathrm{S}}}$ and ${\mathit{X}}_{\mathrm{T}}={\left\{{\mathit{x}}_{\mathrm{T}}^j\right\}}_{j=1}^{n_{\mathrm{T}}}$ are the sets of samples from the source and target domain, respectively. $y_{\mathrm{S}}^i\in Y_{\mathrm{S}}$ and $y_{\mathrm{T}}^j\in Y_{\mathrm{T}}$ are the label set corresponding to ${\mathit{x}}_{\mathrm{S}}^i$ and ${\mathit{x}}_{\mathrm{T}}^j$. $n_{\mathrm{S}}$ and $n_{\mathrm{T}}$ denote the number of samples in the source and target domains, respectively. $c(·)$ is a standard classification loss. Furthermore, $r_{\mathrm{w}}\left(·\right)$ and $r_{\mathrm{u}}\left(·\right)$ are the weight and unsupervised regularizers. The regularizer $L_{\mathrm{W}}$ acts on the set $\Omega$ of indices of the layers whose parameters are not shared and represents the loss of the corresponding layer of the two streams. $L_{\mathrm{DD}}$ encodes the domain differences to produce a similar distribution between the source and target domains’ data representations. It is assumed that the target samples are ordered such that only the first $n_{\mathrm{T}}$ has valid labels where $n_{\mathrm{T}}=0$ in an unsupervised scenario. Since no target-domain labels are available, the optimization function $L_{\mathrm{T}}=0$ for the standard classification loss in the target domain. These regularizers are weighted by coefficients ${\lambda }_{\mathrm{w}}$ and ${\lambda }_{\mathrm{u}}$, respectively. More illustrations can be found in Figure 5.

After that, Rozantsev et al. [48] and Guo et al. [49] proposed auxiliary residual networks with adaptive fine-tuning techniques to selectively freeze and adjust the parameters, respectively. The former used two-stream network structures similar to [47] with residual transformations and performed on each neural network layer to fit the target-domain data. The latter used different parameter-tuning strategies for different instances of the target-domain data. Note that the decision to freeze or fine-tune the parameters of the pre-trained network was generated based on the Gumbel SoftMax distribution. The latest fine-tuning research addresses the overfitting problem when the number of target datasets is small. Li et al. [50] proposed to interpolate between regularization and self-labeling, including layer-wise regularization, self label-correction, and label re-weighting.

2.2. Self-Training

Although fine-tuning has achieved great success in some applications, the flaws of fine-tuning have also been identified by researchers. He et al. [51] first discovered limitations of the fine-tuning model when performing cross-dataset implementations of target detection and semantic segmentation tasks. It is found that the pre-trained model on the ImageNet dataset performed worse for the COCO dataset than the random initialization parameters approach, and ImageNet pre-training can accelerate convergence in the early stages but cannot provide regularization or improve accuracy in the final task. To overcome the disadvantage of fine-tuning, Xie et al. [52] proposed a self-training model for the weakly supervised domain adaptation problem, which uses a small number of labels from the source domain to TL across datasets. The training process is given as follows, with labeled images $\left\{\left({\mathit{x}}^1,y^1\right),\left({\mathit{x}}^2,y^2\right),\cdots ,\left({\mathit{x}}^n,y^n\right)\right\}$ and unlabeled images $\left\{{\tilde{\mathit{x}}}^1,{\tilde{\mathit{x}}}^2,\cdots ,{\tilde{\mathit{x}}}^m\right\}$. $\theta$ is parameters in networks and denotes the networks.

• Train a teacher network $\theta$ which minimizes the cross entropy loss $L\left(·\right)$ on partially labeled images, defined as

  $$\frac{1}{n}\sum _{i=1}^{n}L\left(y^i,f\left({\mathit{x}}^i,\theta \right)\right),$$

  (3)

  where $f\left(·\right)$ is a prediction function which predicts labels using network parameter $\theta$.
• The teacher network $\theta$ is used to predict the unlabeled images, and the predictions ${\tilde{y}}_i$ are used as pseudo-labels. The mathematical model is given by

  $${\tilde{y}}^i=f\left({\tilde{\mathit{x}}}^i,\theta \right),\forall i=1,2,\cdots ,m.$$

  (4)
• Train a student network $\eta$ which minimizes the cross entropy loss $L\left(·\right)$ on labeled and pseudo-labeled samples as follows:

  $$\frac{1}{n}\sum _{i=1}^{n}L\left(y^i,f_{\mathrm{noised}}\left({\mathit{x}}^i,\eta \right)\right)+\frac{1}{m}\sum _{i=1}^{m}L\left({\tilde{y}}^i,f_{\mathrm{noised}}\left({\tilde{\mathit{x}}}^i,\eta \right)\right),$$

  (5)

  where $f_{\mathrm{noised}}\left(·\right)$ is a prediction function. Noises such as dropout, random depth, and data augmentation are added to enhance the representational power of the student network.
• The student network $\eta$ is used as a new teacher network ${\theta }_{*}$; then, return to step 2.

The self-training approach is trained on unlabeled datasets to obtain generalized data representations. The key to self-training is adding noise in the training process of the student network, which enhances the smoothness of the decision function in both labeled and unlabeled data to obtain higher performance than the teacher network. Network performance is continuously enhanced during multiple iterations.

Recently, Zoph et al. [53] compared the self-training with fine-tuning and set up three control experiments through data augmentation and data-annotation addition. It can be concluded from Figure 6 that (i) stronger data augmentation and more labeled data further reduce the value of fine-tuning. (ii) Unlike fine-tuning, self-training always contributes to the training accuracy at any data augmentation strength. (iii) Even when fine-tuning works, a strategy incorporating self-training can improve performance. Therefore, combining self-training with fine-tuning approaches can greatly improve the cross-domain learning performance, which has good prospects for future development.

2.3. Transformer Attention Mechanism

The transformer attention mechanism has been demonstrated to be promising in natural language processing (NLP) [95,96,97,98]. Dosovitskiy et al. [56] successfully introduced a transformer into computer vision by chunking and spreading images into one-dimensional vectors. This mechanism provides a new framework for fine-tuning, which is different from convolutional neural networks (CNN). Chen et al. [54] first implemented a fine-tuning architecture with a transformer. They proposed the image processing converter (IPT), which applies transformers to underlying computer vision tasks. For different settings, transformer modules are shared, and only new head and tail structures need to be replaced according to the task requirements. Figure 7 shows the structure of the fine-tuning transformer. The IPT achieved state-of-the-art performance on several underlying visual tasks such as super-resolution, denoising, and rain sound removal. Bao et al. [55] proposed a self-supervised visual representation model, called Bert pre-training of image transformers (BEiT), which borrows from Bert in the field of NLP. At first, the data are pre-processed to two views: the original image and the image after randomly masking part of the image block. The goal of pre-training is to recover the original image based on the corrupted image by using a transformer. After pre-training BEiT, the model parameters on downstream tasks are fine-tuned directly by appending task layers to the pre-trained encoder. This approach allows the transformer to automatically acquire semantic region knowledge without markers so that the fine-tuning performance is greatly improved.

There are some researchers that are directly optimizing the model architecture. For example, Yang et al. [57] proposed the transferable visual transducer (TVT), which exploits the attention mechanism of the vision transformer (ViT) and the advantages of sequential images for knowledge transfer. They completed the TL by injecting the learned transferability into the attention block through a designed transferability adaptation module (TAM). Xu et al. [58] used cross-attention in transformers for feature alignment. They proposed a weight-sharing three-branch converter framework to apply self-attention and cross-attention for source-target feature learning and source-target domain alignment, respectively. However, the robustness and interpretability of model-based DTL need to be further investigated.

3. Discrepancy-Based DTL

The fine-tuning model often requires the network to contain a large amount of labeled sample data and a similar distribution of data features in the source and target domains, which cannot solve unsupervised and cross-domain problems well. Discrepancy-based DTL further explores the architecture of DNN and features of the source and target domains, which provides good solutions to the drawbacks of fine-tuning approaches. Dual-stream architectures [59,60,61,62,63,64,65,66,67,68,69,70,71,72] and approaches that operate directly on image features [73,74,75] are representative ways to perform discrepancy-based DTL. In addition, optimizing the network architecture [99] and improving the feature alignment [100,101,102,103] have also received attention from researchers in recent years. The essence of the above approaches is to minimize the feature differences between the source and target domain datasets. We give a brief summary of discrepancy-based DTL approaches in

Table 3. A brief summary of discrepancy-based DTL approaches.

Discrepancy-Based DTL Approaches	Representatives	Brief Description
Dual-Stream Architecture	DDC [60], DAN [62], JAN [64]	Reduce domain differences by introducing adaptation layers in DNN
Operate Directly on Image Features	SagNets [73], Yoon et al. [74], Yu et al. [75]	Operate directly on features extracted by DNN to align differences
Other Approaches	Das et al. [99], Li et al. [101], Zhu et al. [103]	Optimize the network architecture and improve the feature alignment

.

3.1. Dual-Stream Architecture

The dual-stream structure [104] is currently an essential framework for discrepancy-based DTL, whose core is the reduction of domain differences by introducing adaptation layers in DNN. This end-to-end learning reduces the error accumulation of multi-step learning in traditional TL. The deep architecture takes original data as input, in which neural networks process data, extract features, and align the domain. Currently, neural networks such as AlexNet [33], VGG [34], GoogleNet [35], and ResNet [36] are used as stream models for dual-stream architectures in the field of image classification and detection. The idea of dual-stream architectures is to use the same neural network to simultaneously train the source and target domains. An adaptation layer is added into the network to minimize the differences between the two domains. In addition to the different networks, the main difference lies in the design of the adaptation layer and the use of loss functions. Maximum mean difference (MMD) loss is the most common alignment in dual-stream architecture, which is formulated as [105]

$$MMD\left({\mathit{X}}_{\mathrm{S}},{\mathit{X}}_{\mathrm{T}}\right)={∥\frac{1}{n_{\mathrm{S}}}\sum _{i=1}^{n_{\mathrm{S}}}\Phi \left({\mathit{x}}_{\mathrm{S}}^i\right)-\frac{1}{n_{\mathrm{T}}}\sum _{j=1}^{n_{\mathrm{T}}}\Phi \left({\mathit{x}}_{\mathrm{T}}^j\right)∥}_{\mathcal{H}}^2,$$

(6)

where ${\mathit{x}}_{\mathrm{S}}^i\in {\mathit{X}}_{\mathrm{S}}$ and ${\mathit{x}}_{\mathrm{T}}^j\in {\mathit{X}}_{\mathrm{T}}$ are the source and target domain samples, respectively. $n_{\mathrm{S}}$ and $n_{\mathrm{T}}$ denote the number of samples in the source and target domains, respectively. $\Phi \left(·\right)$ defines a mapping from raw data to the reproducing kernel Hilbert space (RKHS). $\mathcal{H}$ indicates that the distance is metricized in RKHS.

Ghifary et al. [59] introduced MMD loss in neural networks to improve the domain adaptation performance of the network, which is the first use of MMD in DNN. In the research of the adaptation layer, MMD loss is broadly used to align different domains. Tzeng et al. [60] started the study of dual-stream architecture in TL and proposed the deep domain confusion (DDC) approach. The design of this architecture is shown in the gray part of Figure 8. They optimized a CNN architecture for classification loss $L_{\mathrm{C}}\left(·\right)$ and domain loss $L_{\mathrm{MMD}}\left(·\right)$. An adaptation layer is introduced in the previous layer of the classifier, and a domain confusion loss is computed from the output of the adaptation layer. The MMD distance between the source and target domains features is used as the domain loss, which is minimized to reduce the difference between the source and target domains. The loss function $L_{\mathrm{DDC}}\left(·\right)$ of this approach can be expressed as

$$L_{\mathrm{DDC}}=L_{\mathrm{C}}+\lambda L_{\mathrm{MMD}},$$

(7)

where $L_{\mathrm{C}}\left(·\right)$ and $L_{\mathrm{MMD}}\left(·\right)$ denote the classification loss in the source domain and domain loss, respectively. The hyperparameter $\lambda$ is used to determine the influence of domain confusion on the optimization. They jointly optimized this loss function by minimizing the classification loss and the MMD loss to maximize the domain confusion loss.

After that, Tzeng et al. [61] improved the DDC approach and designed a new CNN structure. The difference with DDC is that the last layer of the target domain network can output soft label loss. The three losses are optimized simultaneously to achieve cross-domain and cross-task recognition. Thus, the loss function can be further improved by

$$L=L_{\mathrm{DDC}}+v{L}_{\mathrm{soft}},$$

(8)

where $L_{\mathrm{soft}}\left(·\right)$ is the soft label loss in the target domain and v determines the soft label weights. This loss trains the network parameters to produce a “soft label” activation that matches the average output distribution of source examples on a network trained to classify source data. For details, please refer to Figure 8. Furthermore, Long et al. [62] proposed a deep adaptation network (DAN) architecture by adding three adaptive layers simultaneously to the first three layers of the classifier for feature constraint, which can match both low-order moments and high-order moments.

The adaptation layer introduced above can be considered a marginal domain adaptation approach. The following presents the conditional distribution adaptation, joint distribution adaptation, and dynamic distribution adaptation approaches. Zhu et al. [63] proposed the depth subdomain adaptation network (DSAN), which is a conditional distribution adaptation approach. DSAN focuses on subdomain adaptation and learns a transfer network by aligning the relevant subdomain distribution of domain-specific layer activations across domains by the local maximum mean difference (LMMD). Long et al. [64] proposed the joint adaptation network (JAN), which is a representative approach to joint distribution adaptation; see Figure 9. JAN learns a transfer network by aligning the joint distribution of multiple domain-specific layers across domains based on the joint maximum mean difference (JMMD). They adopted an adversarial training strategy to maximize JMMD, which leads to more distinguishable distributions in the source and target domains. Wang et al. [65] proposed a dynamic distribution adaptation method (DDAN) by improving the above approach. The same network structure as the JAN and DAN is used. In this work, dynamic adaptation units are embedded in the feature layer to introduce dynamic factors that dynamically adjust the weight of the marginal and conditional distributions.

The dual-stream architecture has been widely used in the field of multi-representation learning. Zhu et al. [67] proposed the multi-representation adaptive network (MRAN) approach. MRAN obtains multiple representations of the original image, and then the feature alignment is performed in different feature spaces separately to improve the accuracy of cross-domain image recognition tasks.

The DTL with dual-stream architecture presented above uses MMD loss as a rule, and the following are other forms of rules in dual-stream architecture. Correlation Alignment (CORAL) loss [68] is a well-known approach for feature alignment in dual-stream architecture, which proposes a simple and effective unsupervised adaptation that extends the linear transformation of the traditional CORAL to a nonlinear transformation. The approach accomplishes the image classification problem when the target domain is unlabeled by optimizing the source domain classification loss and CORAL loss to align the second-order statistics of domain distributions. The central moment discrepancy (CMD) approach [66] is different from the standard matching distribution approach of MMD. CMD matches the higher-order central moments of the probability distribution by sequential moment differences, which provides a new distance function for domain-invariant representation learning. The adaptive batch normalization (AdaBN) approach [69] modulates the statistical information from the source domain to the target domain in all batch normalization layers of DNN and normalizes the data to achieve the DTL. In addition, Wasserstein distance [70] is used to minimize the feature distribution differences and the transportation cost in the optimal transportation-based DTL [71,72].

3.2. Operation with Image Features

The dual-stream architecture requires various loss functions to align the discrepancies that occur when two domains are trained for the network. However, various loss functions suffer from low robustness and do not adapt well to differences in the distribution of the source and target domains. The approaches described below operate directly on image features to compensate for differences between features.

A recent study found that the main reason for the inability of deep learning to transfer across datasets in image classification tasks effectively is that CNN is more sensitive to image texture features. Therefore, the significant differences of image texture features in different datasets are one of the main reasons that prevent TL from performing effectively. Nam et al. [73] proposed style-agnostic networks (SagNets) to achieve separation of style encoding from image content for reducing domain bias. The feature extractor of SagNets (see Figure 10) extracts not only the content of the image but also the image style. In the content-biased network, the styles are randomly initialized by adaptive instance normalization (AdaIN) to make this network focus on the image content. In the style-biased network, the opposite is true. Yoon et al. [74] proposed the knowledge distillation approach, which is the latest research on style features. They generated an assistant feature by transferring an intermediate style between labeled and unlabeled samples. They then trained a TL model by minimizing the output discrepancy between the unlabeled samples and the assistant. In addition, Yu et al. [75] combined meta-learning for learning distribution matching in a data-driven manner to reduce inductive bias and proposed an approach called learning to match (L2M). L2M is a versatile framework that has shown excellent performance in the application of transfer of pneumonia to COVID-19 chest X-ray images.

3.3. Other Approaches of Feature Transfer

Apart from the above two types, some other approaches to feature transfer are described below. Das et al. [99] adapted existing domain adaptation methods to two new methods for the single rare class setting: DeerDANN, based on the Domain-Adversarial Neural Network (DANN), and DeerCORAL, based on deep correlation alignment (Deep CORAL) architectures. The two architectures augment the under-represented classes with synthetic samples, alleviating the lower classification performance for rare classes in both datasets. Li et al. [101] realized TL by using out-of-distribution detection (OOD) approaches in DNN. They trained the model to transfer domain perturbations and achieved better robustness against potential domain shifts by modeling the uncertainty of domain shifts with synthesized feature statistics during training. Although aligning local domains as closely as possible can make the connection between each neighboring domain stronger, it is worse for the alignment of distant domains. Xu et al. [100] proposed to use topology to accomplish domain adaptation. This approach reduces the effect of uniform alignment by using domain maps to encode neighboring domains. For multi-source domains learning, Ghifary et al. [102] proposed an encoder corresponding to multiple decoders. The main idea is to extract features shared across domains by a training autoencoder that reconstructs the data from different domains. The input is image data, and the output is a reconstruction of all domain analogs to that image. Zhu et al. [103] proposed a new framework with two alignment phases, which extracts domain-invariant representations of all domains by aligning the distributions of the source and target domain pairs in the common feature space.

4. GAN-Based DTL

With the great success of the generative adversarial network (GAN) [99] in image processing, researchers have attempted to incorporate the idea of GAN with TL to improve cross-domain learning. GAN is composed of two sub-networks (multilayer perception), including a generator $G\left(·\right)$ and a discriminator $D\left(·\right)$. The objective function is given by

$$\underset{G}{min}\underset{D}{max}{E}_{\mathit{x}\sim P_{\mathrm{x}}}\left(log\left(D\left(\mathit{x}\right)\right)+E_{\mathit{x}\sim P_{\mathrm{z}}}log\left(1-D\left(G\left(\mathit{z}\right)\right)\right)\right),$$

(9)

where $\mathit{x},\mathit{z}\in {\mathbb{R}}^d$ denote samples from $P_{\mathrm{x}}$ and $P_{\mathrm{z}}$, respectively, $G\left(·\right)$ learns the mapping from a priori distributions $P_{\mathrm{z}}$ to true data distributions $P_{\mathrm{true}}$, and $D\left(·\right)$ denotes probability that the input comes from the true data. $G\left(·\right)$ and $D\left(·\right)$ compete with each other to complete adversarial training. $E\left(·\right)$ is the expected probability of different distributions. When the game reaches equilibrium, the generator can generate true-looking samples.

Adversarial transfer learning (ATL) can “translate” between source-domain samples and target-domain samples while preserving the original label information for TL. ATL reduces domain discrepancies by solving the max-min game problem, which differs from the above approaches. Below, we review feature extraction approaches, feature transformation approaches, and other impressive approaches; see

Table 4. A brief summary of GAN-based DTL approaches.

GAN-Based DTL Approaches	Representatives	Brief Description
Feature Extraction Approach	DANN [76], DAAN [79], Long et al. [77]	Extract invariant features from the source and target domains in adversarial training
Feature Transformation Approach	ADDA [84], SimGAN [85], Zhu et al. [81]	Transform the features for reducing the domain bias by adversarial training
Other Approaches	ALI [106], Ma et al. [107], Kang et al. [108]	—

.

4.1. Feature Extraction Approach

The feature extraction approach of ATL extracts invariant features from the source domain and the target domain in adversarial training for TL. Ganin et al. [76] proposed the domain-adversarial neural network (DANN), which is the first approach to add an adversarial mechanism to the training of neural networks. The approach learns domain-invariant features using a feature extractor and domain discriminator competing with each other. At this time, the features extracted from the source and target domains become increasingly similar so that classification tasks can be completed in the target domain by a classifier from the source domain. Figure 11 shows the architecture of DANN. The network requires the presence of labels on the source domain data to obtain the classification loss and the source and target domain data to be separable for obtaining the domain classification loss. It passes the two losses through the gradient reversal layer (GRL) to the feature extractor for back-propagation optimization. The final goal is to make it impossible for $G_{\mathrm{d}}\left({\theta }_{\mathrm{d}}\right)$ to discriminate between the features passed by the feature extractor. The parameters ${\theta }_{\mathrm{f}}$ and ${\theta }_{\mathrm{y}}$ are optimized in this process by minimizing the classification loss for the feature extractor

$$\left({\widehat{\theta }}_{\mathrm{f}},{\widehat{\theta }}_{\mathrm{y}}\right)=\underset{{\theta }_{\mathrm{f}},{\theta }_{\mathrm{y}}}{\arg min}E\left({\theta }_{\mathrm{f}},{\theta }_{\mathrm{y}},{\theta }_{\mathrm{d}}\right).$$

(10)

Then, the loss of $G_{\mathrm{d}}\left(·\right)$ is maximized to optimize the parameters ${\theta }_{\mathrm{d}}$.

$$\left({\widehat{\theta }}_{\mathrm{d}}\right)=\underset{{\theta }_{\mathrm{d}}}{\arg max}E\left({\theta }_{\mathrm{f}},{\theta }_{\mathrm{y}},{\theta }_{\mathrm{d}}\right).$$

(11)

The DANN objective function can be obtained by adding a gradient inversion layer and merging the two training processes:

$$\begin{array}{c}\hfill E\left({\theta }_{\mathrm{f}},{\theta }_{\mathrm{y}},{\widehat{\theta }}_{\mathrm{d}}\right)=\sum _{\mathit{x}\in D_{\mathrm{S}}}{L}_{\mathrm{y}}\left(G_{\mathrm{y}}\left(G_{\mathrm{f}}\left(\mathit{x}\right)\right),y\right)-\lambda \sum _{\mathit{x}\in D_{\mathrm{S}}\cup D_{\mathrm{T}}}{L}_{\mathrm{d}}\left(G_{\mathrm{d}}\left(G_{\mathrm{f}}\left(\mathit{x}\right)\right),d\right),\end{array}$$

(12)

where $L_{\mathrm{y}}\left(·\right)$ and $L_{\mathrm{d}}\left(·\right)$ denote classification loss and discriminator loss, respectively. Note that d is the label of domains: when the data come from the source domain, $d=0$; otherwise $d=1$. $\mathit{x}\in {\mathbb{R}}^d$ and y are the input images and corresponding labels.

The discriminator in DANN receives overall features of the source and target domains, equivalent to directly optimizing the difference between the distribution of $P_{\mathrm{S}}\left(\mathit{X}\right)$ and $P_{\mathrm{T}}\left(\mathit{X}\right)$, which considers the overall distribution of data features and ignores the correlation between categories. Long et al. [77] proposed conditional adversarial domain adaptation for the improvement of the DANN, which adapts both features and categories to obtain the relationship between deep features. The approach uses multilinear mapping to optimize the GAN, which somewhat improves the negative migration. Pei et al. [78] proposed the multiple adversarial domain adaptation (MADA) based on DANN, which captures multi-modal structures to achieve fine-grained alignment of different data distributions based on multiple domain discriminators.

Inspired by the above work, Yu et al. [79] further optimized the DANN and proposed the dynamic adversarial adaptation network (DAAN) for TL. They introduced adaptive factors in the design of domain loss to dynamically and quantitatively evaluate the contribution of both marginal distribution and conditional distribution decisions to adversarial learning. Figure 12 shows the overview of DAAN. Feature extraction is performed by the depth feature extractor (blue). The features are input in the calculation of domain loss, and the dynamic measurement factor $\omega$ is updated by dynamically measuring the weights of the overall feature domain classifier (purple) and multiple local feature classifiers (green). The classifier (orange) uses a stable DNN to solve the classification in the source domain. In addition, data augmentation is also a way to learn domain invariant features. Xu et al. [80] proposed incorporating domain confusion into TL to learn the common features of source and target domain data.

4.2. Feature Transformation Approach

Feature transformation is an important approach of ATL, which transforms or aligns the features as well as reducing the domain bias by adversarial training. Domain mapping is a representative approach for feature transformation in ATL. Adversarial discriminative domain adaptation (ADDA) [84] is an approach that combines discriminative modeling, non-shared weights, and GAN losses. ADDA first learns discriminator representation by using labels in the source domain to produce domain adversarial loss, which is used to learn asymmetric mapping for mapping target-domain data to separate encoding in the same space as the source domain. Distinct from mapping target-domain data in ADDA, simulated GAN (SimGAN) [85] translates the source domain samples to the target domain for learning recognition classifiers available on two domains. Similar to SimGAN, Volpi et al. [83] used adversarial learning to translate labeled source domain data into target-domain samples while retaining the source-domain labels in the process. Domain mapping can also be used to learn domain invariant features. Zhu et al. [81] proposed cycle-consistent adversarial networks, which are trained by measuring the differences between the data after the source–target–source mapping and the original data. Feature alignment by adversarial training has also attracted the attention of researchers in recent years. Kurmi et al. [87] applied the dropout regularization in adversarial training for feature alignment. The approach replaces the point estimates with distribution estimates, which increases the variance of the sample-based distribution and uses the corresponding inverse gradients to align the source and target domain features. The point estimates are obtained by a single discriminator, and the distribution estimates are obtained by a Monte Carlo dropout discriminator. Saito et al. [82] aligned the source and target domain distribution by reducing the decision boundary. In terms of feature space transformation, Hoffman et al. [86] proposed the cycle-consistent adversarial domain adaptation (CyCADA), which combines the ideas of adversarial training and feature space transformation. They trained the model on multiple loss functions while performing feature space and pixel space alignment. Finally, the TL is completed by combining the cyclic consistency loss with the adversarial loss.

The feature transformation approach of ATL that integrates the gradient inversion layer and domain classifier into a deep network has also achieved great results in the field of target detection. Regions with CNN features (R-CNN) [109] is the first model to apply deep learning to target detection successfully. Chen et al. [88] proposed the Faster R-CNN model based on R-CNN to solve the domain offset by training two domain adaptation components through adversarial learning. He et al. [89] was inspired by Faster R-CNN to propose the multi-adversarial Faster-RCNN (MAF) for accurate object detection. MAF uses a multi adversarial domain classifier to design a feature obfuscation layer by layer domain and proposes an information invariant scale reduction module (SRM) for hierarchical feature map resizing. The approach improves the training efficiency of adversarial domain adaptation. Recently, Xu et al. [90] proposed a classification regularization framework to solve the domain discrepancy problem in target detection by matching key image regions and important instances. The ideas of GRL and adversarial training were used to construct the regularization framework.

4.3. Other Approaches of ATL

In this subsection, other approaches of ATL are introduced. Dumoulin et al. [106] proposed adversarially learned inference (ALI), which simultaneously learns the bidirectional mapping between the feature space and data space; see Figure 13. The generator completes the feature space to data space mapping, the encoder learns the reverse mapping, and the discriminator discriminates the data from the bi-directional mapping to complete the adversarial training. Furthermore, using bi-directional generative networks is the bidirectional generative domain adaptive model proposed by Yang et al. [110] for the unsupervised TL, which completes cross-domain training by interpolating two intermediate domains to bridge the source and target domain. In addition to the simultaneous mapping of data and features, Ma et al. [107] proposed the graph convolutional adversarial network (GCAN) for unsupervised domain adaptation, which jointly models data structures, domain labels, and class labels in a deep framework. GCAN is designed with three effective alignment mechanisms, including structure-aware alignment, domain alignment, and center-of-mass alignment, to effectively learn domain invariance and semantic representation to reduce domain differences.

Kang et al. [108] explicitly modeled intra-class and inter-class domain differences for adversarial training, minimizing intra-class domain differences to avoid misalignment and maximizing inter-class domain differences to enhance the generalization ability of the model. Robbiano et al. [111] proposed the adversarial branching architecture search (ABAS) for unsupervised domain adaptation, which was the first time that a neural architecture search was introduced in unsupervised domain adaptation. Wang et al. [112] measured the confidence of the optimized model by the entropy of the model prediction, in which the adversarial training of domain adaptation was accomplished by minimizing the entropy. Mitsuzumi et al. [113] proposed a general representation of the unsupervised domain adaptation, generalized domain adaptation (GDA) [113], which can learn class invariant representations and domain adversarial classifiers without using any domain labels. In addition, Sun et al. [114] proposed a robust integrated network (REN) containing a teacher network and a student network for unsupervised TL.

5. Relational-Based DTL

Relational-based DTL explores the relationships between samples in the source and target domains for cross-domain learning. It involves two mechanisms: reusing the source-domain relationship in a target domain (intra-domain relationship) and constructing a cross-dataset relationship (inter-domain relationship). The Markov logic network (MLN) provides an ideal tool for reusing the source-domain relationship in a target domain, which is a representative approach of the former type. Davis et al. [93] proposed second-order MLN for DTL by extending the first-order MLN. The basic idea is to discover structural laws in the source domain by the Markov logic formulas with relational variables, and the relationship from the target domain is used to instantiate these formulas. After that, Haaren et al. [94] optimized the second-order formulas by directly computing the posterior distribution of second-order formulas, which is taken as the prior distribution of the second-order formulas in the target domain.

Another effective approach of relational-based DTL is constructing a cross-dataset samples relationship. This explores the relationship of different datasets in the source and target domains. Recently, Isobe et al. [91] proposed a collaborative learning framework for the single-source domain and multi-target domains. The teacher network is not a single system but a source domain with all target domains corresponding to n networks. Teacher networks learn different pixel-level classification capabilities by taking advantage of the differences existing in each domain, and the knowledge learned by the different teacher networks is integrated to obtain a network with more powerful generalization capabilities. The implications of the student networks are to make the teacher networks more closely connected by regularizing the weights of the teacher networks. Thereafter, He et al. [92] applied the collaborative learning approach on semantic segmentation tasks to exploit the essential semantic information across source domains.

In addition, open-set TL [115] considers the correspondence between the source and target domain categories, so open-set TL has also been applied in relational-based DTL.

6. Extensions and Additions

6.1. TL in Other Fields

With the rise of various machine learning approaches and the increasing demand for tasks, DTL is no longer satisfied with general classification and regression tasks. Approaches for combining with other machine learning have attracted attention and become frontier research in DTL. These approaches provide solutions for breaking data barriers, securing data, addressing sample shortages, increasing model arithmetic, and explaining deep learning models. This section gives a brief introduction to frontier advances of DTL.

Table 5. A brief summary of TL in other fields.

TL Approaches	Objective
Federated TL [116,117]	Protect the privacy of tasks data when multiple tasks are working together
Safe TL [118]	Reduce the aggressiveness inherited from the pre-trained model
Few-Shot TL [119,120]	Enhance the association of few labeled samples with unlabeled samples
Open Set TL [121]	Solve the problem of inconsistent source and target domain categories
Lifelong TL [122]	Use TL techniques to improve the effectiveness of lifelong learning Adaptively
Reinforcement TL [123,124]	Reduce the interference of environmental changes on reinforcement learning

summarizes various popular TL approaches in other fields.

Although federated learning (FL) is an effective means to break data barriers, FL requires each client to collect its local data independently and thus forms different source domains, resulting in the weak generalization ability of the model. Zhang et al. [117] jointed the adversarial learning approach to solve the weak generalization, which measures and aligns the distributions between different source domains by matching each distribution to a reference distribution. For data security issues, Zhang et al. [116] proposed privacy-preserving TL for data security issues to avoid information leakage when generalizing the domain, which provides security for FL under data isolation. For the safe TL, Zhang et al. [118] applied a related model slicing technique. The approach dramatically improves the transfer accuracy by reducing the defect inheritance during TL while retaining the useful knowledge of the original model. Huang et al. [119] improved few-shot learning through TL. The spatial relationships of local descriptions, which were ignored in previous few-shot learning approaches, are effectively considered to make the learned image similarities better serve the desired domain alignment. In addition, few-shot learning is also combined with meta-learning for domain adaption. Cheng et al. [120] solved the problem of few-shot learning in which the base class and new class data come from different domains by combining meta-learning strategies. In the field of open set TL, Zhuang et al. [121] proposed a self-supervised discovery adapter to discover the implicit classes in both domains and determine the correspondence between the other categories using a part of the categories shared by both domains. In the field of lifelong TL, Yao et al. [122] proposed an adversarial feature alignment approach to address the catastrophic forgetting phenomenon, which focuses on incremental multitask image classification scenarios to provide a solution to the phenomenon in lifelong learning. In the field of reinforcement learning, Driessel et al. [123] introduced TL to transfer parameters of reinforcement learning. This work accomplishes parameter transfer for reinforcement learning by freezing the internal dynamics of learning and the value function. The TL, combined with other machine learning approaches, can utilize the characteristics of each machine learning approach to transfer knowledge in a targeted way and solve problems that cannot be solved in the original learning approach. The approach has a strong potential as a frontier and hot research area in machine learning.

6.2. Recent Advances in UTL

Less labeling or no labeling in the dataset has been a complex problem in machine learning and practical applications. Unsupervised transfer learning (UTL) has been an interesting area of TL. Many effective UTL approaches have been described in previous articles, such as self-training and ATL. This section adds to the recent progress of UTL approaches.

With the development of deep learning, the dimensionality reduction and clustering approaches in unsupervised learning have been optimized by deep learning. For completely unsupervised learning, where both the source and target domains lack labeling, Menapace et al. [125] proposed domain-independent deep clustering models by constructing data collected from multiple source domains. Some scholars have proposed the idea of joint learning by combining the approaches of clustering and dimensionality reduction with other approaches. Tian et al. [126] introduced local manifold learning for TL by combining clustering, center matching, and self-learning. The approach achieved good performance on both unsupervised and semi-supervised learning. For semi-supervised TL, Deng et al. [127], inspired by joint learning, proposed joint clustering and discriminative feature alignment (JCDFA). JCDFA unifies the mining of discriminative features and alignment of class discriminative features into a single framework to solve the discriminative clustering task for unlabeled target domains.

In addition, Luo et al. [128] explored the knowledge transfer mechanism and proposed a conditional kernel Bures (CKB) metric for characterizing differences in conditional distributions to learn the conditional invariant and discriminative features of UTL. Huang et al. [129] proposed the effective label propagation (ELP) to solve the semi-supervised TL, which enhances inter-domain semantic consistency through cyclic discrepancy loss and enhances intra-domain semantic information propagation through a self-training strategy to improve the feature discriminability in the target domain. Sun et al. [130] trained a prediction model by choosing hierarchical generation and decoupling approaches within the framework of a variational auto-encoder, which can be generalized to new domains. In addition, for semi-supervised TL, Sharma et al. [131] proposed instance-level affinity-based domain adaption (ILA-DA) to extract similar and different samples across domains by using a multi-sample comparison loss to drive the domain alignment process. ILA-DA considers both intra-class clustering and inter-class separation to reduce the boundary noise of the classifier. In addition, Wu et al. [132] improved the UTL entropy minimization by introducing diversity maximization to regulate entropy. UTL is the most common problem in practical applications, which deserves further attention by researchers.

7. Summary and Future Prospects

In this paper, we have reviewed the development of deep transfer learning (DTL) in the past decades and summarized the related mechanisms and strategies. According to their models, functions, and operation objects, we have classified DTL into four categories and further divided them into subcategories. In particular, we have demonstrated the representative models and summarized their contributions and weaknesses. Last but not least, we have given extensions and additions to DTL, which include the frontier concerns of TL and the recent advance of unsupervised TL. It is indicated that DTL has enormous advantages over traditional machine learning, and it has great potential for many real-world applications.

Although DTL has achieved great success, some essential directions need to be further investigated.

• Interpretability of DTL is a great challenge to be explored. In the field of deep learning, there is a lack of interpretability of the learning process due to the existence of black boxes. The problem continues in the DTL area, and the development of DTL requires further investigation of the interpretability.
• How to reduce the effects of negative transfer while transferring knowledge from source domains to target domains is also an important issue. Therefore, improving TL algorithms and making theoretical innovations to avoid negative transfer should be considered.
• The single DTL approach has weak ability in practical applications. Joint learning [117,118,123] and multi-view learning [133,134] can provide a good way to solve this problem. It is interesting to integrate these approaches with DTL.
• The current work summarizes existing approaches, and we will compare them on datasets/tasks and hopefully give a ranking.