Global-Local Dynamic Adversarial Learning for Cross-Domain Sentiment Analysis

Abstract

As one of the most widely used applications in domain adaption (DA), Cross-domain sentiment analysis (CDSA) aims to tackle the barrier of lacking in sentiment labeled data. Applying an adversarial network to DA to reduce the distribution discrepancy between source and target domains is a significant advance in CDSA. This adversarial DA paradigm utilizes a single global domain discriminator or a series of local domain discriminators to reduce marginal or conditional probability distribution discrepancies. In general, each discrepancy has a different effect on domain adaption. However, the existing CDSA algorithms ignore this point. Therefore, in this paper, we propose an effective, novel and unsupervised adversarial DA paradigm, Global-Local Dynamic Adversarial Learning (GLDAL). This paradigm is able to quantitively evaluate the weights of global distribution and every local distribution. We also study how to apply GLDAL to CDSA. As GLDAL can effectively reduce the distribution discrepancy between domains, it performs well in a series of CDSA experiments and achieves improvements in classification accuracy compared to similar methods. The effectiveness of each component is demonstrated through ablation experiments on different parts and a quantitative analysis of the dynamic factor. Overall, this approach achieves the desired DA effect with domain shifts.

1. Introduction

Deep neural networks have led to impressive improvements in data mining. However, it is always time-consuming and expensive to acquire sufficient data, and especially to label them. The emergence of domain adaption [1] has significantly reduced the difficulty of labeling data and transferring knowledge between different domains. One of the most challenging problems in domain adaption is how to reduce or even eliminate the differences between the source domain and target domain [2]. In CDSA, these differences are reflected as domain discrepancies owing to the distinct expression of the reviewers’ emotions from various domains [3]. In recent years, adversarial domain adaption [4] has been widely used to reduce different domains’ distribution discrepancies. Most of them depend on either global domain discriminator to reduce marginal probability distribution discrepancy, or several local domain discriminators to reduce conditional probability distribution discrepancy. For example, DANN [4] only implements global adversarial adaption to align global distributions from different domains. Analogously, MADA [5] only utilizes a series of local domain discriminators to align subdomains. In true application scenarios, the global alignment and every single local alignment always contribute differently to the entire domain adaption. When two domains’ marginal probability distributions are more dissimilar, it is obvious that global alignment plays a bigger role. Otherwise, local alignment is more important. Similarly, each local alignment makes different contributions to the total local alignment.

Due to the issues of domain transfer and concept drift, a sentiment classification model trained in one domain may not effectively generalize to other domains. The expressions of sentiment, viewpoints, and emotionally charged words evolve over time, making it challenging to maintain effective sentiment classification models across different domains. Therefore, employing domain adaptation methods can help reduce the drift of sentiment expression [6]. The main purpose of this study is to reduce the data distribution gap between the two emotion domains to enhance the generalization ability of cross-domain emotion classification data models. We want to achieve this goal by utilizing the global-domain discriminator and local-domain discriminator commonly used in traditional natural-language-processing text classifiers to minimize the difference between data margins and global distribution, thereby improving the generalization ability of cross-domain sentiment classification.

In this paper, we propose a novel domain adaption method Global–local Dynamic Adversarial Learning (GLDAL) for unsupervised adaption. GLDAL captures the multi-mode structure of data dynamically through adversarial learning. In CDSA, GLDAL is able to amplify the influence of the pivot words to learn domain-invariant features. The core components of GLDAL are the Global Dynamic Adversarial Factor (GDAF) and Local Dynamic Adversarial Factors (LDAF). The former has the ability to evaluate the relative significance of the marginal and conditional distributions both dynamically and quantitatively. Similarly, the latter plays the same role in the relationship between each conditional distribution. Therefore, GLDAL is able to improve the generalization of adversarial domain adaption.

In the following text, we detail the use of our proposed approach in deep neural network architectures, and experiment on popular sentiment classification datasets (SST, IMDB), where our proposed approach shows improvements compared to the previous state-of-the-art accuracy.

2. Related Work

2.1. Unsupervised Domain Adaption in Transfer Learning

Unsupervised domain adaptation (UDA) is an important branch of transfer learning. There are two main forms of UDA: traditional machine learning and deep learning. UDA based on traditional machine learning can be divided into two categories: (1) Subspace Alignment (SA) [7] and CORAL [8] use the subspace statistical characteristics to eliminate domain differences. (2) Distribution alignments TCA [9], JDA [10], BDA [11] and MEDA [12] are proposed to align the marginal probability distributions or conditional probability distributions between domains.

In the last few years, deep neural networks have been widely used in domain adaption [13,14,15]. Domain adversarial learning, as a branch of deep domain adaption, has been popular in recent years. The idea of adversarial learning comes from Generative Adversarial Networks (GAN) [16,17]. DANN [4] aligns the source and target distributions with only a single global domain discriminator. MADA [5] is able to execute the fine-grained alignment of different data distributions using multi-mode discriminators. DAAN [18] dynamically evaluates the relationship between the marginal and conditional distributions. Our GLDAL is also based on adversarial learning and global attention mechanism. By evaluating the relationship between every local subdomain and the relative importance of the marginal and conditional distributions, GLDAL significantly outperforms existing methods.

2.2. Cross-Domain Sentiment Analysis

Sentiment analysis is also called opinion mining or tendency analysis. It is the process of analyzing and reasoning about subjective texts with emotions. Text representation is an important step in sentiment analysis. In recent years, the most representative work has been the BERT pre-trained model [19], Transformer-XL [20]. However, thegeneral sentiment analysis studies mentioned above only consider the performanc e in a single domain, ignoring the generalization ability, so cross-domain sentiment analysis has quickly become a research hotspot. BERT-DAAT [21] is a pre-trained model based on adversarial training methods that aims to provide the trained BERT with domain awareness. ADS-SAL [22] can dynamically learn an alignment weight for each word, so more important words will obtain higher alignment weights to achieve fine-grained adaptation.

3. Method

3.1. Problem Definition

In CDSA tasks about domain adaption, there are a source domain $D_s={\left\{x_s^i,y_s^i\right\}}_{i=1}^{N_s}$ and a target domain $D_t={\left\{x_t^i\right\}}_{i=1}^{N_t}$ , where $x_s$ and $x_t$ is the set of sentences and $y_s$ are the corresponding sentiment polarity labels. The goal of CDSA is to predict the target label ${\widehat{y}}^t=argmax{G}_y\left(f\left(x_t\right)\right)$ and minimize the target risk ${ϵ}_t\left(G_y\right)=E_{\left(x_t,y_t\right)\sim {\mathcal{D}}_t}\left[G_y\left(f\left(x_t\right)\right)\ne y_t\right]$, where $G_y(·)$ represents the Softmax output and $f(·)$ refers to the feature representation.

3.2. Backbones for Cross-Domain Sentiment Analysis

3.2.1. Bidirectional Gate Recurrent Units with Attention

A Gate Recurrent Unit (GRU) with the attention mechanism [23] is an effective recurrent neural network for sentiment analysis. For a given $\mathrm{n}$-dimensional input $\left(x_1,x_2,\dots ,x_n\right)$, the hidden layer of Bi-GRU outputs $h_t$ at time $\mathrm{t}$. The calculation process is as follows:

$$h_{f_t}=\sigma \left(W_{x{h}_f}{x}_t+W_{h_{f^{h}f}}{h}_{f_{t-1}}+b_{h_f}\right),h_{b_t}=\sigma \left(W_{x{h}_b}{x}_t+W_{h_b{h}_b}{h}_{b_{t-1}}+b_{h_b}\right),$$

(1)

$h_t=h_{f_t}\oplus h_{b_t}$, W is the weight matrix and b is the bias vector; $\sigma$ is the activation function; $h_{f_t}$ and $h_{b_t}$ are the outputs of positive and negative GRU, respectively. ⊕ represents the element-wise summation. The simplification method is to construct a single vector c from the whole sequence, as follows:

$$e_t=\mathit{Attention}\left(h_t\right),{\alpha }_t=\frac{exp\left(e_t\right)}{{\sum }_{k=1}^{T}exp\left(e_k\right)},c={\displaystyle \sum _{t=1}^T}{\alpha }_t{h}_t.$$

(2)

3.2.2. Capsule Neural Network Based on BERT

The BERT [19] model is a language model based on two-way Transformer. We directly use the feature representation of BERT as the word-embedding feature of the task and regard BERT as the upstream network. The Capsule Network [24] is composed of a group of neurons, which are used to represent the parameters of a specific type of object. We regard this network as a downstream network. The unique activation function (“squashing”) is formulated as:

$${\mathbf{v}}_j=\frac{{∥{\mathbf{s}}_j∥}^2}{1+{∥{\mathbf{s}}_j∥}^2}\frac{{\mathbf{s}}_j}{∥{\mathbf{s}}_j∥},$$

(3)

${\mathbf{s}}_j$ is its total input and $v_j$ is the output of capsule $\mathit{j}$ . For capsule $s_j$, the input is determined by multiplying the output $u_i$ of a capsule in the former layer by a weight matrix $W_{ij}$ , $c_{ij}$, which are coefficients for coupling determined by the dynamic routing:

$$s_j=\sum _i{c}_{ij}{\widehat{u}}_{j\mid i},{\widehat{u}}_{j\mid i}=W_{ij}{u}_i.$$

(4)

3.3. Domain Adaption with Adversarial Learning

This adversarial training is like a game with two parts: feature extractor $G_f$ and domain discriminator $G_d$. Through maximizing the loss of $G_d$, the parameters ${\theta }_f$ of $G_f$ are trained while the parameters ${\theta }_d$ of $G_d$ are trained by minimizing the loss of the domain discriminator. The total loss can be formalized as:

$$\begin{array}{c}\hfill L\left({\theta }_f,{\theta }_y,{\theta }_d\right)=\frac{1}{n_s}\sum _{{\mathbf{x}}_i\in {\mathcal{D}}_s}{L}_y\left(G_y\left(G_f\left({\mathbf{x}}_i\right)\right),y_i\right)\\ \hfill -\frac{\lambda }{n_s+n_t}\sum _{{\mathbf{x}}_i\in \left({\mathcal{D}}_s\cup {\mathcal{D}}_t\right)}{L}_d\left(G_d\left(G_f\left({\mathbf{x}}_i\right)\right),d_i\right),\end{array}$$

(5)

Update the parameters of each part as the following equations:

$$\left({\widehat{\theta }}_f,{\widehat{\theta }}_y\right)=arg\underset{{\theta }_f,{\theta }_y}{min}L\left({\theta }_f,{\theta }_y,{\theta }_d\right),\left({\widehat{\theta }}_d\right)=arg\underset{{\theta }_d}{max}L\left({\theta }_f,{\theta }_y,{\theta }_d\right)$$

(6)

Due to the adversarial relationship with parameter updating, the parameters ${\theta }_f$, ${\theta }_y$, ${\theta }_d$ will deliver a saddle point of Equation (6) after the training converges.

3.4. Global-Local Dynamic Adversarial Adaption Network

At present, the mainstream adversarial adaption methods reduce marginal or conditional probability distribution discrepancies. However, it is quite difficult to evaluate the importance of each contribution. Therefore, we should find a novel method that can quantitatively evaluate their importance and dynamically adjust the parameters of the neural network.

In this paper, we propose the Global–local Dynamic Adversarial Learning (GLDAL) shown in Figure 1 to make key improvements. GLDAL can effectively evaluate every single distribution’s importance, aiming to better learn the domain-invariant features through adversarial learning. In GLDAL, the Global domain discriminator $G_d$ and a series of Local domain discriminators $G_d^u$ play a role in the domain adaption of marginal and conditional distributions, respectively. The most important innovation point is that we propose a novel, global-local dynamic training strategy using the Global Dynamic Factor $\omega$ and Local Dynamic Factors${\left\{{\alpha }_u\right\}}_{u=1}^U$.

Figure 1. The architecture of the proposed Global–local Dynamic Adversarial Learning (GLDAL). First, input the Text Sequences into the Feature Extractor to obtain the feature vector distributed in the feature space. Next, there are two main loss functions, namely $L_y$ and $L_d$. $L_y$ is the classification loss. $L_d$ is the loss combination of the global domain discriminator and local domain discriminators.

3.4.1. Label Classifier

The label classifier $G_y$ (The blue part in Figure 1) in trained by samples from the source domain to implement label classification. The loss of $G_y$ is formulated as:

$$L_y=\frac{1}{n_s}\sum _{{\mathbf{x}}_i\in {\mathcal{D}}_s}CrossEntropy\left(G_y\left(G_f\left({\mathbf{x}}_i\right)\right),{\mathbf{y}}_i\right),$$

(7)

where ${\mathbf{x}}_i,{\mathbf{y}}_i$ represents samples and their labels from the source domain; $G_f$ is the feature extractor.

3.4.2. Global Domain Discriminator

Global domain discriminator $G_d$ (The pink part in Figure 1) is wisely implemented to align the marginal probability distributions between two domains (source domain and target domain) in the feature space. We define the loss of global domain discriminator $G_d$ as:

$$L_{global}=\frac{1}{n_s+n_t}\sum _{{\mathbf{x}}_i\in {\mathcal{D}}_s\cup {\mathcal{D}}_t}{L}_d\left(G_d\left(G_f\left({\mathbf{x}}_i\right)\right),d_i\right).$$

(8)

In this formula, $L_d$ represents the loss of domain discriminator. $d_i$ is the domain label of the sample ${\mathbf{x}}_i$ and $G_f$ is the feature extractor.

3.4.3. Local Domain Discriminator

The local domain discriminator $G_d^u$ (The purple part in Figure 1) is composed of a series of U class-wise discriminators $G_d^u$; each is used to match the source domain and target domain data associated with class u. The output of the label classifier ${\widehat{\mathbf{y}}}_i=G_y\left({\mathbf{x}}_i\right)$ to each data point $x_i$ is a probability distribution of the U classes. Therefore, we utilize the probability distribution ${\widehat{\mathbf{y}}}_i$ to measure how many data points $x_i$ should be attended to the U domain $G_d^u$. The loss function of the local domain discriminator is defined as:

$$L_{local}=\frac{1}{n_s+n_t}\sum _{u=1}^U\sum _{{\mathbf{x}}_i\in {\mathcal{D}}_s\cup {\mathcal{D}}_t}{L}_d^u\left(G_d^u\left({\widehat{y}}_i^u{G}_f\left({\mathbf{x}}_i\right)\right),d_i\right),$$

(9)

3.5. Global and Local Adversarial Factors

In this part, we introduce how to quantitively evaluate the global distributions and each local distribution. Due to the adversarial characteristic, it is tricky to determine a concrete scheme. Generally, there are two straightforward ideas: Random Setting and Average Step Searching. The former randomly sets the value of dynamic factors and the latter picks the value of these factors with the same step size. For instance, if the scope of factor $\omega$ is [0, 1], the latter strategy will pick the value of $\omega$ = 0, 0.1, …, 0.9, 1.0 to perform training. However, both strategies are computation-consuming and time-consuming.

Therefore, in this paper, we propose two kinds of factors: global dynamic adversarial factor $\omega$ and local dynamic adversarial factors ${\left\{{\alpha }_u\right\}}_{u=1}^U$ in order to dynamically and quantitively evaluate the importance of each distribution. This strategy has two significant advantages. Firstly, we utilize deep adversarial representations instead of shallow representations to learn these two kinds of factors, making GLDAL more robust. Secondly, every single dynamic factor is determined by the $\mathcal{SA}-distance$ (not hinge loss) of the discriminators, which is more efficient and convenient. In order to calculate the dynamic factors, we denote the global $\mathcal{SA}-distance$ of the global domain discriminator as:

$$d_{\mathcal{SA},g}\left({\mathcal{D}}_s,{\mathcal{D}}_t\right)=2\left(1-2\left(L_{global}\right)\right).$$

(10)

We also denote the local subdomain $\mathcal{SA}-distance$ as:

$$d_{\mathcal{SA},l}\left({\mathcal{D}}_s^u,{\mathcal{D}}_t^u\right)=2\left(1-2\left(L_{local}^u\right)\right).$$

(11)

These two kinds of distances can measure the distribution similarity of the source and target domain to some extent. ${\mathcal{D}}_s^u$ and ${\mathcal{D}}_t^u$ denote samples from class $\mathit{u}$, and $L_{local}^u$ is the local subdomain discriminator loss over the class $\mathit{u}$.

3.5.1. The Global Dynamic Adversarial Factor $\omega$

The global dynamic adversarial factor $\omega$ is used to calculate the weighted sum of global domain loss and local domain loss. $\omega$ is initialized as 1 in the first training epoch. After each training epoch, this factor $\omega$ can be estimated through global and local domain discriminators as:

$$\widehat{\omega }=\frac{d_{\mathcal{SA},g}\left({\mathcal{D}}_s,{\mathcal{D}}_t\right)}{d_{\mathcal{SA},g}\left({\mathcal{D}}_s,{\mathcal{D}}_t\right)+\frac{1}{U}{\displaystyle \sum _{u=1}^U}{d}_{\mathcal{SA},l}\left({\mathcal{D}}_s^u,{\mathcal{D}}_t^u\right)}$$

(12)

3.5.2. The Local Dynamic Adversarial Factors ${\left\{{\alpha }_u\right\}}_{u=1}^U$

The local domain loss $L_{local}$ is composed of a series of local subdomain loss $L_{local}^u$ by weighted summation. The local dynamic adversarial factors ${\left\{{\alpha }_u\right\}}_{u=1}^U$ act as coefficients in the weighted sum. The importance of the alignment of each subdomain is different, so we assign different weights to every subdomain. In general, if the difference between the source subdomain and the target subdomain regarding class u is larger, we should pay more attention to this subdomain, which is manifested by the fact that the weight of the alignment of this subdomain should be increased. Since $\mathcal{SA}-distance$ is an important parameter used to measure the similarity of the probability distributions of two subdomains ${\mathcal{D}}_s^u$ and ${\mathcal{D}}_t^u$, the parameter ${\alpha }_u$ can be estimated as:

$${\widehat{\alpha }}_u=\frac{\frac{1}{U}{\displaystyle \sum _{u=1}^U}{d}_{\mathcal{SA},l}\left({\mathcal{D}}_s^u,{\mathcal{D}}_t^u\right)}{d_{\mathcal{SA},l}\left({\mathcal{D}}_s^u,{\mathcal{D}}_t^u\right)}.$$

(13)

Each subdomain dynamic adversarial factor is initialized as 1 in the first training epoch. This factor-updating strategy follows the principle: If the $\mathcal{SA}-distance$ (Generally less than 0) between the subdomains about class u is larger, the weight for the alignment of subdomains about u will be greater. In this case, the domain discriminator $G_d^u$ of the subdomain u can more easily discriminate the domain label information of the sample in the subdomain, which indicates that the difference between domains about class u is large. Therefore, we need to increase the proportion of $L_{local}^u$ in the total local loss to ensure a better alignment effect about class u.

4. Experiments

In this section, we implement experiments to evaluate the proposed GLDAL against several previous state-of-the-art domain adaption methods. Our method GLDAL is validated on several popular standard datasets regarding cross-domain sentiment analysis. The training process is shown in Algorithm 1.

Algorithm 1  GLDAL

Input:

—samples $S={\left\{x_s^i,y_s^i\right\}}_{i=1}^{N_s}$ and $T={\left\{x_t^i\right\}}_{i=1}^{N_t}$, $\lambda$, $\mu$, $\omega$, ${\left\{{\alpha }_u\right\}}_{u=1}^U$

Output:  Neural Network $\left\{G_y,G_f,G_d\right\}$

while stopping criterion is not meet do           calculate the global domain loss $L_{{global}_i}$: Equation (8)           calculate the global $\mathcal{SA}-distance$: Equation (10)           acquire classification probability vector $\widehat{y_i}$ from $G_y$: $\widehat{y_i}⟵G_y\left(G_f\left({\mathbf{s}}_i\right)\right)$           calculate the sum of all local loss of sample $s_i$: Equation (9)           calculate each local $\mathcal{SA}-distance$ : Equation (11)           if $s_i\in S$ then               calculate the classification loss of: Equation (7)           Backpropagation: ${\Delta }_{\Theta }\leftarrow \frac{\Delta L_{y_i}}{\Delta \Theta }-\lambda \frac{\Delta \left((1-\omega )L_{global}+\omega L_{local}\right)}{\Delta \Theta }$           Update dynamic factor $\omega ,{\left\{{\alpha }_u\right\}}_{u=1}^U$:           $\widehat{\omega }$: Equation (12),  ${\widehat{\alpha }}_u$: Equation (13)     return Output

4.1. Datasets

The Amazon Product Review dataset includes product reviews and metadata from Amazon; due to the uneven distribution of the samples, we set rating 0–1 as negative sentiment, 2–3 as medium and 5 as positive. We chose the following two datasets: Amazon reviews for clothing (C) and Amazon reviews for instant video (I).

SST-5 (S5), SST-2 (S2) are two versions of The Stanford Sentiment Treebank (SST); the former is (S5), with five categories (Very Positive, Positive, Neural, Negative), and the latter is (S2) with two categories (Positive, Negative).

IMDB (IM) is a review rating dataset for movie sentiment analysis, containing the same number of positive and negative sentiment samples. The COVID-19 Review dataset (COR) contains a sample of comments about COVID-19 and the corresponding sentiment scores (Very Positive, Positive, Neural, Negative). Tweet Review dataset TR contains daily comments and sentiment ratings scraped from tweets, with three categories (Positive, Neural, Negative).

We used these domain combinations and built six transfer learning tasks: (C → TR), (TR → I), (COR → S5), (S5 → COR), (IM → S2), (S2 → IM).

4.2. Baselines

Based on two previously mentioned backbones: Bi-GRU-A (Section 3.2.1) and CapsuleNet (Section 3.2.2),We compare our proposed Global–local Dynamic Adversarial Learning (GLDAL) with several state-of-the-art unsupervised deep domain adaption methods: DDC [25], DaNN [26], DANN [4], D-CORAL [15], JAN [27] MADA [5] and DAAN [18].

4.3. Implementation

We implement all methods on the PyTorch framework. For feature extractor backbones CapsuleNet and Bi-GRU-A, we fine-tune all word vectors and all attention layers and train the feature extraction layers with a learning rate of ${10}^{-5}$ and ${10}^{-4}$, respectively. We use approximately 2–5 times the learning rate of the feature extractor to train the label classifier, and use 2–10 times the learning rate to train the local discriminator and global discriminator. The update of the important trade-off factor $\lambda$ follows the Warm Start principles: $\lambda =\frac{2(hi-lo)}{1+exp\left(-\frac{i}{N}\right)}-(hi-lo)+lo$. We set $lo$ as 0, $hi$ as 1, N as 100. Other hyperparameters are tuned via transfer cross-validation. The source code is available at https://github.com/killer2-1/GLDAL-for-CDSA (accessed on 20 May 2023).

4.4. Results

The classification accuracy (%) is shown in Table 1 and Table 2. GLDAL outperforms all comparison methods on most CDSA tasks. It is also remarkable that our GLDAL has better effects than similar approaches such as DANN, MADA and DAAN. From the results, we can obtain some conclusions: (1) Generally, the methods based on adversarial learning (DANN, DAAN) perform better than non-adversarial learning methods (DaNN, DDC), which means that adversarial adaption is more effective. (2) Adversarial methods with an attention mechanism such as GLDAL achieves a better performance than DANN and MADA(no attention). (3) Compared with other methods, GLDAL outperforms almost all traditional comparison methods (DDC, DaNN, DANN) on most transfer tasks, which proves the proposed method is effective. For some of the more advanced methods, such as DAAN, GLDAL slightly outperforms these methods, which verifies our method’s benefits.

Table 1. Accuracy (%) for unsupervised domain adaptation based on Bi-GRU-A(The bold stands for the best performance).

Method	C → TR	TR → I	COR → S5	S5 → COR	IM → S2	S2 → IM	AVG
Bi-GRU-A (No Transfer)	40.96	62.04	31.70	29.53	76.64	74.18	52.51
DDC	49.85	61.88	30.95	32.60	76.99	77.72	55.00
DaNN	45.35	63.65	31.31	32.00	78.46	78.40	54.86
DANN	49.01	65.98	32.84	33.53	80.81	78.82	56.83
D-CORAL	50.43	65.06	33.78	33.70	79.13	77.63	56.62
MADA	50.18	66.39	33.12	33.96	81.16	78.98	57.30
DAAN	50.69	66.05	33.09	34.08	80.99	79.02	57.32
GLDAL (Proposed Approach)	51.25	66.53	33.34	34.12	81.34	79.16	57.62

Table 2. Accuracy (%) of unsupervised domain adaptation based on CapsuleNet.

Method	C → TR	TR → I	COR → S5	S5 → COR	IM → S2	S2 → IM	AVG
CapsuleNet (No Transfer)	47.94	63.93	33.25	30.24	83.51	80.22	56.52
DDC	51.77	64.34	35.09	30.98	85.97	86.07	59.20
DaNN	50.16	66.56	34.15	31.49	85.45	86.14	58.99
DANN	53.55	67.72	35.21	34.45	86.11	85.60	60.44
D-CORAL	56.82	67.08	34.08	34.38	86.05	85.72	60.68
JAN	52.97	68.24	34.99	33.82	86.33	86.58	60.48
MADA	54.31	67.65	35.79	34.64	86.96	86.32	60.95
DAAN	54.76	68.78	34.96	34.24	87.21	86.50	61.08
GLDAL (Proposed Approach)	56.97	68.92	34.91	35.07	87.18	86.83	61.64

4.5. Effectiveness Analysis and Ablation Study

4.5.1. Analysis of the Importance of the Global Dynamic Adversarial Factor (GDAF) $\omega$ in GLDAL

In this section, the importance of GDAF $\omega$ in GLDAL is evaluated. The evaluation involves two key points: (1) Whether we should pay attention to the different effects of marginal and conditional probability in adverarial domain adaption. (2) The effectiveness of our evaluation for $\omega$.

To illustrate the first point, we chose several transfer tasks and analyzed the results of GLDAL under different values of $\omega$, as shown in in Figure 2. It is obvious that paying attention to the different effects of marginal and conditional distribution is important. The value of optimal $\omega$ varies on different tasks. This is because different feature representations are obtained under different $\omega$.

Figure 2. Performance of several tasks when searching $\omega \in [0,1]$.

To illustrate the second point, we compared the accuracy of transfer tasks under different calculation methods for $\omega$: Random Guessing, Average Searching (10 times) and our GLDAL. We also executed an ablation study with DANN ($\omega$ = 1) and MADA ($\omega$ = 0). Combining the results from Table 3, we can conclude that our method outperforms other four methods.

Table 3. Accuracy (%) for the effective analysis and ablation study of $\omega$ based on CapsuleNet.

Transfer Tasks	Average Search	Random Guessing	GLDAL	DANN ($\mathit{\omega }$ = 1)	MADA ($\mathit{\omega }$ = 0)
S2 → IM	86.15	86.52	86.83	85.60	86.32
S5 → COR	34.98	34.86	35.07	34.45	34.64
TR → I	67.73	68.25	68.92	67.72	67.65
AVG	62.95	63.21	63.61	62.60	62.87

4.5.2. Analysis of the Importance of the Local Dynamic Adversarial Factor (LDAF) ${\left\{{\alpha }_u\right\}}_{u=1}^U$ in GLDAL

In this section, based on the point oof whether we should pay attention to the different effect of each local subdomain, we analyze the importance of the local dynamic adversarial factor ${\left\{{\alpha }_u\right\}}_{u=1}^U$ in GLDAL. Like the analysis of $\omega$, we also chose a series of tasks, and adopted the three methods shown below to conduct verification experiments: (1) A series of random number that are greater than 0, and the sum is fixed to U (RNSF); (2) DAAN (all LDAFs are fixed to 1); (3) Our GLDAL. We used $\mathcal{A}-distance$ [28] as a measure of cross-domain discrepancy. In general, the smaller the $\mathcal{A}-distance$, the better the domain adaption effect. The average results in each dataset are shown in Table 4.

Table 4. $\mathcal{A}-distance$ for all unsupervised domain adaptation tasks based on backbone CapsuleNet and Bi-GRU-A.

BackBone	RNSF	DAAN	GLDAL
CapsuleNet	0.988	0.952	0.925
Bi-GRU-A	1.053	1.007	0.978

5. Conclusions

In this paper, we proposed a novel Global–local Dynamic Adversarial Learning (GLDAL) for cross-domain sentiment analysis. Through quantitatively evaluating the relative importance of global distribution and all local distributions, GLDAL is able to dynamically adjust the learning weights of the global discriminator and local discriminators during training. This domain adaption strategy is based on the attention mechanism and has excellent effects on experimental tasks. As a general transfer learning strategy, GLDAL can also be applied to tasks in computer vision and the recommended system. In the future, we plan to extend GLDAL to more challenging transfer learning problems.