A Dual-Template Prompted Mutual Learning Generative Model for Implicit Aspect-Based Sentiment Analysis

Abstract

Generative models have shown excellent results in aspect-based sentiment analysis tasks by predicting quadruples by setting specific template formats. The existing research predicts sentiment elements and enhances the dependency between elements using the multi-template prompting method, but it does not realize the information interaction in the generation process, and it ignores the dependency between the prompt template and the aspect terms and opinion terms in the input sequence. In this paper, we propose a Dual-template Prompted Mutual Learning (DPML) generative model to enhance the information interaction between generation modules. Specifically, this paper designs a dual template based on prompt learning and, at the same time, develops a mutual learning information enhancement module to guide each generated training process to interact with iterative information. Secondly, in the decoding stage, a label marking the interactive learning module is added to share the explicit emotional expression in the sequence, which can enhance the ability of the model to capture implicit emotion. On two public datasets, our model achieves an average improvement of 5.3% and 3.4% in F1 score compared with the previous state-of-the-art model. In the implicit sentiment analysis experiment, the F1 score of the proposed model in the data subset containing implicit words is increased by 2.75% and 3.42%, respectively.

1. Introduction

Aspect-Based Sentiment Analysis (ABSA) is an essential field in natural language processing. The ABSA task [1,2] aims to extract Aspect Terms (AT) and Opinion Terms (OT) from a given sentence and analyze the Sentiment Polarity (SP) and Aspect Category (AC) of aspect terms. For example, in the sentence “The computer is beautiful, but the keyboard is uncomfortable.”, “computer” and “keyboard” are aspect terms, while “beautiful” and “uncomfortable” are opinion terms, thus indicating that the sentiment polarities of “computer” and “keyboard” are “positive” and “negative”, respectively. The Aspect–Category–Opinion–Sentiment (ACOS) quadruple extraction task proposed by [3] includes implicit sentiment, allowing for a more precise expression of sentiment. As shown in Figure 1, in restaurant reviews, the sentence “I’ve waited over one hour for food” does not contain aspect terms or opinion terms but expresses negative sentiment towards the restaurant’s service. The quadruple (NULL, service general, NULL, NEG) can be extracted from the sentence.

For the ACOS task, Cai et al. [3] firstly established a series of pipeline baselines to extract quadruples by modeling them step by step through two extraction tasks and two classification tasks. However, this multi-stage propagation approach can lead to severe error propagation. More research has transformed the quadruple prediction task into a text generation task, adopting an end-to-end approach to predict quadruples [4,5,6]. Zhang et al. [2] transformed the quadruple extraction task into a paraphrasing generation problem, where the attributes of elements {(AT, AO, OT, SP)} correspond to the elements$\{{\mathcal{X}}_{\mathrm{AT}},{\mathcal{X}}_{\mathrm{AC}},{\mathcal{X}}_{\mathrm{SP}},{\mathcal{X}}_{\mathrm{OT}}\}$ in a fixed element order filled into templates. This method has become a new paradigm.

Generation-based aspect-level sentiment analysis models inherently impose a sequence on the quadruplet elements despite their inherent unordered nature. Hu et al. [7] experimentally verified that different ordering of elements had different effects on the results. Gou et al. [8] advanced an element order-based prompt learning method considering multiple element orders to build multiple views. However, their methodology employs pretrained models to select an optimal order for input, obtaining a single prediction using different strategies without engaging in interactive template manipulation during the model’s training and inference processes.

Inspired by [7,8], we propose a dual-template prompted mutual learning generative model to solve the ACOS task. Firstly, according to the disorder of quadruple elements, we set two prompt templates, and the elements in each template are marked by specific labels. Then, we construct a T5-based generation framework to generate quadruples. Secondly, to enhance the model’s ability to capture implicit words, we set the enhanced interaction module of marker-Tag, a label classification-enhanced decoding module, in the decoding stage. We add auxiliary suffixes to the input sequence to make the implicit words explicit and introduce the BIO tag, which combines the specific marker features in the template to share the explicit and implicit aspect words and opinion words in the sentence in the decoding stage of the model. In addition, we set up a mutual learning enhancement module to interact with the label information of the two template generation process decoders, make full use of the marker dependence to break the sequential decoding limit of the decoder, and learn the marker information iteratively and interactively.

The main contributions of this paper are as follows:

1.

In the ACOS task, we propose a dual-template prompted quadruple generation model based on the disorder of quadruple elements. Based on marker template features, we develop mutual learning enhancement modules in the decoding process of the dual template generation to interact with the iterative label information.

2.

In order to enhance the ability of the model to capture implicit words, we propose a marker classification-enhanced decoding module, which adds auxiliary fields in the training phase and incorporates label feature information in the decoding phase.

3.

In the ACOS task, the proposed method outperforms the baseline, with F1 scores improved by 5.3% and 3.4%. In the experiments of implicit sentiment analysis, our method improves the F1 score by 2.75% and 3.42%, respectively, in the subset containing implicit terms, demonstrating an excellent ability to capture implicit sentiment.

2. Related Work

2.1. Aspect-Based Sentiment Analysis

Aspect-based sentiment analysis has attracted extensive attention due to its more fine-grained information extraction compared with traditional sentiment analysis. In an ABSA task, the single-element prediction tasks include Aspect Term Extraction (ATE) [9,10], Opinion Term Extraction (OTE) [11,12], Aspect Sentiment Classification (ASC) [13], and Aspect Category Detection (ACD) [14] tasks. In order to perform fine-grained sentiment analysis more comprehensively, researchers have carried out composite tasks to predict multiple elements such as Aspect–Opinion Pair Extraction (AOPE) [15,16], Aspect–Category Sentiment Analysis (ACSA) [17], Aspect–Sentiment Triple Extraction (ASTE) [18,19,20], Aspect–Sentiment Quadruple Extraction (ASQE) [2,21,22,23], etc. Among them, the most challenging task is the ASQE task because it can predict four sentiment elements at once, providing the most complete sentiment analysis results.

For the ASQE task, Zhang et al. [2] transformed the original four-tuple prediction task into a text generation task and obtained the final target sequence by setting the output template of the target. This end-to-end generative modeling forms a new paradigm. Unlike the traditional autoregressive generation mode, LV et al. [6] proposed an efficient hybrid generation framework to generate emotional tuples in parallel and locate the corresponding emotional entities simultaneously. Mao et al. [24] reformulated the ABSA task as a tree sequence-to-path generation problem. Wang et al. [25] addressed the problem of data scarcity in ASQE tasks by realizing data augmentation through the inverse process of text generation quadruples. Ye et al. [21] modeled ASQE as a multi-round machine reading comprehension task and established the association between different subtasks through a multi-turn hierarchical query strategy.

The prompt template of the above generation method is single, and the disorder of elements is not taken into account. Hu et al. [7] believe that the quadruple attribute itself is disordered and found that the template order affects the performance of sequence-to-sequence learning through experiments, and the changes at different positions had different trends. Specifically, they used minimum entropy to select the appropriate template order and used multiple template orders to enhance the data in ASQE tasks to achieve good results. On this basis, Gou et al. [8] combined training and inference with multi-view prompts based on the disorder of quad-element attributes to guide the model to generate multiple sentiment tuples and then vote to select the final result. Their work selects the element order based on the element order using a pre-trained model, and there is no interactive learning for multiple template cue generation processes.

Unlike existing methods, we design a dual-template prompt for the element order of aspect and opinion terms and add a mutual learning module to enhance the interactive training of knowledge information generated through multiple paths.

2.2. Implicit Sentiment Analysis

Most existing tasks focus on learning explicit sentiment elements, and the implicit aspects and opinions in sentences can also fully show the sentiment polarity. Cai et al. [3] constructed two datasets containing implicit sentiment and jointly extracted attribute-opinion in two steps. Then, they predicted category-sentiment, but this pipeline-based method will cause error propagation. Zhou et al. [23] divided the ACOS task into two sub-tasks and adopted a one-step unified solution to solve the aspect category prediction task and the aspect opinion sentiment extraction task at the same time, which could avoid error propagation. Li et al. [26] focused on overlapping quadruples and linearized the quadruples to improve the efficiency of the model. Ju et al. [27] designed input templates to incorporate parts of speech into the model to guide the model to learning implicit emotions. Bao et al. [5] considered the semantic structure of the input sequence, combined syntactic and semantic features to build an opinion tree, and detected all sentiment elements in the opinion tree.

Contrastive learning has been widely adopted in implicit sentiment analysis, demonstrating excellent performance [28,29,30,31]. The contrastive learning approach pulls representations of entities of the same semantic class closer, while representations of entities of different semantic classes are pushed outward. Li et al. [30] utilized supervised contrastive pre-training techniques to experiment on a large-scale sentiment annotated corpus to better capture implicit and explicit sentiment tendencies in reviews. Peper et al. [28] generated negative samples in the decoding stage based on a perturbation mask to make the distance between positive and negative samples longer. Xiong et al. [29] constructed a supervised contrast learning model based on machine reading comprehension. They randomly replaced aspect and opinion terms in the quadruple to obtain negative samples and constructed questions for machine reading comprehension based on positive and negative samples. Xu et al. [31] created informative and adaptive negative samples to enhance data and combined multiple classifiers to extract quadruples.

Inspired by [31], we designed BIO tags, trying to use the decoder to obtain the position information of aspect words and opinion words. At the same time, we added auxiliary suffixes combined with BIO tags to guide the model to learn implicit sentiment.

3. Methodology

In this section, we give the definition of the ACOS task and elaborate on the constituent modules of the DPML model. As shown in Figure 2, it consists of a dual-template prompted generation module, a marker classification-enhanced decoding module, and a marker-based mutual learning-enhanced module.

3.1. Task Definition

Take a sentence $W=\{w_1,w_2,\dots ,w_n\}$ containing n words. The ACOS task is to predict all sentiment quadruples $Q={\left\{{(x_{\mathrm{AT}},x_{\mathrm{AC}},x_{\mathrm{SP}},x_{\mathrm{OT}})}_{k=1}\right\}}_k^{\left|Q\right|}$ containing four elements, the aspect term (AT), sentiment polarity (SP), aspect category (AC), and opinion term (OT), where SP∈{Positive, Neutral, Negative}. They are defined as follows:

• Aspect terms denote an entity and its aspect, indicating the evaluation target. Terms in a sentence that explicitly refer to an entity are considered explicit aspect terms, such as “computer” and “keyboard” in Figure 1. Implicit aspect terms do not explicitly refer to an entity in the sentence. In the second example in Figure 1, the term is represented as NULL.
• Sentiment polarity indicates the emotional expression displayed by the sentence, including positive, negative, and neutral.
• Aspect category represents the unique predefined category for aspects in a specific domain.
• Opinion terms refer to subjective statements about an aspect, typically subjective words or phrases in the text. Terms in a sentence that explicitly refer to an aspect are considered explicit opinion terms, such as “beautiful” and “uncomfortable” in Figure 1. Implicit opinion terms do not explicitly express an opinion in the sentence, represented as NULL in the second example in Figure 1.

3.2. Dual-Template Prompt Generation Module

To capture the dependencies between emotional elements in a sequence, we introduce a prompt template based on the order of elements. To mark different element types, similar to [7,8], we define the generation prompt template for a sentence as ${\mathcal{T}}_{\mathrm{ACSO}}:\left[\mathrm{AT}\right]x_{\mathrm{AT}}\left[\mathrm{AC}\right]x_{\mathrm{AC}}\left[\mathrm{SP}\right]x_{\mathrm{SP}}\left[\mathrm{OT}\right]x_{\mathrm{OT}}$. The markers for $x_{\mathrm{AT}}$, $x_{\mathrm{AC}}$, $x_{\mathrm{SP}}$, $x_{\mathrm{OT}}$ are [AT], [AC], [SP], and [OT]. The ACOS task predicts that the resulting quadruples are unordered, but the decoding is sequential.

To enhance the model’s perception of aspect terms and opinion terms, we exchange the positions of $x_{AT}$ and $x_{OT}$ while keeping the positions of other elements unchanged, resulting in another template ${\mathcal{T}}_{\mathrm{OCSA}}:\left[\mathrm{OT}\right]x_{\mathrm{OT}}\left[\mathrm{AC}\right]x_{\mathrm{AC}}\left[\mathrm{SP}\right]x_{\mathrm{SP}}\left[\mathrm{AT}\right]x_{\mathrm{AT}}$. To correctly identify the two templates, the prefixes “ACSO:” and “OCSA:” are added to the input. If an input sequence contains multiple quadruples, the special symbol [SSEP] connects them.

We use the pre-trained model T5-base [32] for context encoding based on the transformer architecture [33]. The sentence W is first fed to the T5 encoder to obtain the sentence representation $H_e$. During the decoding stage, the decoding output $H_{dt}$ at the t-th time step is computed based on the previous inputs $H_{d<t}$ and the context input $H_e$, formulated as follows:

$$H_{dt}=T5-Decoder(H_{d<t},H_e)$$

(1)

$${\mathrm{p}}_{\theta }\left(H_{t+1}\right|H_e,H_{d<t})=softmax\left(W^T{H}_{dt}\right)$$

(2)

where W is the transition matrix and $\theta$ is the initialization hyperparameter.

During the training stage, minimizing the cross-entropy serves as the generation loss:

$${\mathcal{L}}_{Gen}=-\sum _{t=1}^{N}log{p}_{\theta }\left(H_{dt}\right|H_e,H_{d<t}).$$

(3)

where N is the length of the sentence. We construct two prompting templates for two relatively independent generation units, where the prompting templates ${\mathcal{T}}_{ACSO}$ and ${\mathcal{T}}_{OCSA}$ lead to generation losses ${\mathcal{L}}_{Gen}^{ACSO}$ and ${\mathcal{L}}_{Gen}^{OCSA}$, respectively. The final result is obtained by intersecting the outputs from the dual-template prompting generation model.

3.3. Marker Classification-Enhanced Decoding Module

A sentence contains multiple quadruples with explicit and implicit aspects and opinion terms. Marker-based template methods can recover quadruples using different markers during the inference process. However, the special markers for implicit and explicit sentiments are the same for sequences with implicit sentiments. The generation model fails to distinguish between explicit and implicit sentiment expressions correctly. To address this issue, we design a Marker Classification Enhanced Decoding (MCED) module, as shown in Figure 3. During training, we introduced the auxiliary token “NULL” to the input sequences to help effectively distinguish between implicit aspect terms and opinion words. For sequences containing implicit or explicit aspect terms, we append an explicit suffix “NULL” to obtain the input sequence $W=\{w_1,w_2,\dots ,w_{n-1},NULL\}$, where the sequence length is N. Special markers [AT] and [OT], containing information about aspect terms and opinion terms, are used during the generation process. The designed markers can assist the model in identifying different types of elements and also guide the localization of element features [7,8,34].

We extend the BIO tags by setting nine types of tags, $T=\{B-EA,I-EA,B-IA,I-IA,O,B-EO,I-EO,B-IO,I-IO\}$, which are explicit or implicit aspect words and opinion words in the tag sequence. We utilize the decoding final layer hidden state $H_{dt}$ as the marker representation and compute the label features for aspect terms and opinion terms based on the label position $P_M$.

$$H{M}_{AT}={\mathrm{H}}_{dt}\otimes {\mathrm{P}}_{\mathrm{M}}^{\mathrm{AT}}$$

(4)

$$H{M}_{OT}={\mathrm{H}}_{dt}\otimes P_M^{OT}$$

(5)

$$M_{AT}=\sigma (W_{1}H{M}_{AT}+b_1)$$

(6)

$$M_{OT}=\sigma (W_{1}H{M}_{OT}+b_1)$$

(7)

where $W_1$ is the transition matrix, $b_1$ is the bias, and $\sigma$ denotes the ReLU activation function. The number of quadruples in a sentence is denoted as K, where $M_{AT}=\{m_1^{AT},m_2^{AT},\dots ,m_K^{AT}\}$ and $M_{OT}=\{m_1^{OT},m_2^{OT},\dots ,m_K^{OT}\}$ represent the sets of label features for aspect and opinion terms, respectively.

Aspect terms and opinion terms within multiple quadruples are mutually independent, but aspect terms or opinion terms among themselves can be shared. To predict the BIO tags from the marker features $M_{AT}$ and $M_{OT}$, using a fully connected network, where $m_i^{AT}\in M_{AT}$, $m_i^{OT}\in M_{OT}$ and $m_i^{AT},m_i^{OT}\in {\mathbb{R}}^{d\times N}$, we can formulate the process as follows:

$$p_i^{AT}=\mathrm{softmax}(W_2{m}_i^{AT}+b_2)$$

(8)

$$p_i^{OT}=\mathrm{softmax}(W_2{\mathrm{m}}_i^{OT}+b_2)$$

(9)

During the decoding stage, the model can enhance its ability to extract aspect terms and opinion terms by sharing explicit and implicit term features in sequences containing multiple quadruples. This is achieved through the interactive learning between BIO tags and marker features. The training loss function for this process is defined as cross-entropy. Specifically, for the generation processes of prompt templates ${\mathcal{T}}_{ACSO}$ and ${\mathcal{T}}_{OCSA}$, the corresponding losses are denoted as $L_{MT}^{ACSO}$ and $L_{MT}^{OCSA}$, respectively, with the following formulas.

$${\mathcal{L}}_{MT}^{ACSO-AT}=-\sum _{i=1}^K\sum _{j=1}^N\sum _{t\in T}({\mathrm{p}}_{ij}^{AT}==\mathrm{t})·log\left({\mathrm{p}}_{ij}^{AT}\right|\mathrm{t}).$$

(10)

$${\mathcal{L}}_{MT}^{ACSO-OT}=-\sum _{i=1}^K\sum _{j=1}^N\sum _{t\in T}({\mathrm{p}}_{ij}^{OT}==\mathrm{t})·log\left({\mathrm{p}}_{ij}^{OT}\right|\mathrm{t}).$$

(11)

$${\mathcal{L}}_{MT}^{ACSO}={\mathcal{L}}_{MT}^{ACSO-AT}+{\mathcal{L}}_{MT}^{ACSO-OT}$$

(12)

3.4. Marker-Based Mutual Learning Enhancement Module

The elements of the quadruple are inherently unordered. After exchanging the positions of aspect terms and opinion terms, we generated aspect terms and opinion terms with different position information based on dual templates. However, the generation processes of different templates are independent, and the model cannot share the decoded features. We have devised an enhanced training approach based on label template mutual learning to facilitate the information exchange between the dual template generation processes, as shown in Figure 2, where N is the sequence length and $H_{AT}$, $H_{OT}\in {\mathbb{R}}^{N\times d}$. The label template features $F_{AT}$ for opinion terms and $F_{OT}$ for aspect terms are obtained through the encoded stages $H_{AT}$, $H_{OT}$ and the decoding stages ${\mathrm{HM}}_{AT}$, ${\mathrm{HM}}_{OT}$, defined as follows:

$$F_{AT}=\sigma (W_3(H_{AT}\oplus {\mathrm{HM}}_{AT})+b_3)$$

(13)

$$F_{OT}=\sigma (W_3(H_{OT}\oplus {\mathrm{HM}}_{OT})+b_3)$$

(14)

where $W_3$ is the weight matrix, $b_3$ is the bias, and $\sigma$ is the ReLU activation function. Feeding $F_{AT}$ and $F_{OT}$ into a multi-layer perceptron and considering prompt templates ${\mathcal{T}}_{ACSO}$ and ${\mathcal{T}}_{OCSA}$, we obtain two decoder’s predictions for aspect terms and opinion terms, respectively, as follows:

$$y_{AT}^{ACSO}={\mathrm{MLP}}_{ACSO}\left(F_{AT}^{ACSO}\right)$$

(15)

$$y_{OT}^{ACSO}={\mathrm{MLP}}_{ACSO}\left(F_{OT}^{ACSO}\right)$$

(16)

$$y_{AT}^{OCSA}={\mathrm{MLP}}_{OCSA}\left(F_{AT}^{OCSA}\right)$$

(17)

$$y_{OT}^{OCSA}={\mathrm{MLP}}_{OCSA}\left(F_{OT}^{OCSA}\right)$$

(18)

In this way, the two decoders can interactively learn the dependencies of different template orders. During the training iteration, the two decoders can engage in knowledge learning, mutually reinforcing each other. We define the mutual learning loss for this stage as KL divergence:

$$\begin{array}{c}{L}_{KL-AT}=\mathrm{KL}(y_{AT}^{ACSO}\left|\right|y_{AT}^{OCSA})\end{array}$$

(19)

$$L_{KL-OT}=\mathrm{KL}(y_{OT}^{ACSO}\left|\right|y_{OT}^{OCSA})$$

(20)

3.5. Training and Inference

Regarding model training, we employ joint training of dual templates to optimize sequences. The loss function of DPML is as follows:

$$\mathcal{L}=\alpha ({\mathcal{L}}_{Gen}^{ACSO}+{\mathcal{L}}_{MT}^{ACSO})+(1-\alpha -\beta )({\mathcal{L}}_{Gen}^{OCSA}+{\mathcal{L}}_{MT}^{OCSA})+\beta ({\mathcal{L}}_{KL-AT}+{\mathcal{L}}_{KL-OT})$$

(21)

In terms of inference, DPML generates two distinct quadruples based on the model trained through prompt learning. To prevent the generated quadruples from deviating from the target template, we employ a Constraint Decoding (CD) strategy to ensure the format’s validity [8]. CD dynamically adjusts the candidate vocabulary during decoding at each time step based on the current token. According to the designed template, the candidate list for “[” should be “AT”, “OT”, “SP”, “AC”. Additionally, to ensure the validity of the content, the emotion element following “[SP]” must be positive, negative, or neutral.

For the two templates ${\mathcal{T}}_{ACSO}$ and ${\mathcal{T}}_{OCSA}$, we obtain two results $Y_{ACSO}$ and $Y_{OCSA}$, respectively, from which their linearized versions yield two emotion quadruples, $Q_{ACSO}$ and $Q_{OCSA}$. The final result $Y_{final}$ is obtained by taking the intersection of $Q_{ACSO}$ and $Q_{OCSA}$.

4. Experiment

4.1. Datasets

According to previous research [3,28], we evaluate our method on two benchmark datasets, Restaurant-ACOS and Laptop-ACOS [3], which contain a large number of explicit and implicit aspect terms and opinion terms. The Restaurant-ACOS dataset is constructed based on the SemEval 2016 Restaurant dataset [35] and its extended datasets [12,36]. The Laptop-ACOS dataset was collected from Amazon and includes data on brand-new laptops from the years 2017 and 2018. It covers ten different laptop models from six major brands: MBP, Acer, MSI, Lenovo, ASUS, and Samsung. The data have been annotated based on the explicit and implicit aspects, categories, and sentiment outlined in references [12,36], as well as additional opinion annotations [3]. Multiple PhD students familiar with aspect-based sentiment analysis jointly annotated these four elements [3,37]. Furthermore, following the approach proposed in [2], we include the Rest15 dataset from SemEval [35] to enhance our study on the Aspect Sentiment Quad Prediction (ASQP) task, focusing specifically on aspect-based sentiment quad prediction. This addition enables us to investigate the model’s generalization capabilities in quad-prediction tasks. Consistent with the experimental setup in [3,28], we divide the original dataset into training, validation, and test sets.

4.2. Implementation Details

We adopt the 770M parameter T5-large [32] from the Huggingface Transformers library as our pretrained generative encoder–decoder model. We adopt the AdamW [38] optimizer with a learning rate of $3\times {10}^{-4}$ to update the model parameters and set the batch size to 16. All experiments were conducted on an NVIDIA RTX 3090 GPU. We use precision (Pre), recall (Rec), and F1 score (F1) as the evaluation metrics.

4.3. Implementation Details for Implicit Sentiment Analysis

To better demonstrate the model’s ability to predict implicit terms, we divide these two datasets into five subsets: EAEO, EAIO, IAEO, IAIO, and Contain-I, following [28,29]. EAEO represents subsets containing both explicit aspect terms and opinion terms, EAIO represents subsets containing explicit aspect terms and implicit opinion terms, IAEO represents subsets containing implicit aspect terms and explicit opinion terms, IAIO represents subsets containing both implicit aspect terms and opinion terms, and Contain-I is the union of EAIO, IAEO, and IAIO, representing subsets containing implicit terms. By partitioning the dataset into five distinct subsets, we can conduct a more granular analysis of the model’s performance across various scenarios. Each subset captures different combinations of explicit and implicit terms, facilitating an evaluation of the model’s ability to handle both types of information. The EAEO subset represents the scenario that is the easiest for the model to handle, where both aspect and opinion terms are explicit. In contrast, the IAIO subset presents the most challenging scenario, where both aspect and opinion terms are implicit. Unlike [28,29], we added a CONTAIN-I subset to investigate how prediction accuracy changes when explicit and implicit information are combined.

Table 1. Statistical analysis of Restaurant-ACOS and Laptop-ACOS datasets. EA denotes an explicit aspect term, EO denotes an explicit opinion term, IA denotes an implicit aspect term, and IO denotes an implicit opinion term. Over 33% of quadruples in Restaurant-ACOS include implicit sentiment expressions, and over 43% of quadruples in Laptop-ACOS include implicit sentiment expressions.

		Restaurant-ACOS	Laptop-ACOS
Categories		13	121
Sentences		2286	4076
Quadruples	EAEO	2429 (66.40%)	3269 (56.8%)
	EAIO	530 (14.49%)	910 (15.80%)
	IAEO	350 (9.57%)	1237 (21.48%)
	IAIO	349 (9.54%)	342 (5.94%)
	Contain-I	1229 (33.6%)	2489 (43.22%)
Quadruples/Sentences		1.6	1.42

shows the statistical results of the datasets. In both datasets, the EAEO subset containing explicit aspect terms and opinion terms is the largest. It accounts for 66.40% in the Restaurant-ACOS quadruple type and 56.8% in the Laptop-ACOS, indicating that explicit sentiment expressions are common in both domains. The statistics of the CONTAIN-I dataset show that over 33% of quadruples contain implicit sentiment expressions in Restaurant-ACOS, and over 43% of quadruples do so in Laptop-ACOS. On average, each sentence in the Restaurant-ACOS dataset contains 1.6 quadruples, while each sentence in the Laptop-ACOS dataset contains 1.42 quadruples. Sentences in both datasets often contain multiple quadruples, exhibiting complex sentence structures that increase the complexity of sentiment analysis tasks.

4.4. Compared Methods

To validate the effectiveness of our method, we compare our approach with recent state-of-the-art models.

Extract–classify [3]: This is a two-step execution strategy and the first method to address the ACOS task. It first performs joint extraction of aspect–opinion pairs and then predicts aspect categories and sentiment polarities by establishing a multi-classification model. PARAPHRASE [2]: This transforms the quadruple prediction task into a T5-based semantic generation task according to the specified template. GEN_SCL_NAT [28]: In the generation method based on the T5 model, GEN_SCL_NAT introduces contrastive learning to predict quadruples with implicit terms. It generates negative samples for aspect terms, opinion terms, and sentiment polarity through perturbation to predict implicit emotions. ILO/DLO [25]: This conducts data augmentation based on template order to predict quadruples. ILO/DLO selects the order of elements based on instance-level and dataset-level sorting. MVP [8]: This is a prompt-learning generation model based on the order of elements. It regards one permutation of the quadruple as a view, predicts multiple emotional quadruples, and then aggregates multiple views.

4.5. Effects of the Coefficient

The values of coefficients $\alpha$ and $\beta$ in Equation (21) are crucial for controlling the relative influence of two different losses. Therefore, we set different values for $\alpha$ and $\beta$ to explore the impact of these parameters on the ACOS task’s F1 score. First, we fix the value of $\beta$ at 1, ensuring that the sum of the loss parameters for the two generation templates equals 1. Then, we vary $\alpha$ from 0 to 1 in increments of 0.1 to examine the impact of template switching between aspect and opinion terms in templates ${\mathcal{T}}_{ACSO}$ and ${\mathcal{T}}_{OCSA}$. As shown in Figure 4, regardless of the dataset (Restaurant-ACOS or Laptop-ACOS), the highest F1 score is achieved when $\alpha =0.5$. This indicates that while the positions of opinion terms and aspect terms differ between the two templates, the semantic information provided by the sequences is equally important.

For the hyperparameter $\beta$, we controlled for equal loss parameters of the two templates, such that in Equation (21), $\alpha =1-\alpha -\beta$. Then, we varied $\beta$ from 0 to 0.9 in increments of 0.1. Higher values of $\beta$ increase the model’s emphasis on interactive learning information, reducing the constraint on the generation process’s loss. As shown in the figure, for both datasets, the highest performance is achieved when $\alpha$ = 0.4 and $\beta$ = 0.2. This demonstrates that the weighting of the interactive learning templates significantly impacts the experiment. Appropriate interactive learning can aid the information learning process for the two-generation templates. We set $\alpha$ and $\beta$ to 0.4 and 0.2 based on the above analysis.

4.6. Result Analysis

Table 2. Experimental results on DPML. The best results of previous baselines are bolded.

Method	Restaurant-ACOS			Laptop-ACOS			Rest15
	Pre	Rec	F1	Pre	Rec	F1	Pre	Rec	F1
Extract–classify	42.95	54.08	47.87	39.21	30.35	34.22	32.93	33.41	35.44
Paraphrase	59.51	59.11	59.31	43.87	42.81	43.33	49.53	48.06	48.85
GEN_SCL_NAT	58.88	59.27	59.07	42.65	42.39	42.52	47.22	47.07	47.50
MVP	59.58	58.73	59.15	42.80	42.65	42.72	50.68	49.09	49.87
ILO	58.55	60.38	59.45	43.71	43.49	43.60	47.23	47.87	47.68
DLO	60.47	60.40	60.43	44.93	44.62	44.77	48.92	48.69	48.86
DMPL	71.07	61.14	65.73	52.26	44.81	47.06	55.34	51.71	52.88

shows the experimental results of our model compared with the baseline models, achieving 65.73% and 47.06%, respectively. According to the observation, the F1 score of our method is better than the baseline model. MVP, ILO, Paraphrase, and other generative models based on t5 model are significantly better than the joint extraction method based on the Bert pre-training model on the ACOS task. MVP, DLO, and ILO all adopt multiple prompt templates. We set the parameters according to the best results in the original paper [8] using five templates for the MVP model and seven templates for DLO and ILO. The more templates are designed, the more paths are generated, and the more time it takes to train. Compared with the generative model with multiple templates, we achieve, on average, 5.3% and 3.4% higher F1 scores on the Restaurant-ACOS and Laptop-ACOS datasets with only two templates. The precision is as high as 71.07% and 52.26%, which is 11.54% and 8.43% higher than other models on average. Compared with F1 and Pre, the Rec value is lower. This is due to the fact that the DMLP model obtains the features of different elements with the help of markers in the template to enhance the interactive iteration of the two template generation processes in the decoding stage.

In addition, we compare the performance of the model on the Rest15 dataset in the ASQP task. The F1 score of DMPL outperforms the existing T5-based model by 4.33% on average, which also shows that DMPL is suitable for all aspect sentiment quadruple prediction tasks.

According to the different types of aspects and opinions, five subsets are constructed by splitting Restaurant-ACOS and Laptop-ACOS.

Table 3. Implicit sentiment analysis.

Method	Restaurant-ACOS					Laptop-ACOS
	EAEO	EAIO	IAEO	IAIO	Contain-I	EAEO	EAIO	IAEO	IAIO	Contain-I
Extract-Classify	48.34	27.52	37.96	36.96	30.92	33.81	15.24	37.42	17.00	29.07
Paraphrase	64.18	40.61	51.02	52.43	47.55	45.09	35.51	52.74	35.15	42.76
GEN_SCL_NAT	63.06	38.37	45.48	55.45	47.44	43.74	31.44	52.94	33.54	40.28
MVP	63.86	42.54	51.05	52.74	47.62	45.43	37.06	57.00	34.18	41.95
ILO	63.29	35.34	51.34	56.46	47.41	46.55	36.09	54.63	38.27	42.37
DLO	63.48	38.89	50.00	56.71	45.87	45.58	37.44	55.64	37.63	42.41
DPML	66.45	47.88	51.79	58.73	50.37	47.50	37.44	58.12	39.44	46.18

The best results of previous baselines are bolded.

presents the F1 scores of our model and the baseline model on the subsets. In this way, the ability of the model to learn implicit words is explored. From Table 3, we can see that the model has the highest quadruple prediction power for the subset of the EAEO type. The EAEO subset only contains explicit aspect terms and opinion terms, while other subsets containing implicit terms pose more challenges to the model. Our method achieves the highest F1 score in the vast majority of subsets. Among them, the datasets containing IA or IO can predict the quadruples more effectively, especially in the Contain-I dataset, which mixes explicit and implicit aspect terms and opinion terms, which shows obvious advantages. The F1 score for the Laptop-ACOS dataset improved by 3.42%, and the F1 score for the Restaurant-ACOS dataset increased by 2.75%. Based on the statistics in Table 1, the IAEO quadruplets in the Laptop-ACOS dataset account for 21.48%, although the EAEO quadruplets, which constitute 56.8% of the total, have a higher F1 score of 58.12%, indicating that the DMPL model excels in predicting implicit opinion terms. In the Restaurant-ACOS dataset, the number of IAEO quadruplets and IAIO quadruplets is approximately 9.5% each, yet the F1 score for the IAIO quadruplets predicted by DMPL is 6.97% higher than that of the IAEO quadruplets. The DMPL model, which incorporates the features of aspect terms and opinion terms during the decoding process, enhances its ability to predict implicit sentiment.

4.7. Ablation Study and Analysis

In order to verify the influence of three key modules in the model, Marker-Based Mutual Learning (MBML)-enhanced training, dual template generation module, and Marker Classification-Enhanced Decoding (MCED) module, we conducted ablation experiments. The results are presented in

Table 4. Ablation study.

Dataset	Method	Pre	Rec	F1
Restaurant-ACOS	DLO	60.47	60.4	60.43
	DPML	71.07	61.14	65.73
	w/o MBML	72.25	58.84	64.86
	w/o MCED	70.35	59.06	64.21
	w/o ${\mathcal{T}}_{OCSA}$	61.63	61.9	61.76
	w/o ${\mathcal{T}}_{ACSO}$	60.72	60.92	60.82
Laptop-ACOS	DLO	44.93	44.62	44.77
	DPML	52.26	44.81	47.06
	w/o MBML	54.54	42.46	47.75
	w/o MCED	52.09	41.95	46.47
	w/o ${\mathcal{T}}_{OCSA}$	45.02	45.13	45.08
	w/o ${\mathcal{T}}_{ACSO}$	45.20	45.05	45.13

Bold numbers represent the best results.

.

Firstly, we ablated the MBML module. Compared with the DMPL method, the F1 scores for both datasets decreased by more than 3%. Additionally, in Section 4.4, we explored the hyperparameter $\beta$. From Figure 4b, we observe that, compared to the model without the mutual learning enhancement module ($\beta =0$), our model shows performance improvements across all values of $\beta$. This indicates that mutual learning enhancement training enables each individual generation module to exchange information, leading to a deeper understanding of the task.

For the MCED module, we first conduct ablation analysis on the Restaurant-ACOS and Laptop-ACOS datasets. The results indicate that the performance degrades when the MCED module is removed, compared to the DPML, which is decreased by 1.52% and 0.47%, respectively. This indicates that the MCED module is able to enhance term feature semantic learning by interacting with aspect terms and opinion terms through BIO tags. Furthermore, to investigate the MCED module’s ability to capturing implicit sentiments, we performed an ablation analysis on subsets focused on implicit sentiment analysis. As shown in

Table 5. Ablation study for implicit sentiment.

Dataset	Method	EAEO	EAIO	IAEO	IAIO	Contain-I
Restaurant-ACOS	DPML	66.45	47.88	51.79	58.73	50.37
	w/o MCED	65.62	31.96	50.34	53.92	49.49
Laptop-ACOS	DPML	47.50	37.44	58.12	39.44	46.18
	w/o MCED	47.2	35.25	56.44	36.00	42.89

, removing the MCED module resulted in average performance declines of 2.79% and 3.02% on the EAIO and IAIO subsets, respectively. MCED uses “NULL” to express explicit terms, which helps the model to distinguish between the semantics of explicit and implicit terms. Additionally, it uses BIO tags containing location information to interact with feature semantics in the decoding stage, which can significantly enhance the model’s prediction of implicit terms.

For the dual-template prompt generation module, we ablate the ${\mathcal{T}}_{ACSO}$ and ${\mathcal{T}}_{OCSA}$ prompt templates, respectively. Removing either prompt template also eliminates the MBML module. As observed in Table 4, the F1 scores with only the ${\mathcal{T}}_{ACSO}$ or only ${\mathcal{T}}_{OCSA}$ templates are significantly lower than the model without the mutual learning training module. The single-template generation model, such as Paraphrase, makes predictions of the elements of a quadruple based on templates, which to some extent restricts the output positions of the elements, as the decoding process can only proceed linearly. In contrast, the dual-template model is built upon the disorder of the four elements, swapping the positions of aspect terms and opinion terms and setting up two predicted outputs for modeling. The multiple paths of output provide the model with various decoding sequences. This indicates that the dual-template generation mitigates the strong dependency on the quadruple element order during the generation and decoding process. The dual templates also enhance the ability of a single template to capture the unordered characteristics of the quadruples, thereby guiding the model’s generation more effectively.

4.8. Case Study

To verify the effectiveness of DMPL, we compare the prediction results of different methods on a sentence of the test set, as shown in Figure 5. Our model is able to identify all quadruples accurately. In Example 1, extract–classify and MVP have difficulty in the extraction of aspect terms and opinion terms. In Example 2, the extract–classify model has trouble targeting the quadruples (NULL, LAPTOP#QUALITY, negative, terrible). Our model has obvious advantages in sentiment prediction with implicit terms. In addition, the classification of the Laptop-ACOS dataset, such as “SOFTWARE#GENERAL” and “SOFTWARE#GENERAL”, has similar semantics and is diverse, which also brings challenges to the model.

5. Conclusions

This paper presents a dual-template prompt mutual learning enhancement model for the ACOS task. We introduce a dual-template generation method based on prompt learning with element order. During the dual-template generation process, the model incorporates mutual learning to facilitate interactions between the template generation processes for aspect and opinion terms. Additionally, in the decoding phase, the DPML model leverages the MCED module to share multiple quadruple features, thereby enhancing the model’s ability to capture implicit sentiments. Experimental results on benchmark datasets demonstrate that our model outperforms all baseline models, where the F1 score is improved by 5.3% and 3.4% on average.

6. Future Work

Our method focuses on solving the problem of quadruple sentiment prediction. A proposal for future work can be summarized in the following four parts:

1.

The model proposed in this paper can be extended to tuples and triples to achieve a unified framework for aspect-based sentiment analysis [39,40]. Expanding the dataset across different domains and languages with the help of data augmentation techniques will also be considered.

2.

Large Language Models (LLMs) have demonstrated their prowess in sentiment analysis tasks, as evidenced by recent studies. In our work, the T5 model serves as the foundation, with comparisons made against T5-based and BERT-based models. Notably, Ref. [41] reported that zero-shot learning using LLMs was comparable to fine-tuning BERT models. Moving forward, we intend to delve deeper into the application of LLM models to address the challenge of implicit sentiment analysis.

3.

Thirdly, this paper focuses on the same domain as commonly used laptop and restaurant reviews and can further explore cross-domain and cross-language sentiment analysis tasks in the future. Furthermore, the dual templates in this paper are predefined and fixed. To facilitate aspect-based sentiment analysis tasks and to allow for greater flexibility in the influence between different templates, we plan to investigate additional template configurations and dynamic input selection to construct a unified framework for aspect-based sentiment analysis.

4.

This paper also focuses on the ABSA task of text. However, multi-modal aspect-based sentiment analysis fuses information from different modalities, which can better capture emotional elements for sentiment analysis. This will also be the direction of our further exploration in the future.