SI-GCN: Modeling Specific-Aspect and Inter-Aspect Graph Convolutional Networks for Aspect-Level Sentiment Analysis

Abstract

Aspect-level sentiment analysis (ALSA) aims to identify the sentiment polarity associated with specific aspects in textual data. However, existing methods utilizing graph convolutional networks (GCNs) face significant challenges, particularly in analyzing sentiments for multi-word aspects and capturing sentiment relationships across multiple aspects in complex sentences. To address these issues, we introduce the Specific-aspect and Inter-aspect Graph Convolutional Network (SI-GCN), which integrates contextual information, syntactic dependencies, and commonsense knowledge to provide a robust solution. The SI-GCN model incorporates several innovative components: a Specific-aspect GCN module that effectively captures sentiment features for individual aspects; a knowledge-enhanced heterogeneous graph designed to manage implicit sentiment expressions and multi-word aspects; and a dual affine attention mechanism that accurately models inter-aspect relationships. Compared to existing state-of-the-art methods, the SI-GCN achieves improvements in performance ranging from 0.9% to 2.3% across four benchmark datasets. A detailed analysis of text semantics shows that the SI-GCN excels in challenging scenarios, including those involving aspects without explicit sentiment indicators, multi-word aspects, and informal language structures.

1. Introduction

Aspect-level sentiment analysis (ALSA) [1,2,3,4,5] presents distinct challenges within the field of natural language processing, focusing on identifying sentiments associated with specific entities, or aspects, within a given text [6,7]. Unlike conventional sentiment analysis, which evaluates the overall sentiment of an entire text, ALSA targets the emotional valence—positive, negative, or neutral—of discrete elements within the text. This focused analysis offers deeper insights into consumer opinions and customer satisfaction regarding specific features of a product or service, which can be invaluable for making recommendations for improvement. The applications of ALSA span several domains, including product recommendations [8,9], political stance evaluation [10,11], medical text analysis [12,13], and public health emergency response [14].

ALSA aims to identify and evaluate the sentiments associated with specific aspects mentioned in a text. For instance, in the sentence “The service at the restaurant was excellent, but the food was mediocre”, two aspects are highlighted: “service” and “food”. In this context, the model should recognize that “excellent” conveys a positive sentiment toward “service”, while “mediocre” indicates a negative sentiment toward “food”. ALSA involves analyzing sentences to uncover the distinct sentiments linked to each aspect, facilitating a nuanced understanding of the opinions expressed in reviews and comments.

Previous studies in ALSA can be categorized into two primary approaches. The first utilizes recurrent neural networks (RNNs), which are designed to handle sequential data by maintaining a memory of previous inputs through hidden states. Many RNN models are augmented with attention mechanisms that enable the model to focus on specific parts of the input sequence during predictions [15,16]. The second approach employs convolutional neural networks (CNNs), which are effective for processing data with grid-like structures, such as images and sequences. CNNs use convolutional layers to automatically learn hierarchical features from the input data [17,18,19].

Both techniques have limitations when applied to aspect-level sentiment analysis. While RNNs are well suited for handling sequential data such as text and can incorporate attention mechanisms to emphasize the relevant sections of the input sequence, these models can be computationally intensive. They require substantial data and resources and are prone to overfitting, particularly when trained on limited or noisy datasets. Furthermore, RNNs may struggle to capture long-range syntactic dependencies between contextual words and the aspects of sentences, leading to potentially inaccurate sentiment interpretations.

An example of an RNN limitation is evident in the sentence “The service was the only thing good about this restaurant” as illustrated in Figure 1a. In this case, the aspect “restaurant” does not convey a clear sentiment. Instead, RNN models equipped with attention mechanisms may mistakenly focus on the syntactically relevant contextual word “good”, which actually relates to “service”. This misalignment in focus can result in inaccurate sentiment analysis; for instance, the attention mechanism might prioritize the positive sentiment associated with “good”, while overlooking the overall negative sentiment implied toward “restaurant”.

CNNs have been employed to process structured data such as text and images. However, a significant limitation of CNNs in ALSA is their inability to account for the sequence in which words or phrases occur within the input data. This limitation can hinder the accurate detection of aspect sentiments, particularly when the context or word order is critical. Additionally, existing neural network models [20,21] that consider syntactic relationships between aspects and contexts often overlook the contextual and syntactic relationships among the words themselves. As illustrated in Figure 1b, the phrase “the servers behind the counter are always friendly” features multiple aspect words: “servers” and “counter”. An effective ALSA model should accurately identify the positive sentiment associated with “servers”, which reflects the overall quality of service, while recognizing that “counter” serves merely as a contextual reference rather than an aspect with emotional weight. Even in phrases consisting of multiple words, existing models may struggle to pinpoint the most influential words that significantly impact the sentiment related to a particular aspect.

Graph Neural Networks (GCNs) [22] are specialized neural networks designed for processing graph-structured data. These networks excel at capturing intricate relationships among nodes by propagating features along the edges of the graph, allowing them to learn both the graph structure and the attributes of its nodes concurrently. GCNs have proven effective in various applications, including social network analysis and recommendation systems. This study introduces a novel model, the Specific-aspect and Inter-aspect Graph Convolutional Network (SI-GCN), aimed at addressing the aforementioned challenges in aspect-level sentiment analysis. The SI-GCN leverages external knowledge to enhance performance, incorporating a bidirectional long short-term memory network (Bi-LSTM) to capture contextual information before processing graph-structured data with a two-layer GCN that encompasses context, syntax, and knowledge graphs. This two-layer structure is designed to capture both specific aspects and multifaceted features through the Specific-GCN and Inter-GCN, respectively. Additionally, the SI-GCN features a module that fuses information between the Specific-aspect and Inter-aspect modules using dual affine processing. Finally, a search-based attention mechanism extracts the relevant emotional features from the contextual representations of specific aspects. In summary, the primary innovations of the SI-GCN model lie in its integration of external knowledge and its effective capture of specific-aspect and inter-aspect dependencies through the use of GCN and Bi-LSTM technologies.

To validate the effectiveness of the proposed SI-GCN model, we conducted comprehensive experiments on four widely used benchmark datasets: Restaurants14, Laptop14, Restaurants15, and Restaurants16. The SI-GCN model consistently outperformed existing baseline models in terms of accuracy and F1 score. For instance, on the Restaurants14 dataset, SI-GCN achieved an accuracy of 83.75% and an F1 score of 76.04%, surpassing the best baseline model, InterGCN, by 1.52% and 2.03%, respectively. On the Laptop14 dataset, SI-GCN improved accuracy by 3.07% compared to InterGCN. These statistical comparisons highlight the SI-GCN model’s superior capability in capturing aspect-level sentiment features, demonstrating its robustness and effectiveness in aspect-level sentiment analysis.

• We propose the SI-GCN, a novel model for aspect-level sentiment analysis that integrates syntactic dependencies, contextual information, and commonsense knowledge within a unified GCN framework. This integrated approach enables the SI-GCN to effectively capture sentiments associated with aspects that lack explicit emotional expressions, addressing the limitations of previous methods.
• We introduce a dual affine attention mechanism in the SI-GCN that combines specific-aspect and inter-aspect emotional features. This mechanism allows the model to capture both the direct sentiment linked to a particular aspect and the influence of other aspects within the same context, resulting in a more nuanced sentiment analysis compared to prior models that treat aspects in isolation.
• Extensive experiments on four standard datasets demonstrate that the SI-GCN significantly outperforms existing baseline models in terms of accuracy and F1 score. Its superior performance, particularly in handling multi-word aspects without explicit emotional cues, validates the effectiveness and generalizability of our model in aspect-level sentiment analysis.

The remainder of the paper is organized as follows: Section 2 reviews the existing literature on aspect-level sentiment analysis, highlighting key approaches and their limitations. Section 3 outlines the specific problem addressed by our proposed SI-GCN model, emphasizing the challenges of analyzing multi-word aspects and inter-aspect relationships in complex sentences. Section 4 provides a detailed presentation of the SI-GCN model, including its architecture, key components, and the integration of various information sources such as syntactic dependencies, contextual cues, and external knowledge. In Section 5, we describe the experimental setup, covering the datasets used, evaluation metrics, and baseline models. The results are analyzed in Section 5.4, demonstrating the effectiveness and superiority of the SI-GCN model. Section 5.5 conducts ablation studies to evaluate the contributions of individual modules within the model. Section 5.6 discusses the impact of various factors on the model’s performance, including the number of GCN layers and the handling of multi-aspect sentences. Finally, Section 5.7 presents case studies that provide deeper insights into the model’s capabilities in managing complex scenarios. The paper concludes with Section 6, summarizing the key contributions and proposing potential avenues for future research.

2. Related Work

This section is dedicated to delivering an in-depth summary of the latest advancements and patterns in ALSA and GCN models. Section 2.1 focuses on aspect-level sentiment approaches, which have been the subject of significant research interest in recent years. Section 2.2 presents GCN, a technique that is gaining widespread use across various applications. By examining the most current studies and tendencies in these areas, this section aims to provide a strong foundation for the proposal model below.

2.1. Aspect-Level Sentiment Analysis

Aspect-level sentiment analysis (ALSA) focuses on identifying and classifying sentiments directed at specific aspects or features of entities within a text. This natural language processing (NLP) task involves a detailed examination of the text to comprehend the opinions, attitudes, and emotions expressed by the author or speaker. For instance, in the sentence “I actually enjoy the camera on this phone, but the battery life is terrible”, an ALSA model would recognize a positive sentiment toward the “camera” and a negative sentiment regarding the “battery life”. This nuanced analysis enhances the understanding of consumer preferences and aversions, thereby highlighting specific areas for improvement.

Early methods in ALSA predominantly utilized conventional machine learning techniques [23,24,25,26]. These approaches, such as bag-of-words and affective lexical features, aimed to capture the rich content and syntactic structures within sentences. However, they were heavily dependent on handcrafted features and necessitated extensive amounts of manually annotated data, rendering them time-consuming and costly to develop [27]. Furthermore, these early models often struggled to capture dependencies between specific aspects and their contexts, resulting in poor generalization performance and brittleness. As a result, there has been growing interest in developing more sophisticated frameworks for ALSA.

The emergence of neural networks has significantly transformed ALSA by facilitating the autonomous learning of distributed representations from contextual text. These models excel at recognizing complex patterns and understanding the relationships between words and phrases, closely mimicking human comprehension. Moreover, neural networks can process vast volumes of unstructured and unlabeled data, making them more adaptable than traditional methods. The integration of attention mechanisms further enhances their ability to capture the semantic connections between aspects and their contexts. For instance, Wang et al. [28] pioneered the use of attention mechanisms within a long short-term memory (LSTM) framework, introducing target embeddings that have influenced numerous subsequent attention-based models. Similarly, Tang et al. [29] developed an attention-dependent memory network that retains contextually relevant words, while Ma et al. [6] introduced an interactive attention network (IAN) employing dual attention mechanisms to identify crucial words in both object representation and broader context.

Integrating syntactic knowledge provides valuable insights into the structure of language, enhancing ALSA. Syntax rules that govern sentence construction can reveal relationships between words and phrases, thus improving sentiment categorization. For example, by utilizing syntactic dependencies, models can identify sentiment-bearing words and their pertinent relationships. Syntactic embeddings—continuous vector representations of syntactic structures derived from extensive text corpora—can also be incorporated into neural models. He et al. [30] utilized an attention framework that integrates syntactic features to determine the significance assigned to each word within a sentence. Similarly, Phan et al. [31] harnessed syntactic information for aspectual terminology and sentiment classification, leveraging syntactic proximity to mitigate noise from extraneous words. However, the absence of long-range dependencies in these models limits their effectiveness in capturing inter-aspect relationships. Pang et al. [32] introduced a multi-channel dynamic graph convolutional network (DM-GCN) tailored for sentences lacking significant syntactic structure. While DM-GCN fuses syntactic and semantic information to learn relevant features, the lack of syntactic constraints and long-range dependencies can lead to the recognition of irrelevant contextual information. To address these deficiencies, Zhang et al. [20] introduced a sentence dependency tree-based GCN that captures syntactic properties and word dependencies; however, their approach still neglects the influences of various aspects. Building on this, Liang et al. [33] enhanced the framework by integrating both syntactic and contextual information to consider the correlations among specific aspects and other related elements.

Integrating commonsense knowledge has been shown to enhance models’ understanding of textual context and meaning, resulting in more accurate predictions. For instance, a model informed about typical smartphone features can better interpret the sentiments expressed in related reviews. Recent studies [34,35] have investigated how commonsense knowledge can be incorporated into ALSA tasks. Notably, Yao et al. [36] incorporated emotion-oriented commonsense information into the training of deep neural networks for ALSA. While some methods [37] have combined syntactic information with external commonsense knowledge, they often overlook critical contextual elements. Zhong et al. [38] employed a multi-perspective approach, blending context, syntax, and knowledge into a knowledge graph model; however, this approach struggles to adequately focus on specific multi-word aspects or complex sentences. Our convolutional network model addresses these limitations by leveraging insights from previous research while prioritizing specific aspects and their interrelationships. It incorporates external information to construct syntax and dependency trees, enabling the recognition of aspect-centric versus inter-aspect affective relations from diverse perspectives.

The importance of context in interpreting sentiments related to specific aspects cannot be overstated. Contextual information encompasses not only the immediate words surrounding an aspect but also broader textual and situational factors. For instance, in the review, “The food was great, but the service was slow”, a positive sentiment is associated with “food”, while a negative sentiment pertains to “service”. Ignoring these contextual nuances can lead to inaccurate generalizations. Recent studies have underscored the necessity of integrating context into sentiment analysis models to enhance their accuracy and reliability. Durga et al. [39] developed a hybrid model that combines deep learning with ant colony optimization, highlighting the role of context in aspect-based sentiment classification, particularly for drug reviews. Xu et al. [40] introduced SE-GCN, which improves aspect-level sentiment prediction by incorporating grammatical context, demonstrating the critical impact of contextual information. Yuan et al. [41] proposed a multi-hop information model that emphasizes deeper contextual processing, effectively addressing complex multidimensional contexts in sentiment analysis. Zhao et al. [42] employed structured dependency trees with GCNs to enhance the model’s capacity for interpreting various contextual elements, reinforcing the importance of structural context in improving classification accuracy. Finally, Sun et al. [43] investigated a dual attention network approach that dynamically integrates contextual features for aspect-level sentiment classification. Their findings illustrate that effectively incorporating contextual considerations significantly enhances classification reliability. Collectively, these studies affirm that integrating contextual information is essential for advancing the accuracy and reliability of sentiment analysis models.

Despite the advancements noted above, significant gaps persist in the current research, particularly in addressing multi-word aspects and effectively modeling inter-aspect relationships. These shortcomings motivate our study, which focuses on developing the Specific-aspect and Inter-aspect GCN to fill these gaps and improve the accuracy of aspect-level sentiment analysis.

The proposed SI-GCN model advances previous research by integrating commonsense knowledge, contextual awareness, and syntactic structure. Our work represents a significant enhancement in the understanding and analysis of aspect-level sentiment, providing a robust framework for addressing the complexities inherent in sentiment analysis tasks.

2.2. Graph Convolutional Network

Graph Neural Networks (GNNs) are a powerful class of models specifically designed to process graph-structured data, effectively capturing both local and global dependencies. They have demonstrated exceptional performance in tasks requiring the representation of relational structures, such as social networks, molecular graphs, and knowledge graphs. Among the various GNN architectures, the graph convolutional network (GCN), introduced by Kipf and Welling [22], is particularly notable. GCNs leverage convolutional operations to propagate information across graph nodes, allowing each node to aggregate features from its neighbors and develop enriched representations.

GCNs have shown remarkable potential in tackling NLP tasks by effectively capturing intricate relationships between linguistic entities. They have been applied to various domains, including semantic role labeling [44], relationship extraction [37], text analysis [45], and machine translation [46]. By exploiting syntactic and semantic dependencies embedded within graph structures, GCNs enhance the modeling of linguistic subtleties, thereby improving performance across these tasks.

ALSA is a fine-grained sentiment classification task that aims to identify and categorize sentiment related to specific aspects within text. Recent research has underscored the effectiveness of GCNs in addressing ALSA, as they are particularly adept at leveraging syntactic structures and contextual information to enhance sentiment understanding. Yao et al. [47] introduced a heterogeneous text graph model that captures co-occurrence patterns and semantic relationships between words and documents. By applying GCNs to this graph, their study demonstrated notable improvements in text classification performance, highlighting the capability of GCNs to encode heterogeneous information. Liang et al. [33] proposed a GCN-based framework that incorporates syntactic knowledge and word dependencies derived from sentence dependency trees. This method enables the model to extract aspect-specific features, leading to more accurate ALSA predictions. Similarly, Sun et al. [48] developed a GCN model utilizing sentence dependency trees to enhance feature representation. By integrating these syntactic features with Bi-LSTM embeddings, their approach achieved superior results in aspect-based sentiment analysis tasks.

Recent advancements have significantly enhanced the capabilities of GCNs for ALSA through the introduction of innovative strategies:

• Zhao et al. [49] proposed a GCN framework that incorporates a bidirectional attentional mechanism. By leveraging positional information, their model effectively captures aspect-specific sentiment signals, demonstrating improved performance in sentiment analysis tasks.
• Global and Local Dependency Guided GCNs (GL-GCN): Zhu et al. [50] introduced the GL-GCN, a novel architecture that integrates both global and local dependency information. This approach effectively captures structural characteristics, emphasizing the importance of combining different dependency types for a more comprehensive understanding of sentiment.

Despite these advancements, research on GCNs for ALSA still faces notable limitations. Most existing studies concentrate on isolated improvements in specific aspects, such as syntax or context, rather than addressing the comprehensive integration of syntactic, semantic, and contextual graphs. Additionally, the heavy reliance on predefined dependency structures restricts the adaptability of these models to diverse linguistic inputs, limiting their flexibility and generalization capabilities.

To bridge these gaps, this study proposes a novel GCN framework that integrates syntactic, contextual, and semantic graphs for ALSA. By unifying these three essential components, the framework offers a more holistic understanding of aspect-level sentiment. This integrated approach not only overcomes the limitations of previous models but also provides a robust mechanism to capture the dynamic interplay between syntax, semantics, and context. Experimental results validate the effectiveness of the proposed framework, highlighting its potential to enhance ALSA performance and advance the field.

3. Problem Definition

In aspect-level sentiment analysis (ALSA), the primary objective is to determine the sentiment expressed toward specific aspects of an entity within a given text. Unlike general sentiment analysis, which assesses the overall sentiment of an entire text, ALSA operates at a more granular level, identifying and categorizing sentiments related to particular aspects explicitly mentioned in the text. This finer focus is especially valuable in applications such as product reviews, customer feedback, and social media analysis, where understanding sentiment toward individual features or attributes is critical for actionable insights.

For example, in a product review, a customer might commend the “camera quality” of a smartphone while criticizing its “battery life”. ALSA enables the differentiation of these contrasting sentiments, allowing for more precise and meaningful analyses. However, this level of detail introduces several challenges, such as managing aspects composed of multiple words (e.g., “customer service”) and addressing sentences with multiple aspects, as in, “The screen resolution is excellent, but the battery drains quickly”. Furthermore, aspects often appear within diverse syntactic and contextual structures, necessitating models that can effectively capture the intricate relationships between aspects, their modifiers, and the surrounding context.

To address these challenges, ALSA is formally defined as a task that operates on a sentence–aspect pair, denoted as $\{S,T\}$, where we have the following:

• $S=\{w_1,w_2,\dots ,w_m\}$ represents a sentence consisting of m words.
• $T=\{w_{start},w_{start+1},\dots ,$$w_{start+n-1}\}$ defines an aspect consisting of n words which form a sub-sequence within sentence S. The “start” index specifies the initial position of the aspect within the sentence, highlighting its contextual boundaries.

Each sentence can contain one or multiple aspects—either as single words or clusters of words. The key objective of ALSA is to ascertain the sentiment polarity $y\in \left\{0,1,2\right\}$ for the sentence S concerning a specified aspect T:

• (0) for neutral;
• (1) for positive;
• (2) for negative.

To illustrate the structured conceptualization provided by the formal definition of ALSA, we present a detailed example demonstrating how ALSA operates in practice. This example highlights how the model deconstructs and analyzes sentiment in complex sentences containing multiple aspects. Consider the sentence S: “The camera quality of this smartphone is excellent, but its battery life is disappointing”. The sentence can be represented as $S=\{w_1:“\mathrm{The}”,w_2:“\mathrm{camera}”,w_3:“\mathrm{quality}”,w_4:“\mathrm{of}”,w_5:“\mathrm{this}”,w_6:“\mathrm{smartphone}”,w_7:“\mathrm{is}”,w_8:“\mathrm{excellent}”,w_9:“\mathrm{but}”,w_{10}:“\mathrm{its}”,w_{11}:“\mathrm{battery}”,w_{12}:“\mathrm{life}”,w_{13}:“\mathrm{is}”,w_{14}:“\mathrm{disappointing}”\}$ This sentence explicitly mentions two distinct aspects that require separate sentiment analyses:

(1)

Aspect 1: camera quality

• Aspect Sub-sequence: $T_1=\{w_2:“\mathrm{camera}”,w_3:“\mathrm{quality}”\}$.
• $start=2$.
• Contextual Analysis: The sentiment toward this aspect is influenced by the phrase “is excellent”, which positively describes the camera quality.
• Sentiment Polarity: $y_1=1$ (Positive).

(2)

Aspect 2: battery life

• Aspect Sub-sequence: $T_2=\{w_{11}:“\mathrm{battery}”,w_{12}:“\mathrm{life}”\}$.
• $start=11$.
• Contextual Analysis: The sentiment toward this aspect is influenced by the phrase “is disappointing”, which negatively describes the battery life.
• Sentiment Polarity: $y_2=2$ (Negative).

By individually analyzing each aspect, ALSA generates two sentence–aspect pairs: $\{S,T_1\}$, classified as $y_1=1$, and $\{S,T_2\}$, classified as $y_2=2$.

4. Methodology

In this section, we present the Specific-aspect and Inter-aspect Graph Convolutional Network (SI-GCN) model, designed to enhance aspect-level sentiment analysis. This study investigates the effectiveness of SI-GCN in addressing key challenges in ALSA, particularly its capacity to manage multi-word aspects and sentiments expressed across multiple aspects within complex sentence structures.

• Research Goals: The primary objectives of this research encompass two aspects. First, we aim to develop an innovative method for ALSA that integrates various data sources, including syntactic dependencies, contextual information, and commonsense knowledge. Second, we seek to evaluate the performance of the SI-GCN model against existing state-of-the-art models within specific sentiment analysis tasks, thereby establishing its efficacy and robustness.
• Theoretical Framework: This study is grounded in the foundational principles of Graph Convolutional Networks (GCNs), while also extending and innovating upon this framework to address the complex challenges of aspect-level sentiment analysis. We introduce two novel variants of GCNs: the Specific-aspect GCN and the Inter-aspect GCN. These models are specifically designed to capture the intricacies of sentiment expression with remarkable precision. The Specific-aspect GCN focuses on isolating and analyzing sentiment related to individual aspects, ensuring that the model’s attention is not diluted by irrelevant information. This specialized GCN employs an advanced graph construction technique that explicitly represents the relationships between aspect-specific features and their corresponding sentiments, thereby enhancing the model’s ability to detect subtle variations in sentiment. Complementing the Specific-aspect GCN is the Inter-aspect GCN, which is designed to capture the interdependencies and interactions among different aspects. This component enables a comprehensive understanding of the sentiment landscape by modeling how the sentiment of one aspect may influence another, effectively addressing a critical limitation of existing GCN-based approaches.

The general structure of the proposed SI-GCN model is illustrated in Figure 2. This section is structured to reflect the hierarchical architecture of the SI-GCN model, progressing from the bottom layer to the output layer. The specific contents are organized as follows.

(1)

Initial Feature Extraction (Section 4.1): The process begins with advanced embedding techniques, utilizing pre-trained models like BERT to generate high-dimensional semantic representations of input text. These embeddings encode rich contextual information, serving as the foundational input for subsequent modules.

(2)

Graph Construction (Section 4.2): The SI-GCN model constructs interconnected graphs that represent diverse linguistic and semantic relationships essential for ALSA tasks. Four graph types are developed:

• Context-Based Graph $G^c$ (Section 4.2.1): This graph highlights interactions between aspect terms and their surrounding context, with connections determined using semantic measures such as cosine similarity and term frequency-inverse document frequency (TF-IDF).
• Syntax-Based Graph $G^s$ (Section 4.2.1): Derived from syntactic dependency parsing, this graph models structural relationships, including subjects, objects, and predicates, quantified using Pointwise Mutual Information (PMI).
• Context–Syntax Graph $G^{cs}$ (Section 4.2.1): By merging the context and syntax graphs, this unified graph captures both semantic and syntactic dependencies, providing a holistic view of textual relationships.
• Knowledge-Based Graph $G^k$ (Section 4.2.2): Integrating external commonsense knowledge from SenticNet6 [51], this graph links textual words to conceptual nodes (e.g., emotions and polarity) to enrich semantic understanding.

(3)

Specific-Aspect Module (Section 4.2.1): The Specific-aspect GCN processes the context–syntax graph $G^{cs}$ to analyze the target aspect in relation to its contextual and syntactic surroundings. Iterative graph convolutions produce node-level representations that encapsulate aspect-specific sentiment information.

(4)

Inter-Aspect Module (Section 4.2.2): Leveraging the knowledge graph $G^k$, the Inter-aspect GCN models interactions among multiple aspects within a sentence. By incorporating commonsense relationships and implicit connections, this module captures relational sentiment dynamics across aspects.

(5)

Mutual Bi-Affine Module (Section 4.3): The outputs from the Specific-aspect GCN and Inter-aspect GCN are refined through the Bi-affine module, which employs dual transformations to enhance feature interactions. Average pooling aggregates the aspect node representations, enabling efficient feature fusion.

(6)

Attention Scores and Prediction (Section 4.4): A retrieval-based attention mechanism dynamically assigns weights to context terms, highlighting the most relevant features associated with aspect terms. The aggregated features are used to refine the aspect-specific input. The final layer applies a softmax function to classify the sentiment polarity (positive, negative, or neutral) for each aspect. This layer leverages the fused representations and attention-enhanced features for precise predictions.

(7)

Comprehensive Execution Steps of the SI-GCN Model (Section 4.5): This section provides a detailed explanation of the SI-GCN algorithm, outlining its comprehensive workflow.

4.1. Initial Feature Extraction

Feature extraction is a fundamental step in any natural language processing task. In our sentiment analysis model, SI-GCN, we employ advanced word embedding techniques and sequence modeling methods to capture both semantic and contextual features from textual data.

4.1.1. Word Embedding: Foundation of Semantic Analysis

In natural language processing, converting words into numerical vectors through word embeddings is a fundamental process. These vectors encode semantic relationships, enabling algorithms to effectively interpret and process textual data. Our SI-GCN model leverages pre-trained embeddings, such as the Global Vectors for Word Representation (GloVe) [52] and the Bidirectional Encoder Representations from Transformers (BERT) [53], to initialize its embedding matrix.

BERT represents a significant advancement in embedding technologies. Unlike traditional models, which process text sequentially in a single direction (either left-to-right or right-to-left), BERT simultaneously analyzes text bidirectionally. This dual perspective is particularly critical in complex linguistic contexts, where the meaning of a word often depends on its surrounding words. When the SI-GCN model processes a sentence, it uses BERT to transform each word into a high-dimensional vector. For example, in the model setup, if a sentence consists of n words, an embedding matrix $X=\{x_1,x_2,\dots ,x_n\}$ is generated, where each $x_i\in {\mathbb{R}}^m$ corresponds to the BERT embedding of word $w_i$. The dimension m of these vectors is predefined by the BERT model, typically containing hundreds of attributes that encode various aspects of the word’s usage in language. The ability of BERT to consider the full context of a sentence (both left and right surroundings of each word) when generating word embeddings sets it apart from earlier models. This feature allows BERT to capture subtleties in meaning that might be lost in models that only consider one direction of text.

During both the training and fine-tuning stages of the SI-GCN model, BERT embeddings play a critical role. They provide a nuanced and contextually enriched representation of words. Specifically, when a sentence pair is provided—comprising a sequence of contexts (s) and aspects (e)—our model transforms this pair into “[CLS]+s+[SEP]” and “[CLS]+e+[SEP]” formats. Here, [CLS] represents a special classification token added at the beginning of the input sequence, and [SEP] is a separator token used to indicate the boundary between different segments of the input. These tokens are integral components of the BERT input formatting. This restructuring is designed to better distinguish between general sentence information and specific aspects, enabling the model to focus on relevant sentiments.

4.1.2. Bi-LSTM: Contextual Embeddings from Sequential Data

The Bidirectional Long Short-Term Memory (Bi-LSTM) [54] model is a refined version of traditional recurrent neural networks, designed to enhance the capture of contextual features and long-range dependencies within text. Bi-LSTM achieves this by integrating two LSTM networks, one reading the sentence forward ($\overrightarrow{LSTM}$) and the other in reverse ($\overleftarrow{LSTM}$), thereby gathering comprehensive context from both directions.

In the SI-GCN model, following the initial feature extraction described in Section 4.1.1, where BERT embeddings provide a deep semantic representation of each word in a sentence, the Bi-LSTM layer takes these embeddings as input. This setup allows the Bi-LSTM network to leverage the rich context-sensitive embeddings produced by BERT:

(1)

Input to Bi-LSTM: The output from the BERT embeddings (Section 4.1.1) serves as the input to our Bi-LSTM network. Each word’s BERT embedding, which encapsulates both the lexical semantics and the contextual nuances thanks to the BERT attention mechanisms, is processed by the Bi-LSTMs to extract even more refined features that consider bidirectional contextual dependencies.

(2)

Operation of Bi-LSTM: The Bi-LSTM processes these embeddings to produce two sets of hidden states: forward hidden states ($\overrightarrow{h_i}$) and backward hidden states ($\overleftarrow{h_i}$). These hidden states of a word are generated as follows:

$$\begin{array}{cc}\hfill & \overrightarrow{h_i}=\overrightarrow{LSTM}\left(x_i,\theta \right),i\in [1,n]\hfill \\ \hfill & \overleftarrow{h_i}=\overleftarrow{LSTM}\left(x_i,\theta \right),i\in [n,1]\hfill \\ \hfill & h_i=[\overrightarrow{h_i};\overleftarrow{h_i}]\hfill \end{array}$$

(1)

where $x_i$ represents the embedded representation of the i-th word from the BERT model, $\theta$ denotes the parameters of the LSTM, and $h_i=[·;·]$ represents the vector concatenate operation. To distinguish the different focuses of feature extraction by Bi-LSTM in our SI-GCN model, we denote the hidden contextual representations derived from Bi-LSTM as $H_{ctx}=\left\{h_c^1,h_c^2,\dots ,h_c^n\right\}$ for context sequences and $H_{asp}=\left\{h_a^1,h_a^2,\dots ,h_a^n\right\}$ for aspect sequences.

In the SI-GCN model, the rich, contextual embeddings extracted by the Bi-LSTM serve as preparatory inputs for further processing layers. Specifically, these embeddings form an excellent foundation for the next stage of our model involving a graph convolutional network (GCN). The Bi-LSTM network outputs a matrix of hidden states $H\in {\mathbb{R}}^{n\times m}$, where n represents the sequence length, and m denotes the dimensions of hidden layers. This structured sequence of hidden states forms a suitable input for the GCN, with each word’s hidden state serving as the feature vector for corresponding nodes in the graph.

4.2. Core Analytical Framework: Graph Convolutional Networks

Graph convolutional networks (GCNs) are highly effective tools in deep learning for modeling relationships within data structured as graphs. A typical graph consists of vertices (or nodes) and edges connecting these vertices. In the context of sentiment analysis, each word in a sentence can be considered a vertex, and the relationships between these words (like syntactic dependencies or semantic associations) form the edges. Specifically, the graph representation in ALSA task can be constructed as follows:

• Vertices: Represent words in the sentences.
• Edges: Depict relationships between words. These could be syntactic, as derived from dependency parsing, or semantic, sourced from domain knowledge or commonsense relationships.
• Adjacency Matrix A: The matrix effectively represents these connections, where $A_{ij}=1$ if there is a relationship between the i-th and j-th word and 0 otherwise.

GCNs utilize the structure of data arranged as graphs to execute convolution operations directly on these graphs. These operations facilitate the capture of both local and global relationships among nodes. As illustrated in Figure 3, a GCN consists of multiple layers that systematically transform the input data to capture complex relationships between nodes. More formally, we examine a regular graph $G=\left(V,E,A\right)$, wherein $V=\{v_1,v_2,\dots ,v_n\}$ and $E=\{e_{ij}\mid i,j\in \{1\dots n\}\}$ represent the sets of nodes and edges, respectively. $A={\{0,1\}}^{n\times n}$ indicates the group adjacency matrix. The convolution process in the GCN can be represented by the equation:

$$H^{(j+1)}=\sigma \left(\widehat{A}{H}^{\left(j\right)}{W}^{\left(j\right)}\right)$$

(2)

where $\widehat{A}=D^{-\frac{1}{2}}A{D}^{-\frac{1}{2}}$ is the normalized symmetric adjacency matrix. D is the degree matrix of A with respect to $D_{ii}={\sum }_j{A}_{ij}$. $H^{\left(j\right)}$ is the hidden state matrix from the j-th layer of the GCN. $W^{\left(j\right)}$ is the weight matrix for the j-th layer. $\sigma$ refers to the Rectified Linear Unit (ReLU) activation function, introducing non-linearity into the model.

The initial input matrix ($H^{\left(0\right)}=H$) typically contains features derived from previous processing stages, such as those generated by Bi-LSTM layers, which capture deep contextual embeddings from BERT. At each layer, the matrix $H^{\left(j\right)}$ is transformed, capturing and integrating the features and relationships represented in the graph. This transformation enables the model to distill higher-level relational attributes as the number of layers increases. This multilayer processing, illustrated in Figure 3, allows the GCN to progressively capture increasingly complex relationships within the data. Each layer applies the convolution operation followed by the ReLU activation function.

Building upon the comprehensive framework established by the generalized GCN structure, we present two specialized adaptations tailored to the requirements of aspect-level sentiment analysis: the Specific-aspect GCN (Specific-GCN) and the Inter-aspect GCN (Inter-GCN). These models are designed to manage specialized graph structures:

(1)

Specific-aspect GCN (Specific-GCN): This GCN employs a context–syntax graph to model the intricate relationships within sentence structures, emphasizing syntactical and contextual interdependencies pertinent to the identified aspects. This enables the Specific-GCN to effectively focus on and identify sentiment polarities associated with particular aspects.

(2)

Inter-aspect GCN (Inter-GCN): This GCN leverages a knowledge graph to integrate extensive semantic relationships and commonsense knowledge, enhancing the contextual understanding required for accurate sentiment interpretation across multiple aspects. This approach enables sentiment analysis to go beyond the surface-level text, incorporating inferred meanings and relational knowledge for a deeper understanding.

These models offer an integrated framework that combines syntax, context, and knowledge, demonstrating significant effectiveness in capturing subtle sentiment polarities for both individual and multiple aspects within complex textual data. This overview sets the stage for the detailed analyses presented in Section 4.2.1 and Section 4.2.2, respectively.

4.2.1. Specific-Aspect Module

The Specific-aspect module forms a key element of our approach, incorporating two foundational graphs: a context-based graph $G^c$ and a syntax-based graph $G^s$. The context graph leverages contextual information to highlight the relationships between the target word and its surrounding words. Meanwhile, the syntax graph is derived from the syntactic dependency tree, representing the structural relationships between the target word and other words in the sentence. These graphs are merged to create a context–syntax graph $G^{cs}$, which combines contextual and syntactic information to extract aspect-specific details. The integrated graph $G^{cs}$ serves as input for the Specific-aspect GCN, enabling the model to analyze the intricate relationships captured within the synthesized graph.

(1)

Construction of context-based graph: $G^c$.

Leveraging insights from existing research [55], our methodology introduces a sophisticated context-based graph, $G^c$, designed specifically to refine sentiment prediction for targeted aspects. This graph diverges from conventional approaches by not solely focusing on the aspect word and its direct neighbors. Instead, it establishes a broader network that captures intricate relationships between the aspect word and its wider linguistic environment, including both surrounding words and entire sentences.

The context-based graph, denoted as $G^c=\left(V^c,E^c,A^c\right)$, consists of the following components:

• Vertices: $V^c$: These represent not only individual words within a sentence but also the sentence as a whole, thereby enhancing contextual understanding.
• Edges: $E^c$: These connections span all vertices to capture the diverse relationships that may influence sentiment.
• Adjacency Matrix: $A^c$: This matrix is essential for representing the intricate connections within the graph, constructed based on varying degrees of contextual association among nodes.

The specific relationships are quantified within the adjacency matrix $A^c$ as follows:

$$A_{ij}^c=\left\{\begin{array}{cc}error\theta ,\hfill & \mathrm{if}{v}_{ij}\in S\wedge v_i^c\ne v_j^c\hfill \\ TF-ID{F}_{ij},\hfill & \mathrm{if}{v}_i^c\in S\wedge v_j^c\in W\hfill \\ 1,\hfill & \mathrm{if}{v}_i^c=v_j^c\hfill \\ 0,\hfill & \mathrm{otherwise}\hfill \end{array}\right.$$

(3)

where $cos\theta$ represents the cosine similarity between sentences, emphasizing relational weights based on semantic closeness. $TF-ID{F}_{ij}$ indicates the term frequency-inverse document frequency, a statistical measure used to evaluate how important a word is to a sentence within the corpus. S denotes the entire set of sentences being analyzed, and W represents the aggregate set of words. The value of $cos\theta$ can be calculated as Equation (4):

$$cos\theta =\frac{{\sum }_{i=1}^n\left(s_i\times s_j\right)}{\sqrt{{\sum }_{i=1}^n{\left(s_i\right)}^2}\times \sqrt{{\sum }_{i=1}^n{\left(s_j\right)}^2}}$$

(4)

For example, consider the application of $G^c$ in analyzing the phrase “the service was the only thing good about this restaurant”. The adjacency matrix of the context-based graph can be shown in Figure 4. In this graph, the nodes include each distinct word as well as the entire sentences. Edges connect related words and sentences, with weights assigned based on their semantic proximity and contextual relevance. This graphical structure enables our model to discern not only the explicit sentiments associated with “service” and “menu” but also how these sentiments are influenced by their surrounding textual contexts. By introducing a comprehensive network of connections among aspect words, surrounding words, and entire sentences, this approach can ensure a richer, more nuanced analysis of sentiment.

(2)

Construction of Syntax-based Graph: $G^s$.

Our approach introduces the syntax-based graph, $G^s$, for the syntax branch of our model, which is essential for parsing the structural relationships between words within a sentence. This graph leverages the advanced capabilities of the SpaCy toolkit (https://spacy.io/ (accessed on 12 August 2024)) to construct detailed syntactic dependency trees, forming the backbone of our analysis.

The syntax-based graph, denoted as $G^s=\left(V^s,E^s,A^s\right)$, consists of the following components:

• Vertices: $V^s$: Represent all words in the sentence, encapsulating each word as a discrete node within the graph.
• Edges: $E^s$: These are formed based on syntactic dependencies identified between words, such as nominal subjects (nsubj), objects (pobj), and determiners (det).
• Adjacency Matrix: $A^s$: This matrix captures the presence and strength of syntactic connections between word pairs within the sentence.

This study improves the syntactic graph proposed in [37]. Specifically, the strength of these connections is quantified using Pointwise Mutual Information (PMI), which measures the association strength between pairs of words based on their co-occurrence patterns relative to individual occurrences:

$$A_{ij}^s=\left\{\begin{array}{cc}PMI\left(v_i^s,v_j^s\right),\hfill & \mathrm{if}{v}_i^s,v_j^s\mathrm{are}\mathrm{words},PMI(i,j)>0\hfill \\ 1,\hfill & \mathrm{if}{v}_i^s,v_j^s\in W\wedge v_i^s=v_j^s\hfill \\ 0,\hfill & \mathrm{otherwise}\hfill \end{array}\right.$$

(5)

$PMI$ is calculated as follows, providing a statistical basis for understanding word relationships within the syntax graph:

$$PMI(i,j)=log\frac{P(i,j)}{P\left(i\right)P\left(j\right)}$$

(6)

Here $P(i,j)$ represents the joint probability of words i and j occurring together, while $P\left(i\right)$ and $P\left(j\right)$ are the probabilities of words i and j occurring independently.

Figure 5 illustrates the adjacency matrix generated by the syntax-based graph $G^s$ proposed in our syntax branch for setting up the sentence “The servers behind the counter are always friendly”. Our graph $G^s$ delineates the dependency roles connecting ’servers’ as the subject linked to the predicate ’are’. This model facilitates nuanced insight into how words structurally interact within sentences, significantly enhancing our semantic analysis.

(3)

Synthesize: $G^c$ and $G^s$ to construct context and syntax graph: $G^{cs}$.

The integration of context and syntax into a unified graph, denoted as $G^{cs}=\left(V^{cs},E^{cs},A^{cs}\right)$, represents a significant methodological advancement in capturing the nuanced interplay between syntactic structures and contextual semantics within textual data. This integration fundamentally enhances the model’s ability to analyze and interpret complex linguistic elements cohesively. The structural composition of $G^{cs}$ can be represented as follows:

• Vertices: $V^{cs}$: The vertices consist of the union of the nodes from both the context-based and syntax-based graphs $V^{cs}=V^c\cup V^s$, incorporating all words and sentence-level entities. This allows $G^{cs}$ to leverage both contextual and syntactic properties of the text.
• Edges: $E^{cs}$: The edge set combines relationships defined in both $E^c$ and $E^s$, i.e., $E^{cs}=E^c\cup E^s$, facilitating a comprehensive representation that incorporates both types of linguistic connections—contextual and syntactic.
• Adjacency Matrix: $A^{cs}$: The adjacency matrix for $G^{cs}$ is a fusion of the matrices from $G^c$ and $G^s$, where the interactions between nodes are based on multiple dimensions of textual relationships:

  $$A_{ij}^{cs}=\left\{\begin{array}{cc}error\theta ,\hfill & \mathrm{if}{v}_{ij}\in S\wedge v_i^c\ne v_j^c\hfill \\ TF-ID{F}_{ij},\hfill & \mathrm{if}{v}_i^c\in S\wedge v_j^c\in W\hfill \\ PMI\left(v_i^s, v_j^s\right),\hfill & \mathrm{if}{v}_i^s,v_j^s\mathrm{are}\mathrm{words},\mathit{PMI}(\mathit{i},\mathit{j})>0\hfill \\ 1,\hfill & \mathrm{if}{v}_i^c=v_j^c\hfill \\ 1,\hfill & \mathrm{if}{v}_i^s,v_j^s\in W\wedge v_i^s=v_j^s\hfill \\ 0,\hfill & \mathrm{otherwise}\hfill \end{array}\right.$$

  (7)

Combining these metrics allows $G^{cs}$ to robustly model not just isolated syntactic or semantic patterns but their convergence, enabling deeper insights into how context influences syntactic structures and vice versa. Figure 6 illustrates such integration, where nodes represent both contextually and syntactically significant elements, and edges embody the comprehensive linguistic relationships. This synthesized graph structure empowers our analytic models to perform more nuanced and contextually aware analyses.

(4)

Implementation of Specific-aspect GCN.

After combining the context graph $G^c$ and the syntax graph $G^s$, the resulting integrated context–syntax graph $G^{cs}$ is processed by the GCN module, referred to as the Specific-aspect GCN. This module is essential for advancing ALSA, as it systematically examines the intricate relationships represented within the synthesized graph. The transformation of node features within the Specific-aspect GCN is achieved using a graph convolution operation, mathematically expressed as

$$H^{cs(j+1)}=\sigma \left({\widehat{A}}^{cs}{H}^{cs\left(j\right)}{W}^{cs\left(j\right)}\right).$$

(8)

where ${\widehat{A}}^{cs}=D^{cs-\frac{1}{2}}{A}^{cs}{D}^{cs-\frac{1}{2}}$ represents the normalized symmetric adjacency matrix that ensures an equitable propagation of node features by mitigating the influence of node degree variations. $H^{cs\left(j\right)}$ denotes the hidden states at the j-th layer, capturing the interim representations of aspects as influenced by their syntactic and contextual relationships. $W^{cs\left(j\right)}$ is the weight matrix specific to the j-th layer, tasked with transforming node features into higher-level representations poignant for sentiment determination. $\sigma$ stands for the activation function ReLU which introduces the non-linearity necessary for capturing complex patterns within data.

The final output $Z^{cs}$ from the Specific-aspect GCN is represented as $H^{cs\left(l\right)}$, where l indicates the depth of the network. It captures comprehensive node-level outputs that represent aspect-specific sentiments, derived from the integrated analysis of both syntactic and contextual information. The running process of the Specific-aspect GCN module is described in Algorithm 1.

Algorithm 1 Running process of Specific-aspect GCN via context–syntax graph.

Require: A sentence–aspect pair $(S,T)$ Ensure: The output $Z^{cs}$ of Specific-aspect GCN 1: for $i=1\leftarrow n$; $j=1\leftarrow n$ do 2:    ▹ Construct a Contextual adjacency matrix $A_{ij}^c$ 3:    if $v_{ij}\in S\wedge v_i^c\ne v_j^c$ then 4:        $A_{ij}^c\leftarrow cos\theta$ 5:    else if $v_i^c\in S\wedge v_j^c\in W$ then 6:        $A_{ij}^c\leftarrow TF-ID{F}_{ij}$ 7:    else if $v_i^c=v_j^c$ then 8:        $A_{ij}^c\leftarrow 1$ 9:    else 10:        $A_{ij}^c\leftarrow 0$ 11:    end if 12:    ▹ Construct a Syntax adjacency matrix $A_{ij}^s$ 13:    if $v_i^s,v_j^s\mathrm{are}\mathrm{words},PMI(i,j)>0$ then 14:        $A_{ij}^S\leftarrow PMI\left(v_i^s,v_j^s\right)$ 15:    else if $v_i^s,v_j^s\in W\wedge v_i^S=v_j^s$ then 16:        $A_{ij}^s\leftarrow 1$ 17:    else 18:        $A_{ij}^s\leftarrow 0$ 19:    end if 20:    ▹ Construct a context–syntax adjacency matrix $A_{ij}^{cs}$ 21:    Similar to the above, we obtain $A_{ij}^{cs}$ by Equation (7) 22:    Finally, $Z^{cs}$ is obtained from $A_{ij}^{cs}$ via Equation (8) 23:end for

4.2.2. Inter-Aspect Module

The Specific-aspect module is designed to handle sentences with a single aspect term; however, its effectiveness may decline when processing sentences containing multiple aspect terms. In such cases, accurately determining the sentiment orientation for a particular aspect may require considering the sentiment orientation of other aspects. To address this limitation, we developed an Inter-aspect interaction module. This module incorporates commonsense knowledge as a knowledge source and combines it with the contextual and syntactic information derived from the Specific-aspect module to interactively extract relationships between aspects. The Inter-aspect module employs a GCN framework that operates on a graph representation of the sentence, where each aspect term is represented as a vertex in the graph.

(1)

Construction of Knowledge-based Graph: $G^k$.

To augment our model with commonsense knowledge, we incorporate the SenticNet6 knowledge base [51], which is structured into three layers: primitive, concept, and entity levels. This extensive resource encompasses over 100,000 concepts along with their associated emotional attributes, offering a wealth of semantic information such as emotion, polarity, similarity, opposition, and subordination. Leveraging this knowledge base, we employ the SenticNet sentiment dictionary to extract parameters related to the key words in our sentences. Figure 7 demonstrates this process using example words like “good” and “restaurant”, from which the respective parameters are derived.

The SenticNet6 sentiment dictionary provides several parameters associated with words, which can be utilized to determine their emotional attributes. These parameters include the following:

• Polarity_label: This parameter indicates the sentiment classification of a word as positive, negative, or neutral, with several subcategories for each label.
• Polarity_value: This parameter quantifies the emotional intensity or polarity of a word, typically represented as a specific value. Higher values correspond to greater emotional strength.
• Sentics: This parameter offers fine-grained sentiment analysis, encompassing dimensions such as introspection, sensitivity, attitude, and temper.
• Semantics: This parameter identifies words with semantic relationships to the specified term. For example, the word “good” is semantically related to “significant”, among others.

To enable advanced sentiment analysis across multiple aspects in texts, our model incorporates a knowledge graph represented by an adjacency matrix $G^k=\left(V^k,E^k,A^k\right)$. This graph is specifically designed to capture the complex relationships and semantic connections between different aspects, utilizing the computational capabilities of GCNs. The knowledge-based graph $G^k$ consists of the following components:

• Vertices: $V^k$: The graph’s vertices include two primary types of nodes: word nodes and knowledge nodes. Word nodes represent the actual words or terms within the analyzed text. Knowledge nodes represent conceptual or categorical information related to the words, derived from extensive knowledge bases though SenticNet. They encapsulate broader semantic and relational data that enrich the contextual understanding of words.
• Edges: $E^k$: The edge set includes connections between all pairs of nodes, regardless of their type. These edges play a vital role in capturing both explicit and implicit relationships.
• Adjacency Matrix: $A^k$: This matrix forms the core of the graph’s structural definition and is detailed by the following rules:

  $$A_{ij}^k=\left\{\begin{array}{cc}1,\hfill & \mathrm{if}{v}_i^k\in S\wedge v_j^k\in \mathrm{Knowledge}\mathrm{node},\hfill \\ 1,\hfill & \mathrm{if}{v}_i^k,v_j^k\in \mathrm{Knowledge}\mathrm{node}\hfill \\ 1,\hfill & \mathrm{if}{v}_i^k=v_j^k\hfill \\ 0,\hfill & \mathrm{otherwise}\hfill \end{array}\right.$$

  (9)

Figure 8 illustrates the knowledge-based graph constructed as part of our model’s analysis process. The graph consists of nodes representing both textual words (e.g., “service”, “good”, and “restaurant”) and conceptual knowledge derived from SenticNet6 (e.g., “importance” and “happy”). These nodes are connected by edges that encode the relationships between the words and their corresponding semantic or emotional attributes. In the adjacency matrix shown in Figure 8, each cell indicates whether a connection exists between a pair of nodes. A value of 1 represents the presence of an edge, while a blank indicates no connection. For instance, “good” is linked to conceptual knowledge node “importance”, which is extracted from SenticNet6 and provides deeper insights into its semantic and emotional attributes. Similarly, “restaurant” shares connections with “happy”, highlighting the positive sentiment associated with this aspect. This graph demonstrates hsow our model integrates textual data and commonsense knowledge to build a comprehensive representation of semantic and emotional relationships.

(2)

Implementation of Inter-aspect GCN.

After constructing the contextual knowledge graph $G^k$, it is essential to utilize its structured data within the GCN model, referred to as the Inter-aspect GCN. This model employs GCNs to effectively process and analyze the complex interrelationships represented in $G^k$. Node representations within the Inter-aspect GCN are computed iteratively across successive layers using the following operational formula:

$$H^k(j+1)=\sigma \left({\widehat{A}}^k{H}^{k\left(j\right)}{W}^{k\left(j\right)}\right).$$

(10)

In this equation, ${\widehat{A}}^k=D^{k-\frac{1}{2}}{A}^k{D}^{k-\frac{1}{2}}$ represents the normalized symmetric adjacency matrix, ensuring that the convolution process distributes influence evenly among nodes regardless of their degree. $H^{k\left(j\right)}$ denotes the node features at layer j, encapsulating the evolving semantic and relational information of each node within the graph. $W^{k\left(j\right)}$ is the weight matrix for the j-th GCN layer, responsible for transforming node features between layers. The non-linear activation function $\sigma$, specifically ReLU, introduces non-linearity into the feature transformation process, enabling the model to capture complex patterns effectively.

The matrix $D^k$ serves as the degree matrix for $A^k$, with each diagonal element $D_{ii}^k$ computed as the sum of the connections for node i (${\sum }_j{A}_{ij}^k$). This component is crucial for normalizing the influence of each node based on its connectivity. After processing through l layers, the output of the Inter-aspect GCN, denoted as $Z^k=H^{k\left(l\right)}$, encapsulates refined node-level outputs, capturing comprehensive sentiment and semantic information shaped by both direct and inferred semantic relationships. Using Equation (2), the representations of contextual and syntactic structures, expressed as $Z^k=\left\{h_1^k,h_2^k,...,h_n^k\right\}$, are extracted from the model, providing nuanced insights into the aspects’ characteristics and interrelations. The hidden representations for specific aspect nodes, denoted as $\left\{h_{a1}^k,h_{a2}^k,...,h_{an}^k\right\}$, represent detailed sentiment features associated with each respective aspect. The process of the Inter-aspect module is described in detail in Algorithm 2.

Algorithm 2 Running process of Inter-aspect GCN via knowledge-based graph.

Require: A sentence–aspect pair $(S,T)$ Ensure: The output $Z^k$ of Inter-aspect GCN 1: for $i=1\leftarrow n$; $j=1\leftarrow n$ do 2:    ▹ Construct the adjacency matrix $A_{ij}^k$ of a knowledge graph 3:    if $v_i^k\in S\wedge v_j^k$ then 4:        $A_{ij}^k\leftarrow 1$ 5:    else if $v_i^k,v_j^k\in \mathrm{Knowledge}\mathrm{node}$ then 6:        $A_{ij}^k\leftarrow 1$ 7:    else if $v_i^k=v_j^k$ then 8:        $A_{ij}^k\leftarrow 1$ 9:    else 10:        $A_{ij}^k\leftarrow 0$ 12:    Finally, $Z^k$ is obtained from $A_{ij}^k$ via Equation (10) 13: end for

4.3. Mutual Bi-Affine Module: Enhancing Feature Interactions

The Bi-affine mechanism [56] is an advanced attention mechanism designed to enhance interactions between different components within a neural network framework. This mechanism utilizes a Bi-affine transformation, enabling the model to capture complex relationships among input features more effectively. By integrating this approach, the Bi-affine mechanism improves the model’s focus on relevant information during predictions. Its effectiveness has been demonstrated in syntactic dependency parsing [56]. In this study, we employ the Bi-affine module to extract interactive features between the Specific-aspect module and Inter-aspect module. To minimize noise and redundant information, average pooling is applied to process the output from the Bi-affine module.

In Figure 9, the structure of the Bi-affine module is depicted, illustrating the interactions between the outputs of the Specific-aspect GCN and Inter-aspect GCN modules. The central part of the figure illustrates the core processes of the Mutual Bi-affine Module. Specifically, the outputs from the Specific-aspect GCN, denoted as $Z^{\mathrm{cs}}$, and the outputs from the Inter-aspect GCN, represented as $Z^{\mathrm{k}}$, are fed into the Bi-affine transformation and produce the Bi-affine module’s output $Z^{\mathrm{cs}\prime }$ and $Z^{\mathrm{k}\prime }$. The equations governing this interaction are expressed as

$$Z^{\mathrm{cs}\prime }=softmax\left(Z^{\mathrm{cs}}{W}_1{\left(Z^{\mathrm{k}}\right)}^{\mathrm{T}}\right)Z^{\mathrm{k}}$$

(11)

$$Z^{\mathrm{k}\prime }=softmax\left(Z^{\mathrm{k}}{W}_2{\left(Z^{\mathrm{cs}}\right)}^{\mathrm{T}}\right)Z^{\mathrm{cs}}$$

(12)

where $W_1$ and $W_2$ represent the learnable parameters facilitating the transformation of feature data, enabling the module to adapt and optimize based on the relational dynamics of the input features. To enhance the clarity and utility of the outputs from the Bi-affine module, average pooling is applied across aspect nodes within both the Specific-aspect GCN and Inter-aspect GCN. Specifically, the equations used for this process are given by Equations (13)–(15):

$$h_a^{cs}=f(h_{a_1}^{cs},h_{a_2}^{cs},...,h_{a_n}^{cs})$$

(13)

$$h_a^k=f(h_{a_1}^k,h_{a_2}^k,...,h_{a_n}^k)$$

(14)

$$r=\left[h_a^{cs},h_a^k\right]$$

(15)

Figure 9. The structure of the Bi-affine module.

Figure 9. The structure of the Bi-affine module.

4.4. Attention Scores: Prioritizing Relevant Features

The SI-GCN model incorporates a retrieval-based attention mechanism as a critical component of its hierarchical structure. This approach is designed to extract the crucial features associated with aspect terms from the hidden state vector and assign a retrieval-based attention weight to each context term. This mechanism operates through a series of computations to refine the aspect-specific input. Initially, attention scores are calculated using Equations (16) and (17):

$${\beta }_t=\sum _{t=1}^n{\mathbf{h}}_c^{t\top }r$$

(16)

$${\alpha }_t=\frac{exp\left({\beta }_t\right)}{{\sum }_{i=1}^{n}exp\left({\beta }_i\right)}$$

(17)

where ${\beta }_t$ and ${\alpha }_t$ denote the preliminary attention score and normalized attention weight, respectively. These equations allow the model to dynamically allocate higher weights to context terms more relevant to the target aspect.

The aspect input is then formulated by aggregating the weighted hidden states of the context terms as defined in Equations (18) and (19), and the aspect input passed through a softmax layer to produce the final prediction y in Equation (18). Here, $W_c$ and $b_c$ represent the trainable parameters optimized during training:

$$y=Softmax\left(W_{c}p+b_c\right)$$

(18)

$$p=\sum _{i=1}^n{\alpha }_i{h}_c^i$$

(19)

The training process is guided by the loss function defined in Equation (20), which calculates the cross-entropy between predicted and true sentiment distributions. To mitigate overfitting, the loss function incorporates an $L2$ regularization term weighted by $\lambda$:

$$\mathcal{L}=-\sum _{i=1}^S\sum _{j=1}^C{\widehat{y}}_i^j·log\left(y_i^j\right)+\lambda {\parallel \Theta \parallel }_2$$

(20)

In this equation, S represents the total number of training samples. C denotes the number of sentiment categories. $\widehat{y}$ refers to the actual sentiment distribution. $\Theta$ encapsulates all trainable parameters of the model.

This hierarchical and retrieval-based attention mechanism operates synergistically with the Mutual Bi-affine Module and GCN outputs as depicted in Figure 2. It enables the SI-GCN model to prioritize contextually relevant features, facilitating nuanced sentiment analysis across multiple aspects while maintaining consistency with its modular design.

4.5. Comprehensive Execution Steps of the SI-GCN Model

This section presents a comprehensive explanation of the SI-GCN algorithm (Algorithm 3), detailing its sequential workflow. The process begins with feature extraction performed by the Specific-aspect and Inter-aspect modules. The outputs are then refined using the Bi-affine module and aggregated through average pooling to integrate node representations. Finally, a retrieval-based attention mechanism highlights the relevant features, and a softmax classifier predicts the sentiment polarity for the given aspect.

Algorithm 3 Workflow of the SI-GCN model.

Input: Sentence–aspect couple $(S,T)$, where $S=\{w_1,w_2,\dots ,w_m\}$ represents the sentence with m words, $T=\{w_{start},w_{start+1},\dots ,w_{start+n-1}\}$ denotes the aspect with n-words Output: Aspect sentiment polarity y 1: Obtain output $Z^{cs}$ through Specific-aspect module 2: Obtain output $Z^k$ through Inter-aspect module 3: Using the Bi-affine module to obtain the output $Z^{\mathrm{cs}\prime }$ and $Z^{\mathrm{k}\prime }$ of both modules by Equations (11) and (12) 4: Using average-pooling to fuse the node representations of the two modules r by Equations (13)–(15) 5: Leveraging a retrieval-based attention strategy to extract key features linked to aspect words ${\alpha }_t$ as defined by Equations (16) and (17) 6: Using softmax to obtain the final classification result y by Equation (18) 7: return  $y\in \left\{0,1,2\right\}$

5. Experiments

The purpose of this section is to showcase the utility of the SI-GCN model via a series of experimental evaluations. In Section 5.1, we present the datasets employed for our experimental analysis and detail the configuration of our experimental framework. Section 5.2 outlines the metrics employed to assess the model’s performance. The baseline model, which we use for comparative purposes, is detailed in Section 5.3. The outcomes of our experiments and their analysis are provided in Section 5.4. Section 5.5 is dedicated to ablation studies, aimed at deeper insights into the model’s workings. An analysis of the model’s key components from three different perspectives is offered in Section 5.6. The document concludes with Section 5.7, where a selection of case studies is presented to illustrate the efficacy of our proposed model.

5.1. Datasets and Experiment Setting

We assessed the effectiveness of the proposed method using four benchmark datasets widely employed in sentiment analysis research: Restaurant14, Laptop14, Restaurant15, and Restaurant16. (Restaurant14, Laptop14 are published in the SemEval2014 (https://aclanthology.org/S14-2004/ (accessed on 20 July 2024)). Restaurant15 is published in the SemEval2015 (https://aclanthology.org/S15-2082/(accessed on 20 July 2024)) and Restaurant16 is published in the SemEval2016 (https://aclanthology.org/S16-1002/(accessed on 20 July 2024)).) These datasets are well-established resources in the field. Restaurant14 and Laptop14 were introduced during the SemEval2014 ALSA challenge, while Restaurant15 and Restaurant16 were featured in SemEval2015 and SemEval2016, respectively. Each sentence in these datasets is annotated with aspect terms and their corresponding sentiment polarities, with every sentence containing at least one aspect term.

Table 1. Statistics for the four experimental datasets.

Dataset	Positive		Neural		Negative
	Train	Test	Train	Test	Train	Test
Rest14	2164	728	637	196	807	196
Lap14	994	341	464	169	870	128
Rest15	1178	439	50	35	382	328
Rest16	1620	597	88	38	709	190

provides a summary of these datasets along with their associated statistical details.

In this study, the SI-GCN model was implemented using Python 3.6, PyTorch 1.0.0, and SpaCy 2.0.18 frameworks. Pre-trained GloVe: (https://nlp.stanford.edu/projects/glove/ (accessed on 9 July 2024)), and BERT models were used to initialize the word embeddings. Specifically, the GloVe vectors initialized each word as a 300-dimensional embedding, while the hidden vector representations also had a dimensionality of 300. The model comprised two GCN layers. The Adam optimizer, with a learning rate of ${10}^{-6}$, was employed to train the model, and a batch size of 16 was used. The detailed experimental parameters are summarized in

Table 2. Specific experimental parameter settings.

Parameter	Value	Explanation
dim_w	300	Dimensionality of word embeddings, initialized using pre-trained models.
dim_h	300	Dimensionality of the hidden layer representation.
dim_k	200	Dimensionality of knowledge graph embeddings, representing conceptual relationships.
dropout_rate	0.1	Dropout rate applied to mitigate overfitting during training.
learning_rate	${10}^{-6}$	Learning rate for the optimizer to control model updates.
n_epoch	50	Number of epochs for model training.
num_layers	2	Number of GCN layers to define the model’s depth.
bs	64	Batch size for processing the dataset during training.
seed	29	Seed value for random number generation to ensure reproducibility.

.

We integrated the BERT model and the Adam optimizer into the SI-GCN framework to enhance both the accuracy and efficiency of training. BERT generates deep, contextualized word embeddings by pre-training on extensive textual corpora, enabling it to capture complex linguistic patterns and subtle nuances. This capability allows the model to produce rich semantic representations that are context-sensitive—an essential feature for aspect-level sentiment analysis, where sentiment polarity often depends on the surrounding context. By fine-tuning BERT within the framework, we harness its advanced language understanding abilities to improve sentiment classification accuracy.

The Adam optimizer was selected for its efficiency in handling sparse gradients and its ability to adaptively adjust learning rates for individual parameters. By combining the strengths of AdaGrad and RMSProp, Adam achieves fast convergence and robustness to hyperparameter configurations. This optimizer accelerates training, improves stability during optimization, and enhances overall efficiency. Consequently, its use enables the SI-GCN model to achieve superior performance in less time compared to traditional optimization methods.

5.2. Evaluation Metrics

Throughout this manuscript, we evaluate the performance of the proposed SI-GCN model using Accuracy (Acc) and Macro-F1 (F1) as performance metrics. These metrics are widely used to assess the effectiveness of models in multi-class classification tasks. Accuracy measures the proportion of correctly predicted samples, while Macro-F1 represents the harmonic mean of accuracy and precision, balancing both metrics. The formulas for calculating Acc and F1 are as follows:

$$Precision=\frac{TruePos}{TruePos+FalsePos}$$

(21)

$$Recall=\frac{TruePos}{TruePos+FalseNeg}$$

(22)

$$F1Score=2\times \frac{Rec\times Pre}{Rec+Pre}$$

(23)

$$Accuracy=\frac{TruePos+TrueNeg}{TotalSamples}$$

(24)

5.3. Baseline Methods

This section aims to comprehensively evaluate the efficacy and robustness of our proposed models by comparing them with four well-established baseline models commonly used in sentiment analysis.

(1)

Syntax-based Methods:

• R-GAT [57]: This method introduces an innovative aspect-centric dependency tree structure to effectively encapsulate syntactic details. By refactoring and pruning the standard dependency parse tree to focus on the target aspect, R-GAT enhances the model’s ability to capture crucial syntactic information relevant to the aspect.
• DGEDT [58]: The DGEDT model employs a dual-transformer architecture that combines planar representations derived from the transformer with graph-based representations from the dependency graph through iterative interactions. This approach effectively leverages the strengths of both representations, resulting in improved performance in modeling dependency relationships for aspect-level sentiment analysis tasks.
• LSTM+synATT+TarREP [30]: This model fuses syntactic details with an attention mechanism to bolster the representation of target terms.

(2)

Context-based Methods:

• TD-LSTM [59]: This model captures the context surrounding an aspect word using two LSTM networks. One LSTM processes the context from the left of the aspect word, while the other processes it from the right. The aspect word serves as the dividing marker for these directional models.
• ATAE-LSTM [28]: This model excels at identifying the relevance of context words to a given aspect by integrating an attention mechanism with LSTM. This combination enables effective semantic modeling of sentence structure with a focus on aspect-specific context.
• MemNet [29]: This model integrates the content and position of a specified aspect into a memory network by constructing memory representations based on contextual information. Through an attention mechanism, it captures the most relevant information to determine the emotional tendencies of different aspects.
• IAN [6]: To strengthen the attention mechanism on both aspect and context, IAN employs interactive learning between them. This approach generates representations for both targets and contexts, enabling the acquisition of context-dependent aspect representations.
• MGAN [60]: This model combines dependency trees with neural networks to extract sentence feature representations using Bi-LSTM. The architecture incorporates a broad-category aspect classification task to complement the detailed aspect item classification task and introduces an innovative attention mechanism to align features between the tasks effectively.

(3)

Knowledge-based Methods:

• Sentic-LSTM [60]: This model extends the traditional LSTM by integrating commonsense knowledge to guide information flow, enhancing its ability to process sentiment-related information effectively.
• SK-GCN1 and SK-GCN2 [37]: These models combine a syntactic dependency tree with a commonsense knowledge graph using GCN, forming a hybrid syntactic and knowledge-driven architecture. This integration significantly improves aspect-level sentiment analysis by leveraging both structural and conceptual relationships.

(4)

Other Neural Network Methods:

• ASGCN-DT and ASGCN-DG [20]: These models leverage GCN-based dependency trees to extract syntactic information and word dependencies. By utilizing directed and undirected graphs, they identify and capture sentiment specific to particular aspects.
• AGCN-MEAN and AGCN-MAX [61]: These models enhance the capability of graph convolutional networks to model emotions by incorporating two distinct aggregator functions. These functions gather information from adjacent nodes using different aggregation methods.
• AFGCN and InterGCN [33]: These approaches construct heterogeneous graphs for each instance by capturing specific-aspect focus and inter-aspect contextual dependencies. The interaction is facilitated by extracting emotional features from both aspect-focused and inter-aspect perspectives.

5.4. Main Results and Analysis

The goal of this section is to evaluate the performance of the proposed SI-GCN model across four distinct datasets. Its efficacy is benchmarked against four types of baseline models. Accuracy (Acc) and Macro-F1 (F1) are used as the evaluation metrics. The experimental results highlight the effectiveness of the SI-GCN model, with comprehensive findings summarized in

Table 3. Experimental results on Restaurants14, Laptop14, Restaurants15, and Restaurants16. The best scores are in bold.

Category	Model	Rest14		Lap14		Rest15		Rest16
		Acc	F1	Acc	F1	Acc	F1	Acc	F1
Syntax	R-GAT [57]	83.30	76.08	77.42	73.76	-	-	-
	DGEDT [58]	83.30	76.08	77.42	73.76	76.39	58.70	82.16	54.21
	LSTM+SynATT [30]	79.43	69.25	70.87	66.53	78.03	58.30	83.27	65.76
Context	TD-LSTM [59]	78.00	66.73	71.83	68.43	76.39	58.70	82.16	54.21
	ATAE-LSTM [28]	78.60	67.02	68.88	63.93	78.48	60.53	83.77	61.71
	MemNet [29]	78.16	65.83	70.33	64.09	77.89	59.52	83.04	56.91
	IAN [6]	77.86	66.31	71.79	65.92	78.58	54.94	82.42	57.12
	MGAN [60]	81.25	71.94	75.39	72.47	79.36	57.26	87.06	62.29
Knowledge	SENTIC LSTM [60]	79.43	70.32	70.88	67.19	79.55	60.56	83.01	68.22
	SK-GCN [37]	80.54	70.14	73.04	68.41	79.76	61.37	83.82	64.84
	SK-GCN [37]	80.36	70.43	73.20	69.18	80.12	60.70	85.17	68.08
others	ASGCN-DT [20]	80.86	72.19	74.14	69.24	79.34	60.78	88.69	66.64
	ASGCN-DG [20]	80.77	72.02	75.55	71.05	79.89	61.89	88.99	67.48
	AGCN-MEAN [61]	80.02	71.02	75.07	70.96	80.07	62.70	87.98	65.78
	AGCN-MAX [61]	80.52	72.33	74.61	70.58	80.62	62.55	86.20	66.58
	AFGCN [33]	81.79	73.42	76.96	73.29	81.55	65.08	89.12	70.60
	InterGCN [33]	82.23	74.01	77.86	74.32	81.76	65.67	89.77	73.05
	SI-GCN (ours)	83.75	76.04	80.93	73.09	81.95	66.37	90.10	74.71
	SK-GCN1+BERT [37]	81.87	73.42	79.31	75.11	82.72	63.34	86.03	69.69
	SK-GCN2+BERT [37]	83.48	75.19	79.00	75.57	83.20	66.78	87.19	72.02
	AFGCN+BERT [33]	86.57	80.36	82.53	79.10	85.02	70.89	90.76	76.88
	InterGCN+BERT [33]	87.12	81.02	82.87	79.32	85.42	71.05	91.27	78.32
	SI-GCN+BERT (ours)	88.96	83.67	84.02	78.54	87.45	71.83	93.23	78.21

and Figure 10.

The experimental results demonstrate that the SI-GCN model surpasses various benchmark models in terms of Acc and F1. This highlights the advantages of integrating contextual information, syntactic dependency structures, and commonsense knowledge within the model for aspect-level sentiment analysis. By leveraging the strengths of graph convolutional networks and combining them with syntactic dependency trees and commonsense knowledge, the model effectively extracts aspect-specific features. The integration of these diverse information sources allows for a more nuanced understanding of sentiment within sentences. Additionally, the model successfully addresses the complexities of aspect-level sentiment variations observed in real-world datasets. These findings suggest that the proposed method holds substantial potential for applied sentiment analysis tasks. A detailed breakdown of scores and contributions is provided below.

• Within the Laptop14 dataset, although the F1 score of the proposed SI-GCN model is slightly lower than that of the InterGCN model by 0.23%, the Acc is significantly higher, showing an improvement of 3.07%. Moreover, the SI-GCN model consistently outperforms other models that utilize syntactic, contextual, and knowledge-based information across the remaining three datasets. These experimental results confirm the effectiveness of SI-GCN for aspect-level sentiment analysis. The findings demonstrate that integrating multiple sources of information—context, syntax, and knowledge—from various perspectives significantly enhances sentiment analysis performance.
• Furthermore, the SI-GCN model achieves superior performance compared to ASGCN-DT and AGDCN-DG, which rely solely on heterogeneous graphs over syntactic dependencies. This underscores the importance of incorporating commonsense knowledge and contextual information alongside syntactic dependency trees for more effective aspect-level sentiment analysis.
• An intriguing observation from the experimental results is that models incorporating commonsense knowledge generally achieve higher average performance compared to those relying solely on syntactic or contextual information. This underscores the greater potential of commonsense knowledge in aspect-level sentiment analysis. Incorporating commonsense knowledge into the Specific-aspect module enables better identification of sentiment polarity in complex sentences and provides significant advantages in handling sentences with implicit sentiment polarity.
• In the Restaurant14 dataset, 36.58% of sentences contain multifaceted words. Compared to the best baseline model, InterGCN, the Acc and F1 values of SI-GCN are increased by 1.52% and 2.03%, respectively. These results demonstrate that our model excels in processing sentences with multiple aspect terms, highlighting the importance of the interactive extraction capabilities of both the Specific-GCN and Inter-GCN modules. The effectiveness of SI-GCN in handling multifaceted sentences emphasizes the critical role of incorporating both specific-aspect and inter-aspect contextual dependencies in aspect-level sentiment analysis tasks.
• Additionally, the SI-GCN model exhibits substantial improvements over the pre-trained BERT model. This indicates that our method can effectively augment the capabilities of pre-trained language models, thereby enhancing the performance of sentiment analysis tasks.

5.5. Ablation Study

This section presents a comprehensive evaluation of the individual contributions of the modules within the proposed SI-GCN model through a series of ablation studies. The SI-GCN model is built around two primary modules: the Specific-aspect module and the Inter-aspect module. During the ablation experiments, these modules were systematically removed or included to assess their individual and combined impacts on the model’s overall performance. By testing various configurations, the experiments aimed to validate the unique contributions of each module and their synergistic effects in enhancing the model’s efficacy. The findings from these ablation studies are detailed in the subsequent sections, with the experimental setup outlined below.

• w/o Knowledge: This condition eliminates the SI-GCN knowledge branch (Inter-aspect module) and retains solely the Specific-aspect module.
• w/o Syn and Con: This condition eliminates the SI-GCN knowledge branch (Inter-aspect module) and retains solely the Specific-aspect module.
• w/o Syntax: This condition excludes syntactic information from the Specific-aspect module, maintaining both contextual information and the Inter-aspect module.
• w/o Context: This condition removes contextual information from the Specific-aspect module while preserving syntactic information and the Inter-aspect module.
• w/o Bi-affine: This condition eliminates the dual affine module, preventing interaction between the Specific-aspect module and the Inter-aspect module.

The results of our ablation studies, which evaluate the Acc and F1 score of the SI-GCN model, are presented in

Table 4. Outcomes from the ablation analysis experiments. The best scores are in bold.

Model	Rest14		Lap14		Rest15		Rest16
	Acc	F1	Acc	F1	Acc	F1	Acc	F1
SI-GCN w/o Knowledge(Specific-GCN)	82.11	73.71	77.74	73.78	82.10	65.70	89.61	71.71
SI-GCN w/o Syntax	81.97	73.75	77.42	73.76	81.57	65.63	89.56	71.44
SI-GCN w/o Context	82.02	73.08	76.48	72.65	81.22	65.01	88.79	70.83
SI-GCN w/o Syn and Con(Inter-GCN)	81.41	74.13	75.89	72.14	80.49	63.57	87.94	70.01
SI-GCN w/o Bi-affine	82.43	74.15	78.32	72.68	80.67	65.49	89.12	73.25
SI-GCN	83.75	76.04	80.93	73.09	81.95	66.37	90.10	74.71

and Figure 11. These experiments were conducted to assess the influence of each module within the SI-GCN model on its overall performance. The findings reveal that removing the commonsense knowledge base and relying solely on the Specific-aspect GCN module results in a significant decline in performance. This suggests that considering only one specific aspect, without acknowledging the interrelationships among aspects, leads to a substantial reduction in accuracy. Furthermore, excluding either syntax or context from the Specific-aspect GCN module also results in decreased performance, highlighting the critical significance of both syntactic dependency structures and contextual details in capturing the sentiment orientation of the target term.

Moreover, relying exclusively on the Inter-GCN module to capture the sentiment polarity of aspectual words, without the support of the Specific-GCN module, results in suboptimal performance across all datasets. This highlights the necessity of incorporating information specific to the aspectual words themselves, rather than depending solely on their interactions. Additionally, the removal of the Bi-affine module, which enables interaction between the Specific-GCN and Inter-GCN modules, significantly impairs the overall performance of the SI-GCN model. This underscores the importance of integrating diverse types of information from multiple perspectives to enhance performance in aspectual sentiment analysis.

5.6. Discussion

This section examines the impact of several additional factors on the model’s performance, aiming to provide a more detailed analysis of the SI-GCN model. These factors include the number of GCN layers, multifaceted word analysis, and sentence visualization. Each of these elements significantly influences the model’s performance and is essential for a comprehensive understanding of the strengths and weaknesses of our proposed methodology. By analyzing these factors, we can assess the efficacy of our model and identify areas for further improvement.

5.6.1. Influence of GCN Layer Configuration on Model Performance

To evaluate the effect of the number of GCN layers on the performance of the SI-GCN model, we conducted experiments using the restaurant and laptop datasets, assessing models with one to seven GCN layers. The results of these experiments are presented in Figure 12. Our findings reveal that the performance of the SI-GCN model significantly decreases when the number of GCN layers reaches three, with optimal performance achieved at two GCN layers. Therefore, we chose to utilize two GCN layers in our proposed model.

5.6.2. Model Accuracy for Multi-Aspect Discrimination

This section presents experiments conducted separately on the Restaurant14 and Laptop14 datasets to evaluate the accuracy (Acc) of our model in distinguishing words with multiple aspects. The statistics indicate that over 50% of the sentences in the datasets were multifaceted. Additionally, we observed a decline in accuracy when the number of aspects exceeded three. Details of the experiment are shown in Figure 13.

5.6.3. Visualization

This section begins by selecting two representative sentences to assess the sentiment polarity of our model for specific aspectual words. We will subsequently conduct a visual analysis of these sentences, considering their context, syntax, and knowledge representations. This analysis will illustrate the complementarity of the proposed multi-view representation approach.

(1)

Special Sentences

A visualization analysis was performed on selected representative sentences to examine the weights assigned by the SI-GCN model. Sentences containing multiple aspect words, some with more than one word, were chosen for this purpose. The SI-GCN model effectively identifies the focus words in these instances. In analyzing sentences with multiple aspect words, we noted that certain aspect words exhibit distinct emotional characteristics, while others may not. In such cases, the model leverages the semantic, syntactic, and contextual details of the sentence, along with other aspect terms, to determine the sentiment orientation of the specific aspect. The following sentences were used for the visualization analysis:

• The servers behind the counter are always friendly.
• The service was the only thing good about this restaurant.

The visualization results of the two sentences are illustrated in Figure 14. In the first sentence, the SI-GCN model effectively highlights significant specific-aspect words, particularly those associated with the term “servers”, which consists of multiple words. In the second sentence, the aspectual word “service” is clearly assigned a positive sentiment, while the aspectual word “restaurant” does not have a distinct sentiment marker. Nevertheless, our model accurately infers the sentiment of “restaurant” based on the sentiment of “service”. These experiments showcase the model’s remarkable ability to detect sentiment associations between the focal aspects and other related aspects.

(2)

Effect of Different Modules:

To evaluate the effectiveness of the syntactic, contextual, and knowledge graphs utilized in our two modules for identifying specific aspects and their relationships, we conducted a comprehensive visualization analysis of the aforementioned sentences. In addition to the initial weight analysis of the SI-GCN model, we examined the sentences from three additional perspectives: syntax, context, and knowledge. An attentional weight graph was employed to assess the influence of each aspect on the overall sentiment of the sentence. The experimental outcomes are illustrated in Figure 15. This analysis aims to provide comprehensive insights into our model’s ability to discern sentiment relationships between specific aspects and those within sentences containing multiple aspect terms.

The first sentence posed a challenge in identifying the aspect of “servers behind the counter” from a knowledge perspective. This aspect was accurately captured by focusing on the contextual and syntactic details of the sentence. In the second example, we encountered difficulties in accurately capturing both aspects of the word from a contextual perspective. Instead, the syntactic perspective highlighted certain syntactical words such as “was” and “only”. However, by adopting a knowledge-based approach, we were able to accurately and effectively extract both aspects of the words.

5.7. Case Study

We chose several sentences from the dataset to perform a case study for more in-depth analysis. We generated attention heat maps using AT-LSTM, ASGCN-DG, and other methods, where the darker shade indicates a higher level of attention given to the word. The experimental design is illustrated in

Table 5. Visualize the attention scores of our SI-GCN model alongside the best baseline model, along with their respective predicted and true labels. Darker shades denote higher attention weights assigned to words. The values −1, 0, and 1 signify negative, neutral, and positive sentiments, respectively. In the table, ✓ indicates a correctly predicted sentiment label, while ✗ indicates an incorrectly predicted sentiment label.

Model	Aspect	Attention Visualization	Prediction	TrueLabel
ATAE-LSTM [28]	food, service	$\mathrm{Great}$ food but the service was $\mathrm{dreadful}$!	(1✓,0✗)	(1,−1)
	place	i recommend this $\mathbf{place}$ to everyone	0✗	1
	size	the $\mathrm{smaller}$ size was a bonus because of $\mathrm{space\; restrictions}$	−1✗	1
	logic board	more likely it will require $\mathrm{replacing}$ the logic board once they $\mathrm{one}$ they have a problem and come up with a $\mathrm{solution}$	1✗	−1
ASGCN-DG [20]	food, service	Great $\mathbf{food}$  $\mathrm{but}$ the service was $\mathrm{dreadful}$!	(−1✗,−1✓)	(1,−1)
	place	I recommend $\mathrm{this}$ place to $\mathrm{everyone}$	0✗	1
	size	the $\mathrm{smaller}$ size was a bonus because of space $\mathrm{restrictions}$	−1✗	1
	logic board	$\mathrm{More\; likely}$ it will require $\mathrm{replacing}$ the logic board once they admit they have a problem and come up with a solution	1✗	−1
Inter-GCN [33]	food, service	Great food $\mathrm{but}$ the service was $\mathrm{dreadful}$!	(0✗,−1✓)	(1,−1)
	place	I $\mathrm{recommend}$ $\mathrm{this}$  $\mathbf{place}$  to everyone	1✓	1
	size	the smaller size was a bonus because of space $\mathrm{restrictions}$	−1✗	1
	logic board	More likely it will $\mathrm{require\; replacing}$ the logic board $\mathrm{once}$ they $\mathrm{admit}$ they have a $\mathrm{problem}$ and come up with a solution	1 ✗	−1
Specific-GCN	food, service	$\mathrm{Great}$ food but the service $\mathrm{was}$ dreadful!	(1✓,0✗)	(1,−1)
	place	I recommend this $\mathbf{place}$ to $\mathrm{everyone}$	0✗	1
	size	the smaller $\mathbf{size}$ was a bonus because of space restrictions	1✓	1
	logic board	More likely it will $\mathrm{require}$ replacing the logic board once they admit they have a $\mathrm{problem}$ and colorboxb onecome up with a solution	−1✓	−1
SI-GCN	food, service	$\mathrm{Great}$ food but the service was $\mathrm{dreadful}$!	(1✓,−1✓)	(1,−1)
	place	I $\mathrm{recommend}$ this place to $\mathrm{everyone}$	1✓	1
	size	the $\mathrm{smaller}$ $\mathbf{size}$ was a bonus because of space $\mathrm{restrictions}$	1✓	1
	logic board	More likely it will require replacing the $\mathbf{logic\; board}$ once they admit they have a $\mathrm{problem}$ and come up with a solution	−1✓	−1

. The outcomes indicate that the GCN-based approach is adept at accurately identifying important words in a sentence related to a given aspect.

(1)

In the first example, the sentence “food is good, but service is dreadful” demonstrates that all models successfully identify the relevant opinion word “great” associated with the aspect term “food”, resulting in an accurate prediction. This success is attributed to the close association between the aspect word “food” and its corresponding opinion word “great”. However, for the aspect term “service”, both the AT-LSTM and Specific-GCN models fail to accurately predict the corresponding opinion word “dreadful”.

(2)

In the analysis of the second example, the aspect word “place” lacks an explicit sentiment, and relying solely on its inherent meaning may lead other models to misclassify it as neutral. However, our SI-GCN model effectively determines the sentiment orientation of the aspect by integrating multi-view representations.

(3)

The third example involves a sentence with an aspect composed of multiple words, making it challenging for the model to accurately identify the focal point, specifically the word “size”. In such cases, models are more likely to focus on the modifying word “smaller”, resulting in incorrect predictions of negative sentiment. Nevertheless, our experimental results demonstrate that only the Specific-GCN and SI-GCN models accurately predict the sentiment polarity, underscoring the effectiveness of our model in capturing critical aspect words.

(4)

The fourth case involves a long, complex sentence. Our experiments showed that most models tend to focus on the words “replacing” and “problem” in the subordinate clause, leading to biased predictions for the aspect words.

(5)

Further analysis reveals that the context-based model ATAT-LATM and the syntax-based model Inter-GCN perform worse than combined models, such as Specific-GCN, which integrate both context and syntax. This observation suggests that a synergistic integration of these approaches can enhance the encoding of syntactic and contextual information, thereby establishing deeper relationships between aspects and their corresponding opinion terms.

The experimental findings indicate that our proposed SI-GCN model achieves high prediction accuracy, underscoring its effectiveness and superiority over other models. Notably, the second, third, and fourth examples illustrate that our model effectively handles various complex scenarios through multi-view representation learning. These scenarios include dealing with aspect words that lack explicit emotional information, aspect words consisting of multiple words, and complex, informal sentences.

6. Conclusions and Future Works

This paper presents a novel approach to aspect-level sentiment analysis through the development of the Specific-aspect and Inter-aspect Graph Convolutional Network (SI-GCN). This approach addresses key challenges faced by existing methods, particularly in effectively capturing sentiments associated with multi-word aspects and their interrelations within complex sentences. By integrating graph convolutional networks with specific-aspect and inter-aspect components, the SI-GCN incorporates a diverse array of information, including syntactic dependencies, contextual cues, and external knowledge, thereby enhancing both the accuracy and comprehensiveness of sentiment analysis.

The key contributions of this research are as follows: First, we introduce a new graph structure that captures interdependencies among word nodes while considering the relationships among aspectual words, contextual words, and complete sentences. This holistic approach facilitates the extraction of more comprehensive sentiment information compared to traditional methods, which often focus solely on individual words or phrases. Second, the SI-GCN effectively analyzes multi-word aspects, addresses ambiguous sentiment polarities, and manages sentences containing multiple aspects—areas where other models typically struggle. By incorporating these elements, our model offers a more precise representation of sentiment. Finally, the integration of an external knowledge base significantly enhances the model’s ability to disambiguate sentiment, providing deeper contextual insights that improve the overall sentiment analysis process.

Extensive experimentation on four publicly available benchmark datasets confirms the reliability and robustness of the SI-GCN. Our results demonstrate that the SI-GCN outperforms existing state-of-the-art methods, achieving improvements in sentiment classification accuracy ranging from 0.9% to 2.3%. These findings indicate that the SI-GCN is well suited for real-world applications, particularly in scenarios involving complex sentence structures, multi-word aspects, and informal language.

Future research will focus on enhancing the adaptability of the SI-GCN model to better handle dynamic and evolving datasets, particularly in real-time sentiment analysis scenarios. Additionally, we aim to explore the integration of more advanced external knowledge bases and domain-specific ontologies to further improve the accuracy of aspect-level sentiment analysis and enhance contextual understanding.