KGCFRec: Improving Collaborative Filtering Recommendation with Knowledge Graph

Abstract

Traditional collaborative filtering (CF)-based recommendation systems are often challenged by data sparsity. The recent research has recognized the potential of integrating new information sources, such as knowledge graphs, to address this issue. However, a common drawback is the neglect of the interplay between user–item interaction data and knowledge graph information, resulting in insufficient model performance due to coarse-grained feature fusion. To bridge this gap, in this paper, we propose a novel graph neural network (GNN) model called KGCFRec, which leverages both Knowledge Graph and user–item Collaborative Filtering information for an enhanced Recommender system. KGCFRec employs a dual-channel information propagation and aggregation mechanism to generate distinct representations for the collaborative knowledge graph and the user–item interaction graph. This is followed by an attention mechanism that adaptively fuses the knowledge graph with collaborative information, thereby refining the representations and narrowing the gap between them. The experiments conducted on three real-world datasets demonstrate that KGCFRec outperforms state-of-the-art methods. These promising results underscore the capability of KGCFRec to enhance recommendation accuracy by integrating knowledge graph information.

1. Introduction

Recommendation systems (RSs) [1] have become widely used in multimedia applications [2,3,4], social networks [5,6], and e-commerce platforms [7,8,9] with the growth of internet services. Great success has been achieved by collaborative filtering (CF) recommendation, which takes into account the users’ previous interaction behavior and makes recommendations based on potential similar preferences of users who share similar interests. The CF-based RS, however, generally suffers the data sparsity problem. In order to overcome the problem, some recent research [10] has focused on using side information, such as user attribution and item description. The representation of relationships between users and items can be enhanced and complemented by the side information of the knowledge graph (KG) [11,12]. From certain perspectives, KGs can partially mitigate the issue of the data sparsity problem.

However, the recently proposed KG-based RS models have shortcomings in information modeling and information fusion, resulting in sub-optimal recommendation performance. The existing methods, which independently model knowledge graphs and collaborative information, are faced with problems such as insufficient capability for long-distance information modeling [13,14] and complex task-oriented design [15,16]. In the feature fusion stage, these models tend to ignore the gap between the representation distribution and combine the knowledge graph and the collaborative information in a coarse-grained way [17,18]. How to perform adaptive feature fusion from the knowledge graph side and the collaborative information side in an end-to-end manner to improve the recommendation performance is still a common challenge facing the current field.

More recently, creating end-to-end models based on a graph neural network (GNN) has been a popular technical approach. The key idea is to utilize the message passing and aggregation mechanism, which can effectively integrate the KG and collaborative information into the embedding space. Motivated by this, we created a novel framework we called KGCFRec, which is a kind of two-channel information propagation and aggregation method aimed to produce the corresponding representations of collaborative knowledge graphs and user–item interaction graphs.

KGCFRec consists of two components to address the challenge in the appropriate way: (1) Knowledge and Collaborative-aware representation learning. We predefine the knowledge graph, the user–item interactions graph, and the collaborative-knowledge graph. Then, we design a novel aggregation mechanism of GNN for knowledge- and collaborative-aware representation learning. (2) Adaptive feature fusion. In order to achieve adaptively fine-grained fusion of the two types of features, an attention mechanism is designed to learn the weights between nodes on the two graphs throughout the propagation process. At last, the collaborative knowledge graph representation for Top-N recommendation is generated.

In summary, our work makes the following contributions:

• The knowledge graph and user–item interactions graph are combined into a single collaborative-knowledge graph in order to address the data sparsity problem.
• KGCFRec, an end-to-end model, is proposed that fuses knowledge and collaboration features for recommendation via a novel attention mechanism.
• Experiments conducted on three real-world datasets demonstrate that our proposals perform significantly better than baselines.

The organization of our paper is shown as follows: we first present related work in Section 2 and then provide the problem formalization in Section 3. Section 4 describes the knowledge- and collaborative-aware representation learning module and the adaptive feature fusion module. Section 5 describes the experimental setup. The experiment results and discussion are provided in Section 6. In the end, the conclusion and future work are shown in Section 7.

2. Related Work

There are three primary types of current study on knowledge-graph-enhanced recommendation: embedding-based, path-based, and GNN-based.

2.1. Embedding-Based Methods

Embedding-based methods primarily extract the data from the first-order neighboring nodes in the knowledge graph (KG) and the user–item interaction graph [19]. First of all, these approaches frequently use KG embedding techniques like TransE [20] and TransH [21] to improve the entity embeddings. These entity embeddings are then used as item semantic characteristics for supplying the recommendation model. For instance, TransE is utilized by collective knowledge embedding (CKE) [13] to extract entity embeddings from the KG and effectively incorporates them into matrix factorization (MF). Knowledge-aware transferable user preference (KTUP) [14] constructs a multi-task framework that applies TransH to both user–item interactions and KG triplets concurrently, aiming to learn user preferences and accomplish KG completion in unison. However, instead of directly recommending, KG embedding algorithms usually concentrate on modeling strict semantic relationships, which are better suited for link prediction tasks. Furthermore, embedding-based methods frequently ignore higher-order connectivity and lack end-to-end training regimes. This oversight prevents them from capturing the extensive semantics or sequential dependencies along the paths between nodes, thereby constraining their capability to reveal the deeper, underlying user–item relationships.

2.2. Path-Based Methods

Path-based methods consider the relational patterns linking users and items through KG entities, often referred to as meta-paths [22,23,24], to enhance the predictive capabilities of recommendation systems. In order to extract paths that convey valuable higher-order information, researchers have resorted to either deploying algorithms that pinpoint significant paths or crafting predefined meta-path templates to limit the scope of potential paths. These identified paths are subsequently harnessed to anticipate user preferences through the use of recurrent neural networks and memory networks. A case in point is RippleNet [15], which concentrates on the propagation of user interests and identifies potential paths from items interacted with by users to other items, without the need for predefining meta-paths or meta-graphs. In order to improve top-N recommendations, MCRec [16] has built a deep neural network with a co-attention mechanism that takes advantage of the rich context provided by meta-paths within various information networks. Path-based techniques rely heavily on manually generated meta-paths/meta-graphs, which might complicate performance tweaking in real-world scenarios, even though they can more organically incorporate KG insights.

2.3. GNN-Based Methods

GNN-based methods exploit the information aggregation prowess of GNNs to derive node representations. The key of GNN-based methods is to develop powerful node representations [25,26,27]. They achieve this by integrating information from the immediate neighboring nodes to refine the representations of the central nodes. Through iterative application of this process, information from nodes several hops away can be incorporated into the representations. This enables these methods to capture extended connectivity patterns. To enhance the vector representations of items, knowledge graph convolutional networks (KGCN) [28] utilize an end-to-end design, a biased neighbor aggregation strategy, and a graph convolution sampling technique to capture distant inter-item interactions. KGNN-LS [29], on the other hand, computes the embedding of each item node by aggregating feature information within the node’s local network neighborhood. By designing unique aggregation algorithms for each, CKAN [18] distributes and aggregates data throughout the KG and the user–item interactions graph, respectively. KGAT [17] integrates user–item interactions with the KG into a unified heterogeneous graph and employs an attention mechanism to iteratively propagate embeddings from neighboring nodes—be they users, items, or attributes—to further refine each node’s embedding.

While numerous studies have integrated KG and GNN techniques into recommender systems, the fusion of KG information with collaborative data has not been extensively explored. KGCFRec addresses this gap by learning dual information weights through an attention-based fusion mechanism, enhancing the accuracy and robustness of recommendation. Compared with embedding-based methods such as MF [13], KGCFRec is good at modeling long-distance information. Compared with path-based methods such as CKE [30], the task oriented complex design is eliminated; Compared to GNN-based methods [17,18,29,31,32,33], efficient modeling knowledge and the collaborative perception of dual-channel information can be adaptively fused. The substantial benefits of information fusion are also demonstrated by the end-to-end framework architecture.

3. Preliminaries

The aim of this work is to study KG-augmented recommendation, where the recommender system uses knowledge graph information to improve performance and make recommendations for items to users. We begin by describing the underlying structural data, including the knowledge graph and the user–item interactions graph, followed by a formulation of our task.

Knowledge Graph: The power of the KG approach lies in its ability to store structured auxiliary data—such as item attributes, taxonomies, or external common-sense knowledge—in the form of a directed graph, denoted as $KG=<E,R,T>$. Here, E and T represent the sets of entities, while R signifies the set of relations. We formally organize this side information as a knowledge graph, defined as $G_{KG}=\{(h,r,t)\mid h,t\in ϵ,r\in R\}$, where $ϵ$ encompasses real-world entities, and R comprises the set of relations. For instance, the tuple (Tom Hanks, ActorOf, Forrest Gump) indicates that Tom Hanks is an actor in the film Forrest Gump. By mapping items to KG entities ($I\subset ϵ$), item i can be correlated with an entity $h,t$ within the KG. Consequently, the KG can provide detailed profiles of items and offer supplementary information to the interaction data.

User-Item Interaction Graph: In this work, we focus on implicit feedback within recommendations, where users’ historical interactions, indicative of their preferences, are indirectly expressed through actions such as viewing, clicking, and purchasing. We conceptualize these interaction data as a user–item bipartite graph, denoted as $G_{UI}$, which is formalized as $\{(u,d_{ui},i)\mid u\in U,i\in I\}$. Here, $U=\{u_1,\dots ,u_N\}$ represents the set of users, and $I=\{i_1,\dots ,i_M\}$ represents the set of items. A link with $d_{ui}=1$ signifies an interaction between user u and item i, whereas $d_{ui}=0$ indicates the absence of such interaction.

Task Description: Hence, for the KG-based recommendation approach we are introducing, we formalize the recommendation task as follows:

$$\mathbb{F}\left(G\mid {\mathsf{\Theta }}_{\mathbb{F}}\right)\to \mathbb{F}\left(G_{KG},G_{UI}\mid {\mathsf{\Theta }}_{\mathbb{F}}\right)\to \widehat{y}$$

(1)

where ${\mathsf{\Theta }}_{\mathbb{F}}$ represents the parameters of the model that need to be optimized.

Input: A collaborative knowledge graph G, which encompasses the knowledge graph $G_{KG}$ and the user–item interactions graph $G_{UI}$.

Output: A prediction score ${\widehat{y}}_{ui}$, representing the likelihood of user u engaging with item i.

4. The Proposed Methodology

We now introduce our model, a GNN-based framework designed for KG-augmented recommendation that operates on a collaborative knowledge graph. As depicted in Figure 1, KGCFRec comprises three principal components: (a) knowledge-aware representation learning, (b) collaborative-aware representation learning, and (c) adaptive feature fusion.

GNNs progressively enrich the representation of a node by integrating information from its neighboring nodes. As a result, the node’s representation after k iterations encapsulates the structural nuances and attributes of nodes within its k-hop vicinity. This approach mitigates the challenges posed by data sparsity. The pivotal aspect of GNN-based recommender systems lies in the design of the neighborhood aggregation strategy. Specifically, the representation vector of an entity is refined by iteratively aggregating and transforming the representations of its multi-hop neighbors.

However, it is challenging to accurately model user and item representations by directly fusing graph and collaborative features in a coarse-grained manner without considering the distributional gap between them. We leverage GNN layers to extract user and item representation from both the knowledge graph and collaborative information channels. Additionally, we introduce a novel attention mechanism to adaptively refine these representations, thereby enhancing the performance of recommendation tasks.

4.1. Knowledge-Aware Representation Learning

Knowledge-aware representation learning is depicted in Figure 2. We define $G_{KG}=\{(h,r,t)\mid h,t\in ϵ,r\in R\}$, where r denotes the relation from entity h to entity t. Given that entities are connected through one or more varying relations, we contend that relation modeling is as crucial as entity modeling. GNNs excel at modeling entities and relationships, prompting us to adopt a GNN-based approach for the representation learning of entities and relations. An entity can be characterized by different relations across multiple KG triplets, illustrating the content similarity among items from various perspectives. For instance, the movie “Forrest Gump” can be characterized by its director, Robert Zemeckis, or its star, Tom Hanks. The process of information aggregation within the knowledge graph, as facilitated by our model, is defined in Equation (2):

$$e_i^{KG(k+1)}=f_{\mathrm{KG}}\left(\left\{\left(e_i^{KG\left(k\right)},e_r,e_v^{KG\left(k\right)}\right)\mid (r,v)\in {\mathcal{N}}_i^{KG}\right\}\right)$$

(2)

where $f_{\mathrm{KG}}\left(.\right)$ is an aggregation function that aggregates the features of neighbors of item i, ${\mathcal{N}}_i^{KG}$ is a set of the neighbor of item i in the knowledge graph, which consists of some tuples, including relation and entity. Here, we implement $f_{\mathrm{KG}}\left(·\right)$ as Equation (3):

$$e_i^{KG(k+1)}=\frac{1}{\parallel {\mathcal{N}}_i^{KG}\parallel }\sum _{(r,v)\in {\mathcal{N}}_i^{KG}}{e}_r\odot e_v^{KG\left(k\right)}$$

(3)

where $e_v^{KG\left(k\right)}$ is the embedding including knowledge information of entity v of k layer, $e_r$ is the embedding of relation r, ⊙ is the element-wise product. The reason of design relational message $e_r\odot e_v^{KG\left(k\right)}$ is to obtain new $e_i^{KG(k+1)}$, including different meanings of $e_r$ and $e_v^{KG\left(k\right)}$.

Additionally, modeling the user’s knowledge graph embedding is treated consistently, which is defined as Equation (4):

$${\mathrm{e}}_u^{KG(k+1)}=\frac{1}{\parallel {\mathcal{N}}_u^{KG}\parallel }\sum _{i\in {\mathcal{N}}_u^{KG}}{\mathrm{e}}_i^{KG\left(k\right)}$$

(4)

where ${\mathcal{N}}_u^{KG}$ is a set of the neighbor of user u in the knowledge graph.

4.2. Collaborative-Aware Representation Learning

Collaborative-aware representation learning is depicted in Figure 3. The collaborative information [17] holds significant value in characterizing user preferences, assuming that users with similar behaviors tend to share similar preferences for items. Consequently, we refine this collaborative information from the user–item interactions (UI) graph. In Equations (5) and (6), we leverage a GNN with K layers to harmonize the information within the UI graph. Specifically, for the k-th layer, we update the user representation $e_u^{UI\left(k\right)}$ and the item representation $e_i^{UI\left(k\right)}$ as follows:

$$e_u^{UI(k+1)}=f_{\mathrm{UI}}\left(e_i^{UI\left(k\right)},\left\{e_i^{UI\left(k\right)}:i\in {\mathcal{N}}_u^{UI}\right\}\right)$$

(5)

$$e_i^{UI(k+1)}=f_{\mathrm{UI}}\left(e_u^{UI\left(k\right)},\left\{e_u^{UI\left(k\right)}:u\in {\mathcal{N}}_i^{UI}\right\}\right)$$

(6)

where $f_{\mathrm{UI}}$ serves as an aggregation mechanism that compiles the features of neighboring nodes within the user–item interactions graph. $e_u^{UI\left(k\right)}$ and $e_i^{UI\left(k\right)}$ represent the k-th layer’s representation, which incorporates the collaborative information of user u and item i, respectively. ${\mathcal{N}}_u^{UI}$ denotes the set of item neighbors for user u, while ${\mathcal{N}}_i^{UI}$ signifies the set of user neighbors for item i.

The essence of graph convolutional networks (GCNs) lies in the design of the $f_{\mathrm{UI}}$ function. In contrast to the prevalent approaches that integrate feature transformation or nonlinear activation within the $f_{\mathrm{UI}}$ function, we streamline the GCN design to enhance its conciseness and suitability for recommendation tasks, like LightGCN [34]. The graph convolution operation is delineated in Equations (7) and (8):

$${\mathrm{e}}_u^{UI(k+1)}=\frac{1}{\parallel {\mathcal{N}}_u^{UI}\parallel }\sum _{i\in {\mathcal{N}}_u^{UI}}{\mathrm{e}}_i^{UI\left(k\right)}$$

(7)

$$e_i^{UI(k+1)}=\frac{1}{\parallel {\mathcal{N}}_i^{UI}\parallel }\sum _{u\in {\mathcal{N}}_i^{UI}}{e}_u^{UI\left(k\right)}$$

(8)

It is important to highlight that our aggregation process exclusively considers the neighboring nodes that are directly connected and does not incorporate the target node itself. This is because we aim to generate a comprehensive embedding representation by combining the representations from each layer. Following the application of the $f_{\mathrm{UI}}$ function, the user and item representations, which encapsulate collaborative information, can be derived.

4.3. Adaptive Feature Fusion

It is crucial to harmoniously integrate the representations that encapsulate both knowledge and collaborative information to derive a unified final representation for users and items. In scenarios where user–item interactions are plentiful, the model can benefit from learning a wealth of collaborative insights. Conversely, in cases where interactions are scarce, the model should prioritize capturing the knowledge-rich information within the graph.

To fuse the knowledge graph and collaborative information, we define a fusion function f, shown in Equations (9) and (10).

$$e_i^{\left(k\right)}=f_i\left(e_i^{KG\left(k\right)},e_i^{UI\left(k\right)}\right)$$

(9)

$$e_u^{\left(k\right)}=f_u\left(e_u^{KG\left(k\right)},e_u^{UI\left(k\right)}\right)$$

(10)

Specifically, in this paper, we implement the fusion function f as Equations (11) and (12):

$$\begin{array}{c}\hfill e_i^{\left(k\right)}=Att\left(e_i^{KG\left(k\right)},e_i^{(k-1)}\right)e_i^{KG\left(k\right)}+Att\left(e_i^{UI\left(k\right)},e_i^{(k-1)}\right)e_i^{UI\left(k\right)}\end{array}$$

(11)

$$\begin{array}{c}\hfill e_u^{\left(k\right)}=Att\left(e_u^{KG\left(k\right)},e_u^{(k-1)}\right)e_u^{KG\left(k\right)}+Att\left(e_u^{UI\left(k\right)},e_u^{(k-1)}\right)e_u^{UI\left(k\right)}\end{array}$$

(12)

Hence, we introduce an attention score $Att\left(·\right)$ to distinguish the importance of knowledge and collaborative information in Equations (13)–(16):

$$Att\left(e_i^{KG\left(k\right)},e_i^{(k-1)}\right)=\frac{exp\left(e_i^{(k-1)}·e_i^{KG\left(k\right)}\right)}{e_i^{\left(k\right)}}$$

(13)

$$Att\left(e_i^{UI\left(k\right)},e_i^{(k-1)}\right)=\frac{exp\left(e_i^{(k-1)}·e_i^{UI\left(k\right)}\right)}{e_i^{\left(k\right)}}$$

(14)

$$Att\left(e_u^{KG\left(k\right)},e_u^{(k-1)}\right)=\frac{exp\left(e_u^{(k-1)}·e_u^{KG\left(k\right)}\right)}{e_u^{\left(k\right)}}$$

(15)

$$Att\left(e_u^{UI\left(k\right)},e_u^{(k-1)}\right)=\frac{exp\left(e_u^{(k-1)}·e_u^{UI\left(k\right)}\right)}{e_u^{\left(k\right)}}$$

(16)

Specifically, we define $e_i^{\left(k\right)}$ and $e_u^{\left(k\right)}$ in Equations (17) and (18):

$$e_i^{\left(k\right)}=exp\left(e_i^{(k-1)}·e_i^{KG\left(k\right)}\right)+exp\left(e_i^{(k-1)}·e_i^{UI\left(k\right)}\right)$$

(17)

$$e_u^{\left(k\right)}=exp\left(e_u^{(k-1)}·e_u^{KG\left(k\right)}\right)+exp\left(e_u^{(k-1)}·e_u^{UI\left(k\right)}\right)$$

(18)

Upon completing L layers of representation learning, we acquire the user u and item i representations across layers 0 to L. These representations are then aggregated to construct the ultimate user and item embeddings. Specifically, we aggregate the user (item) embeddings from each layer to derive the comprehensive final representation. This aggregation is achieved by summing the embeddings from all layers:

$$e_i^{*}=\frac{1}{L+1}\sum _{j=0}^L{e}_i^{\left(j\right)}$$

(19)

$$e_u^{*}=\frac{1}{L+1}\sum _{j=0}^L{e}_u^{\left(j\right)}$$

(20)

The ultimate representations encapsulate various semantics from each layer, as they are derived through the layer combination defined by Equations (19) and (20).

4.4. Model Optimization

Following the integration of the layers, the model’s predictive function is formulated as the dot product between the user’s and item’s final representations, as outlined in Equation (21):

$${\widehat{y}}_{ui}=e_u^{*T}{e}_i^{*}$$

(21)

${\widehat{y}}_{ui}$ is employed to estimate the probability of a user adopting an item, which serves as the ranking metric for generating recommendations.

To optimize the model, we utilize the Bayesian personalized ranking (BPR) loss [35], a pairwise loss function that encourages the model to rank observed entries higher than unobserved ones, as depicted in Equation (22):

$${\mathcal{L}}_{\mathrm{BPR}}=\sum _{(u,i,j)\in O}-ln\sigma \left({\widehat{y}}_{ui}-{\widehat{y}}_{uj}\right)$$

(22)

where $O=\left\{(u,i,j)\mid (u,i)\in O^{+},(u,j)\in O^{-}\right\}$ represents the training dataset composed of observed interactions $O^{+}$ and unobserved counterparts $O^{-}$; $\sigma \left(·\right)$ denotes the sigmoid function. Prior to the commencement of training, the items with which the user has interacted are chosen to form a positive sample with the user. For negative samples, we randomly select items that the user has not interacted with, maintaining a ratio of 1:1 to the positive samples.

To mitigate overfitting, we minimize the objective function outlined in Equation (23) to train the model’s parameters effectively.

$${\mathcal{L}}_{\mathrm{KGCFRec}}={\mathcal{L}}_{\mathrm{BPR}}+{\lambda }_{\mathsf{\Theta }}{\parallel \mathsf{\Theta }\parallel }^2$$

(23)

where ${\lambda }_{\mathsf{\Theta }}{\parallel \mathsf{\Theta }\parallel }^2$ is the regularization term. Upon the computation of the loss, the parameters of the KGCFRec model are updated via the gradient descent algorithm Adam [36]. Training ceases after a predefined number of iterations or when the loss does not decrease for a specified number of iterations.

Once training is complete, the process of item recommendation for a user unfolds as follows: the user’s embedding vector and the embedding vectors of all items with which the user has not interacted are input into the prediction function ${\widehat{y}}_{ui}$ in pairs. The preference scores for all uninteracted items are then determined, and the top N items with the highest scores are selected for recommendation to the user after ranking.

5. Experimental Setup

In this section, we delineate the dataset utilized for our experiments, the baseline approach and the evaluation metrics employed for comparison, and provide an overview of the experimental setup.

5.1. Datasets

To validate the efficacy of the model presented in this paper, we employ three widely recognized recommendation datasets across distinct domains. These datasets include Last-FM, a music recommendation dataset; Amazon-book, a book recommendation dataset; and Alibaba-iFashion, a fashion clothing recommendation dataset. A comprehensive description of the fundamental aspects of these datasets is provided below.

Amazon-book is a subset of the Amazon-review dataset, a well-known product recommendation dataset published by Amazon.com. It pertains specifically to book recommendations, where the items are books and the user ratings represent the interaction actions. In this study, we treat high user ratings as positive samples, thereby transforming display feedback into implicit feedback. To ensure the quality of the dataset, we employ a preprocessing method to eliminate user and item data with fewer than 10 interactions. Amazon-book is openly available on Github at https://nijianmo.github.io/amazon/index.html, accessed on 15 May 2023.

Last-FM [37] is derived from the Last.fm online music system and represents music listening habits. Each music piece in the dataset is considered an item, and users’ music listening patterns constitute the interactions. The data used in this study span the period from January 2015 to June 2015. Last-FM is openly available from the Last.fm website, http://www.lastfm.com, accessed on 18 May 2023.

Alibaba-iFashion is a fashion apparel recommendation dataset sourced from Alibaba’s e-commerce platform. The various fashion outfits are treated as items, and user behaviors such as purchases and clicks are considered interactions. The knowledge graph for this dataset is constructed using the attributes of the fashion outfits. Alibaba-iFashion is openly available on Github at https://github.com/renesemela/lastfm-dataset-2020, accessed on 20 May 2023.

The three datasets used in this paper are open source and can be accessed by visiting their official links. A detailed statistical comparison of these three datasets is presented in

Table 1. Statistics of the datasets.

Dataset	#Users	#Items	#Interactions	#Entities	#Relations	#Triplets
Amazon-book	70,679	24,915	847,733	88,572	39	2,557,746
Last-FM	23,566	48,123	3,034,796	58,266	9	464,567
Alibaba-iFashion	114,737	30,040	1,781,093	59,156	51	279,155

. This comparison reveals that the datasets vary in terms of various data metrics, which will facilitate the assessment of the model’s effectiveness proposed in this paper.

In the context of each dataset, we randomly select 80% of the historical interaction data for each user to constitute the training set. The remaining 20% is designated as the test set. Within the training set, we further segregate 10% as the validation set, which aids in refining the model’s hyper-parameters by evaluating their impact on performance.

5.2. Baseline Methods

This subsection outlines the baseline methods that are used for comparative evaluation against KGCFRec. In this study, these baselines are categorized into three distinct groups: solely utilizing user–item interaction data without incorporating knowledge graph information (MF); embedding-based methods (CKE); and graph-neural-network-based methods (KGAT, KGNN-LS, CKAN, R-GCN, Simple-HGN, KGCN). The specific details of these baseline methods are detailed below:

• MF [13] is a conventional matrix factorization approach that exclusively relies on user–item interaction data for recommendations. MF projects users and items into a latent vector space and refines their vector representations by decomposing the co-occurrence matrix, thereby capturing the collaborative signals between user–item interactions.
• CKE [30] is a notable example of an embedding-based knowledge graph recommendation. It learns the embedding vector representation of items within the knowledge graph through the knowledge graph embedding algorithm TransR and integrates these item embeddings into the matrix factorization framework.
• KGAT [17] is a state-of-the-art knowledge graph recommendation algorithm that leverages graph neural network technology. It implements a neighbor aggregation mechanism based on an attention mechanism, enabling the selective propagation of node information. KGAT operates on a collaborative knowledge graph that includes a user–item interaction graph and a knowledge graph, outperforming other methods in this setting.
• KGNN-LS [29] is another knowledge graph recommendation algorithm based on graph neural networks. It differs from KGAT in that it does not operate on a collaborative knowledge graph but rather uses the transformation of knowledge graph information into a user–item interaction graph. KGNN-LS takes into account the propagation of user preferences across the knowledge graph.
• CKAN [18] is an extension of the KGNN-LS model. It differs from KGNN-LS in that it propagates and aggregates information on the user–item interaction graph and the knowledge graph separately, and it designs two distinct aggregation mechanisms.
• R-GCN [32] is a graph neural network framework. It treats the relationships in the knowledge graph as distinct information dissemination channels and utilizes different channels for aggregating node information based on their relationships.
• Simple-HGN [33] is a novel, simple, and effective method for modeling heterogeneous graphs. The model is applicable to knowledge-aware recommendation and has shown promising results.
• KGCN [31] is a recommender algorithm that leverages knowledge reasoning optimization. It involves two key steps: knowledge graph completion, which uses a reasoning algorithm to enrich the graph with better user interaction and semantic information, and the application of KGCN to capture higher-order features for improved personalized recommendations through embedding and aggregation of the completed knowledge graph.

5.3. Evaluation Metrics

The model proposed in this study is assessed as a top-n recommendation model. During the evaluation phase, two prevalent evaluation metrics in contemporary recommendation systems are employed: sample evaluation metrics [29] and all-ranking evaluation metrics [38]. Sample evaluation entails presenting a user with a recommendation list and selecting a predefined number of negative samples for final evaluation, rather than predicting all potential negative samples. Conversely, the all-ranking evaluation approach involves predicting all unadopted items by a user and then providing a top-n list of recommendations for evaluation. While sample evaluation is more expedient, all-ranking evaluation offers a more thorough and precise assessment. In this paper, we opt for the all-ranking evaluation method.

To evaluate the efficacy of the model, two commonly employed evaluation metrics, recall [38] and normalized discounted cumulative gain (NDCG) [39], are employed. Recall is defined as shown in Equation (24).

$$Recall=\frac{\#TP}{\#TP+\#FN}$$

(24)

The recall metric is typically assessed by setting a fixed number of items, denoted as K, that each user is recommended. This is represented as Recall@K. Recall is a prevalent metric for assessing the performance of a ranking model, quantifying the proportion of correct recommendations to the total number of correct items available.

The discounted cumulative gain (DCG) metric is introduced to define the normalized discounted cumulative gain (NDCG), as depicted in Equation (25).

$$\mathrm{DCG}@\mathrm{K}=\sum _i^K\frac{2^{r\left(i\right)}-1}{{log}_2(i+1)}$$

(25)

where $r\left(i\right)$ represents the relevance score of the item at position r in the recommendation list. In this study, we define $r\left(i\right)$ as 1 if the recommended item has been interacted with by the user; otherwise, $r\left(i\right)$ is set to 0. Subsequently, the normalized discounted cumulative gain (NDCG) is formulated as in Equation (26).

$$\mathrm{NDCG}@\mathrm{K}=\frac{DCG@K}{IDCG@K}$$

(26)

The ideal discounted cumulative gain (IDCG@K) represents the optimal discounted cumulative gain scenario, where the items in the recommendation list are those that the users have interacted with. NDCG places greater emphasis on the sequential ordering of correctly classified samples, ensuring that they are ranked higher than the recall metric.

5.4. Parameter Settings

We developed our KGCFRec model using PyTorch. Training was conducted on a Nvidia GeForce RTX 3060 with Adam as the optimizer, with a fixed batch size of 1024. To ensure a level playing field with the baseline methods, we standardized the embedding dimension $\left(d\right)$ at 64. The model parameters were initialized using Xavier [40]. We use the default model configuration for all baseline methods, keeping the training strategy consistent. At the same time, the grid search traversed the hyper parameters as the parameter of the comparison method to ensure a fair comparison. A grid search was conducted to optimize hyper-parameters: the learning rate $\left(\sigma \right)$ was varied within $\{5\times {10}^{-2},5\times {10}^{-3},5\times {10}^{-4}\}$, and the $L_2$ normalization coefficient ${\lambda }_{\mathsf{\Theta }}$ was explored in $\{{10}^{-3},{10}^{-4},{10}^{-5}\}$. To enhance the model’s generalization capabilities, we employed dropout techniques, specifically node dropout and edge dropout, with their respective ratios adjusted in $\{0.0,0.1,0.3,0.5,0.7\}$. Each dataset underwent 600 epochs of training, and the parameters yielding the best performance are tabulated in

Table 2. The parameters of best result.

Dataset	d	$\mathit{\delta }$	${\mathit{\lambda }}_1$	Node Dropout Rate	Edge Dropout Rate	#Triplets
Amazon-book	64	$5\times {10}^{-4}$	${10}^{-4}$	0.5	0.1	2,557,746
Last-FM	64	$5\times {10}^{-4}$	${10}^{-4}$	0.5	0.1	464,567
Alibaba-iFashion	64	$5\times {10}^{-4}$	${10}^{-4}$	0.3	0.3	279,155

. The impact of the depth of the graph neural network layers is discussed in Section 6.

6. Results

In this section, we present the results of our experiments. Initially, we compare the performance of our model against that of the baseline methods. Next, we conduct an ablation study to illustrate the efficacy of our model. Finally, we conduct hyper-parameter studies to reveal the sensitivity of each module of the model.

6.1. Recommendation Performance

We report the results of the baseline methods and our model in

Table 3. Overall performance comparison.

Model	Amazon-Book		Last-FM		Alibaba-iFashion
	Recall@20	NDCG@20	Recall@20	NDCG@20	Recall@20	NDCG@20
MF [13]	0.13	0.0678	0.0724	0.0617	0.1095	0.067
CKE [30]	0.1342	0.0698	0.0732	0.063	0.1103	0.0676
KGAT [17]	0.1487	0.0799	0.0873	0.0744	0.103	0.0627
KGNN-LS [29]	0.1362	0.056	0.088	0.0642	0.1039	0.0557
R-GCN [32]	0.122	0.0646	0.0743	0.0631	0.086	0.0515
CKAN [18]	0.1442	0.0698	0.0812	0.066	0.097	0.0509
Simple-HGN [33]	0.1587	0.0854	0.0917	0.0797	0.0952	0.0566
KGCN [31]	0.1592	0.0864	0.0919	0.0807	0.0971	0.0574
KGCFRec	0.1646	0.0880	0.0933	0.0826	0.1231	0.0769
(%) Imp.	3.39	1.85	1.52	2.35	11.6	13.76

.

In Table 3, the symbol Imp. stands for the relative improvement of the top-performing method (bolded) compared to the most robust baseline methods (underlined). Table 3 reveals the following key insights:

• KGCFRec surpassed the baseline model in all evaluation metrics across all datasets. When contrasted with the traditional collaborative filtering model MF, which relies solely on collaborative data without leveraging knowledge graph information, KGCFRec’s superior performance underscores the significance of integrating knowledge graph data. In comparison with graph neural network models like KGAT and KGNN-LS, which are more focused on encoding knowledge graph information, KGCFRec demonstrates its strength in incorporating collaborative data. Against the CKAN model, which also models both knowledge graph and collaborative data, KGCFRec achieves superior results due to its attention-based information fusion model.
• The experimental findings reveal that MF, which solely relies on user–item interaction data without knowledge graph information, outperforms models like KGAT, KGNN-LS, R-GCN, and CKAN on the Alibaba-iFashion dataset in terms of evaluation metrics. This observation suggests that models that predominantly rely on knowledge graph information without integrating collaborative data may not achieve the expected performance. In contrast, KGCFRec consistently outperforms MF, underscoring the importance of incorporating both collaborative and knowledge information.
• Analyzing the relative gains of KGCFRec over the best baseline across the Amazon-book, Alibaba-iFashion, and Last-FM datasets, the largest gains were observed on the Amazon-book dataset, with 3.39% (Recall@20) and 1.85% (NDCG@20). On the Last-FM dataset, the gains were 1.52% (Recall@20) and 2.35% (NDCG@20). On the Alibaba-iFashion dataset, the gains were 11.6% (Recall@20) and 13.76% (NDCG@20). These differences could be attributed to the varying quality and distribution of the datasets, which may explain the varying gains KGCFRec achieves in fusing collaborative and knowledge information.

6.2. Ablation Study

To further corroborate the efficacy of the attention-based fusion design for fusing knowledge and collaborative information, as presented in Section 4 of this paper, this subsection performs ablation studies on the proposed design. Specifically, the performance of the two models is contrasted across three datasets: Alibaba-iFashion, Last-FM, and Amazon-book. This comparison is achieved by substituting the attention-mechanism-based fusion with a straightforward summation-based fusion approach. In this paper, the altered KGCFRec model is referred to as the KGCFRec-a model. The experimental parameters of the KGCFRec-a model are tuned identically to those of the KGCFRec model, and the experimental outcomes are presented in

Table 4. Impact of the attention mechanism.

Model	Amazon-Book		Last-FM		Alibaba-iFashion
	Recall@20	NDCG@20	Recall@20	NDCG@20	Recall@20	NDCG@20
KGCFRec	0.1646	0.0880	0.0933	0.0826	0.1231	0.0769
KGCFRec-a	0.1583	0.0859	0.0887	0.0786	0.1188	0.074

.

6.3. Hyper-Parameter Studies

In this subsection, we conduct an empirical investigation into the impact of three hyper-parameters of KGCFRec: the embedding dimension, the number of graph convolutional layers, and the dropout ratio.

Given that KGCFRec represents users and items as embedding vectors, the size of these embeddings significantly influences the model’s expressive capabilities. To examine this influence, we varied the embedding size of KGCFRec across the Amazon-book dataset, setting it to 8, 16, 32, and 64, respectively, while the other parameters were determined through grid search. The final experimental outcomes for each embedding size are illustrated in Figure 4.

From Figure 4, it is evident that as the embedding size increases, the recommendation performance of KGCFRec improves, albeit with diminishing returns. This suggests that a judicious increase in embedding size can enhance the model’s expressiveness. However, an excessive increase in size may also complicate the model’s convergence, underscoring the importance of selecting an appropriate embedding size based on the specific characteristics of the data in practical applications.

As KGCFRec is a graph-neural-network-based knowledge graph recommendation algorithm, the graph convolutional layers play a pivotal role in aggregating information. Consequently, the number of these layers is a crucial hyper-parameter that influences the recommendation efficacy. In the experiments conducted in this section, we varied the number of graph convolutional layers in KGCFRec to 1, 2, and 3, respectively, and identified the optimal combination of other parameters through grid search. The experimental results on the Amazon-book and Alibaba-iFashion datasets are presented in Figure 5 and Figure 6.

Figure 5 and Figure 6 indicate that both the Amazon-book and Alibaba-iFashion datasets achieve optimal performance with three graph convolutional layers in terms of Recall@20 and NDCG@20. The general trend suggests that the recommendation performance improves with an increase in the number of convolutional layers, underscoring the soundness of the graph neural network design at the core of KGCFRec. By employing multiple layers of graph convolution, KGCFRec can effectively gather information from more distant nodes, facilitating the learning of a more comprehensive embedding representation for users and items.

Dropout [41] is a technique used to mitigate the risk of overfitting in neural network models. This subsection explores the impact of varying dropout rates on the model’s performance using the Last-FM dataset. The results are presented in Figure 7.

Figure 7 suggests that the dropout ratio has a significant influence on the model’s performance. For node dropout, increasing the dropout ratio leads to a notable improvement in model performance. Conversely, for edge dropout, increasing the dropout ratio has a detrimental effect on the model’s performance, possibly due to the disruption of connections between nodes, which can hinder the model’s convergence. These findings indicate that the judicious application of dropout can effectively reduce the training complexity of the model, aiding in the prevention of overfitting and enhancing the model’s generalization capabilities.

7. Conclusions

In this paper, we have introduced a method for personalized recommendation that fuses collaborative and knowledge information from a knowledge graph. To achieve this, we devised two graph neural networks to collect collaborative information from the user–item interactions graph and knowledge information from the knowledge graph. This design enables the differentiation of the two types of information, thereby facilitating the creation of more robust user and item representations. We then employed an attention mechanism to merge the collaborative and knowledge information embeddings, resulting in a more judicious weighted combination. Extensive experiments conducted on three real-world datasets validate the rationality and efficacy of KGCFRec. The experiment proves that knowledge graph and collaborative information can enhance each other to improve recommendation performance, and further confirms that it is feasible to use side information to improve recommendation performance. Ablation and hyper-parameter studies further corroborate the soundness of the model’s design. In future work, we intend to investigate the further leveraging of the background knowledge of large language models to improve KGCFRec performance while implementing interpretable recommendations.