Session-Enhanced Graph Neural Network Recommendation Model (SE-GNNRM)

Abstract

Session-based recommendation aims to predict anonymous user actions. Many existing session recommendation models do not fully consider the impact of similar sessions on recommendation performance. Graph neural networks can better capture the conversion relationship of items within a session, but some intra-session conversion relationships are not conducive to recommendation, which requires model learning more representative session embeddings. To solve these problems, an improved session-enhanced graph neural network recommendation model, namely SE-GNNRM, is proposed in this paper. In our model, the complex transitions relationship of items and more representative item features are captured through graph neural network and self-attention mechanism in the encoding stage. Then, the attention mechanism is employed to combine short-term and long-term preferences to construct a global session graph and capture similar session information by using a graph attention network fused with similarity. In order to prove the effectiveness of the constructed SE-GNNRM model, three public data sets are selected here. The experiment results show that the SE-GNNRM outperforms the existing baseline models and is an effective model for session-based recommendation.

1. Introduction

With the popularization of the Internet and the advent of the information age, the information that Internet users face and process every day is increasing exponentially. In order to select the content of interest, the recommendation system emerges as the times require. Most previous research work is based on uninterrupted user history for personalized recommendations, but in many services, such as YouTube and TikTok, due to limited storage resources, it is not possible or unnecessarily to track the user identity behavior. In other words, user profiles and long-term historical interactions are not available; only short-term session data from anonymous users are provided. For this scenario, research on session-based recommendation algorithms becomes more and more important [1]. The behavior information of user clicks generated within a certain time interval is a session sequence. In order to process this behavior information, the session-based recommendation system extracts the dependencies of multiple click items in the sequence and captures the importance of sequence prediction [2]. The interests of users in a certain period of time are the same and similar, which is important for the recommendation field, so the recommendation algorithms based on session sequence have received more attention in recent years [3].

At present, many session-based recommendation models have been born. In the early days, the session recommendation of Markov chains was mainly based on the user’s previous behavior to predict the user’s next behavior. With the development of deep learning, recurrent neural network (RNN) methods are gradually applied to session recommendation; such methods input user session click sequences into RNN to generate recommendation results. RNNs are effective to some extent, but most existing RNN-based models have limited representational power in capturing complex transitions between items. Recently, recommendation algorithms based on graph neural networks (GNN) have also become popular. Wu et al. [4] adopted the GNN method to recommend users, converting session data into a graph structure and combining multiple layer gated recurrent units (GRU) to capture users’ long-term interests and current preferences. This type of method inputs the session click item matrices constructed by the session graph and the session click sequence into the GNN for sequence modeling, which solves the problem of weak inter-session dependencies, and its recommendation effect is greatly improved, but it is not well filtered against useless dependencies. With the increasing popularity of deep neural networks, attention networks have been widely developed in many fields (e.g., computer vision, natural language processing, and recommendation systems), using self-attention mechanisms derived from the field of natural language processing to process session algorithms for sequences that are also emerging.

In order to overcome the above limitations, this paper combines graph neural networks (GNN for brevity), self-attention networks (SAN for brevity), soft attention mechanisms, and other related technologies to propose a session-enhanced graph neural network recommendation model (denoted as SE-GNNRM). This model includes an item embedding layer, a session embedding layer, a session interaction layer, and a prediction layer. The item embedding layer includes two modules, GNN and SAN. The item embedding obtained by GNN and SAN is combined through a joint function. In the session embedding layer, the output item embedding of the item embedding layer is combined with the location information, and then the session embedding is learned using attention. The session embedding at this time only represents a single session embedding, and similar sessions may have similar preferences. To solve this problem, this model designs a dynamic global session interaction layer for learning similar session embeddings. After obtaining the global session embedding, the candidate item scores are calculated through the prediction layer, and the final recommendation result is obtained.

The main work contributions of this paper are described as follows.

• An improved session-enhanced graph neural network recommendation model based on a graph neural network and self-attention network, namely SE-GNNRM, is proposed to solve the problem of intra-session conversion relationships and representative session embeddings.
• The graph neural network and self-attention mechanism are employed to capture the complex transitions relationship of items and more representative item features in the encoding stage.
• The attention mechanism is employed to combine short-term and long-term preferences to construct a global session graph and capture similar session information by using a graph attention network fused with similarity.
• For two evaluation metrics, extensive experiments on real-world data sets show that SE-GNNRM significantly outperforms existing baseline methods.

2. Related Work

2.1. Traditional Method

The early S-POP calculates the frequency of each item appearing in the session by counting the user’s interactive behavior in the current session and recommends the top k items to the user. In the case that the current historical items are not full, the most popular items can be extracted from historical behavior as a supplement. S-POP considers the difference of each session but only considers the frequency of items appearing in the current session, which makes the session embedding learned by the model too simple, and it is difficult to fully represent the user’s interest preference. Item-KNN [5] uses cosine similarity to define the degree of similarity between sessions and recommends items that are similar to previous clicks.

Shani et al. [6] consider the long-term effects and expected value Markov decision process; this method loses the balance between users’ general preference and sequential behavior. Rendle et al. [7] proposed a hybrid method considering the combination of matrices factorization and collaborative filtering, embedding Markov chains in Euclidean space, calculating the transition probability between interactions according to Euclidean distance, and then inferring the user’s next action based on the previous action. Because the method assumes that the current item only depends on one or a few adjacent items, the recommendation system based on the Markov chain focuses on the order conversion of two adjacent items, ignoring the interaction between multiple items. Markov chain-based methods are no longer suitable for today’s recommendation scenarios.

2.2. Methods Based on Recurrent Neural Network (RNN)

With the rapid development of deep learning technology, many session recommendation algorithms based on deep learning have been born. The recurrent neural network (RNN) is a mainstream deep learning model for modeling sequential data [8]. Hidasi et al. [9] introduce RNN into the recommendation algorithm, regard the interaction of a session clicking a series of items as a sequence, and input the sequence into the model to predict the next clicked item. Li et al. [10] propose a neural attention recommendation model (NARM), which combines an attention mechanism with a multi-layer gated recurrent unit (GRU) encoder, which is a variant of RNN, in order to capture more representative session embeddings. The model uses the bilinear matching scheme based on the unified session embedding to calculate the recommendation score of each candidate item and then achieve better recommendation performance. The downside is that a session may contain multiple user selections and even noise, so they do not generate all the correct dependencies. Existing session-based recommendation methods mainly focus on the current session, ignoring collaboration information in historical sessions, which are sessions generated by other users and reflect similar user intentions to the current session. In response to this problem, Wang et al. [11] propose a collaborative session-based recommendation approach with parallel memory modules (CSRM), which uses a memory network to encode the current session, taking the information of the current session and neighbor sessions into account based on session suggestions to better predict the intent of the current session. One problem with session-based recommendations is how to generate accurate recommendations at the beginning of a session before knowing the user’s current interests at the start of the session. Song et al. [12] transformed the short-session problem into a few-shot learning (FSL) problem, explicitly designing a local module based on GRU encoding and a global module considering similar historical sessions, and achieved state-of-the-art performance.

2.3. Methods Based on Graph Neural Network (GNN)

As an emerging artificial intelligence technology, graph learning (GL) has developed rapidly and has shown great promise in recent years. GL has a strong ability to learn relational data, and GL-based recommendation systems (RS) have been widely proposed and extensively studied. Most of the data in RS have a graph structure in nature, so GL has a positive impact on recommendation. Wu et al. [4] applied GNN to a sequence recommendation algorithm for the first time and proposed a GNN-based session recommendation model (SR-GNN). SR-GNN first models the session sequence information as a directed graph and uses gated GNN to capture the complex relationship and dependency transformation between each node in the session to obtain the node embedding vector. Then, SR-GNN uses the attention mechanism to obtain the global session embedding and finally combines the long- and short-term interest embedding to obtain the final embedding. To improve the embedding of session sequences, Xu et al. [13] propose an improved GNN-based graph contextual self-attention network model (GC-SAN). It exploits the complementarity between the self-attention network (SAN) and GNN to improve recommendation performance. Qiu et al. [14] propose a weighted graph attention network model as an item feature encoder, which helps transfer information between items more efficiently by learning to assign different weights to different neighbors. Kim et al. [15] proposed a content-rich graph neural network with attention, which includes an attention mechanism for the rich content of each node when computing edge representations, capturing auxiliary behavioral information within the target domain. The above recommendation algorithms are mainly aimed at the modeling of the current session, ignoring the item conversion information between sessions, and the similarity and influence between different sessions are also very important for the recommendation results. Wang et al. [16] proposed a global context-enhanced graph neural network model (GCE-GNN) for session recommendation, which enhances the node’s global embedding capability in the current session by sampling the neighbor nodes of nodes in other sessions.

2.4. Methods Based on Attention

Liu et al. [17] believe that it is meaningful to use RNN to capture general interest but do not consider the influence of current user interest on next behavior. In long sessions, RNN may experience a shift in interest due to user misclicks. Both short-term and long-term interests of users are important, but traditional RNN structures cannot distinguish and exploit both interests at the same time. Liu et al. propose a novel short-term attention memory priority model (STAMP), which is able to capture the user’s general interests from the long-term memory of the session context while considering the user’s current interests from the short-term memory of the last click.

In addition, with the rapid development of the natural language field, the emergence of Transformer completely eliminates the dependence of sequential processing on the architecture of recurrent connections, which is computationally inefficient and difficult to parallelize [18,19,20,21,22,23]. Transformer [24] only relies on the self-attention mechanism to capture the global dependencies between input and output. Transformer shows its powerful processing ability for text sequences and makes progress in sequence recommendation. Even in the case of session recommendation with shorter sequence length, Transformer outperforms the RNN recommendation method [25,26,27,28,29,30,31]. Sun et al. [32] believe that the unidirectional structure limits the ability of hidden embeddings in user behavior sequences and propose a sequential recommendation with bidirectional encoder embeddings from Transformer (BERT4Rec), which adopt deep bidirectional self-attention to model user behavior sequences, allowing each item in the user’s historical behavior to fuse the information on the left and right sides for recommendation. Fang [33] proposed a session-based recommendation with the self-attention network model (SR-SAN), which captures the dependencies of all items in a session through a self-attention mechanism, and uses a single item vector to represent the current and global session embeddings. Yuan et al. [34] propose a dual sparse attention network model (DSAN), which uses a sparse transformation function to replace the original norm function in the self-attention of the embedding layer, which reduces the interference of the noise in the session on the item embedding and the session current, and global embeddings are learned using a target attention mechanism at the target embedding layer, which achieves suitable performance. The attention mechanism can shine in the field of recommendation because of the complex interactive behavior of its users when using various software platforms, and traditional deep learning techniques can well capture the ordering within sessions and the dependencies of item transitions [35]. However, it is difficult to identify complex interactive behaviors by relying too much on the order of training data, which limits the ability of recommendation systems to learn user dynamic preferences. The attention mechanism can identify interactions that are important for recommendation across distances.

3. Session-Enhanced Graph Neural Network Recommendation Model (SE-GNNRM)

The definition of the session recommendation model is given, and then the session-enhanced graph neural network recommendation model (SE-GNNRM) is elaborated. The detailed structure of the SE-GNNRM is shown in Figure 1. In Figure 1, v1–5 represent the contained item in the current session.

3.1. Definition of Session Recommendation Model

Session-based recommendation aims to predict the item that the user clicks next after a given behavior sequence, only based on the user’s current behavior information, without considering the user’s historical clicks and user attributes, and other information. The problem is defined as follows: $V=\left[v_1,v_2,v_3,\dots v_m\right]$ represents the set of all items, $m$ is the total number of items. $S=\left[v_{s,1},v_{s,2},v_{s,3},\dots v_{s,n}\right]$ represents a sequence of sessions sorted by timestamp, where $v_{s,n}\in V$ is the item the user clicked on at step n. The next click predicted by session-based recommendations is $v_{s,n+1}$. Specifically, the recommendation model returns a list of predicted item scores $\widehat{y}=\left[{\widehat{y}}_1,{\widehat{y}}_2,{\widehat{y}}_3,\dots {\widehat{y}}_m\right]$, where ${\widehat{y}}_i$ represents the predicted score associated with the item set $v_i$ and recommends the top k candidate items with the highest score.

3.2. Detailed Structure of SE-GNNRM

SE-GNNRM includes an item embedding layer, a session embedding layer, a session interaction layer, and a prediction layer, which will be described in detail as follows.

3.2.1. Item Embedding Layer

The input to the model is the preprocessed sequence of sessions, and a directed graph of all session sequences is constructed. The item embedding layer contains two modules, a graph neural network (GNN) module and a self-attention (SAN). The directed graph of the session sequence is input into the GNN module, and then the item embedding is learned by using the powerful embedding learning ability of GNN for graph structure nodes. The session sequence is input into the SAN module, and the SAN learns the correlation between the session sequences through the self-attention mechanism in order to obtain a more representative embedding of the item.

GNN generates node embeddings on topological graphs to model the dependencies of item feature transformations, and GNN is suitable for session-based recommendations. Each session sequence $S$ preprocessed is modeled as a directed graph $G_s=\left(V_s,E_s\right)$ as the input to the GNN module, where each node represents an item $v_{s,i}\in V_s$, and each connection in the directed graph represents connections between items in a session. Representing each item into a unified space, the resulting vector $v_i\in R^d$ is the $d$-dimensional embedding of $v_i$ obtained using GNN. Based on the idea of RNN, Li et al. [36] introduce the GRU unit into GNN and propose a variant gated graph neural network (GGNN) to make it more suitable for processing sequence data. This model uses GGNN to extract node features. The updated propagation formula for node $v_{s,i}$ in session $S$ is described as follows.

$$a_{s,i}^{\left(t\right)}=A_{s,i:}{[v_1^{\left(t-1\right)},\dots ,v_n^{\left(t-1\right)}]}^T{W}_H+b_1$$

(1)

$$z_{s,i}^{\left(t\right)}=\sigma \left(W_z{a}_{s,i}^{\left(t\right)}+U_Z{v}_i^{\left(t-1\right)}+b_2\right)$$

(2)

$$r_{s,i}^{\left(t\right)}=\sigma \left(W_r{a}_{s,1}^{\left(t\right)}+U_r{v}_i^{\left(t-1\right)}+b_3\right)$$

(3)

$${\tilde{v}}_i^{\left(t\right)}=\tanh \left(W_o{a}_{s,i}^{\left(t\right)}+U_o(r_{s,i}^{\left(t\right)}\odot v_i^{\left(t-1\right)})\right)$$

(4)

$$v_i^{\left(t\right)}=(1-z_{s,i}^{\left(t\right)})\odot v_i^{\left(t-1\right)}+z_{s,i}^{\left(t\right)}\odot {\tilde{v}}_i^{\left(t\right)}$$

(5)

where $W_H\in R^{d\times 2d},W_z,U_Z,W_r,U_r\in R^{d\times 3d},W_o,U_o\in R^{d\times d}$ are the learnable weight matrices. $b_1\in R^d \mathrm{and} b_2,b_3\in R^{3d} \mathrm{are} \mathrm{the} \mathrm{bias} \mathrm{vectores}, \mathrm{and} z_{s,i}^{\left(t\right)}$ and $r_{s,i}^{\left(t\right)}$ are reset and update gates, respectively. The reset gate determines how to combine the new input information with the previous information. The update gate defines what the previous information is saved to the current time step. $[v_1^{\left(t-1\right)},\dots v_n^{\left(t-1\right)}]$ is the node vector embedding in session s, $\sigma (\cdot )$ is a sigmoid function, and $\odot$ is element-wise multiplication. $A_s\in R^{n\times 2n}$ is defined as the concatenation of two adjacent order matrices, $A_s^{\left(out\right)}$ and $A_s^{\left(in\right)}$, representing the weighted connection of incoming and outgoing edges of the session graph. The adjacency matrices $A_s$ distinguish the types of edges in the directed graph and defines how the nodes in the directed graph are connected to each other through incoming and outgoing edges. $A_{s,i}\in R^{1\times 2n}$ is the i-th row in $A_s$, which is the part corresponding to the node $v_{s,i}$. In addition, since duplicate items may appear in the session sequence, a normalized assignment is calculated for each edge, the initial value of the edge divided by the number of incoming or outgoing edges of the node.

Self-attention (SAN) is an attention mechanism that computes sequence embeddings related to different positions of a single sequence and has been successfully applied in many fields. The self-attention mechanism reduces the dependence on external information and is better at capturing the internal correlation of features. In natural language processing, it mainly solves the problem of long-distance dependence by calculating the mutual influence between words, filtering out important item information in the sequence, and assigning greater weight to it. In order to learn more representative session features, self-attention is used to encode session sequences in this paper.

The self-attention network in this paper follows the mechanism, which consists of an attention network and a feed-forward network. The session sequence $S=\left[v_{s,1},v_{s,2},v_{s,3},\dots v_{s,n}\right]$ is input into the attention network, and the attention function C consists of a map of the query and key-value pairs outputs:

$$C=softmax(\frac{\left(S{W}^Q\right)\left(S{W}^k\right)}{\sqrt{d}})\left(S{W}^V\right)$$

(6)

where $W^Q,W^k,W^V\in R^{d\times d}$ is the learning weight matrices, and $\sqrt{d}$ is the normalization factor in preventing the calculation value of the inner product from being too large.

The feed-forward network consists of two layers of neural networks. Relu is selected as the first layer activation function to add non-linearity to the self-attention module. At the same time, transmission loss may occur in the self-attention network. Using residual connections can make full use of the bottom layer information. The feed-forward network layer formula is as follows:

$$F=H+Relu\left(C{W}_1+b_4\right)W_2+b_5$$

(7)

where $W_1\in R^{d\times 4d},W_2\in R^{4d\times d}$ is the weight matrices that can be learned, $b_4,b_5\in R^d$ is the bias vector. For session S, $V_s=\left[v_{s,1},v_{s,2},v_{s,3},\dots ,v_{s,n}\right]$ and $F_s=\left[f_{s,1},f_{s,2},f_{s,3},\dots ,f_{s,n}\right]$ are obtained after encoding by the GNN module and the SAN module, respectively. Then, the two item embeddings are combined to complement each other. The joint function is described as follows.

$$O_s=V_s+\theta F_s$$

(8)

where $\theta$ is a weight parameter, which represents the importance of the item embedding learned by the SAN module, and is used to adjust the proportions of the two modules.

3.2.2. Session Embedding Layer

Although the item embedding layer of the model learns the transition dependencies of items in the session, it does not contain any recurrent or convolutional mechanisms, resulting in the model not fully considering the sequentiality of the session and therefore not knowing the position of the previous items in the session [37]. In session recommendation, each session is a sequence combination of a set of ordered items, where the position order encoding takes into account the different effects of relative positions between different elements in a sentence or a sequence [15,38]. In the model proposed in this paper, in order to strengthen the sequentiality of the learned session embedding, the reverse position information is incorporated into the session embedding through the position matrices $P=\left[p_1,p_2,p_3,\dots ,p_n\right]$, where $p_i\in R^d$ is the position vector of position $i$. The session sequence $O=\left[o_{s,1},o_{s,2},o_{s,3},\dots o_{s,n}\right]$ is output by the joint function in the item embedding layer. After O is added to the position information, the i-th item is represented as:

$$z_i=\tanh \left(W_3\left[o_{s,i}\Vert p_{n-i+1}\right]+b_6\right)$$

(9)

where $W_3\in R^{d\times 2d}$ is the weight matrices that can be learned, $b_6\in R^d$ is the bias vector, $\tanh (\cdot )$ represents a bitangent activation function, $\Vert$ represents the splicing operation, and the item represents $v_{o,i}$ is concatenated with the position vector $p_{n-i+1}$, where n is the length of the current session. Inspired by STAMP and NARM, the session embedding that fuses the long-term and short-term interest preferences of a session can better reflect the characteristics of the current session. The purpose is to emphasize the importance of the current preference to adapt to the complex and changeable interaction behavior of users. Next, the soft attention mechanism is used to calculate the weighted summation of the attention coefficient ${\alpha }_i$ of the item in the session and the item embedding $z_i$ to obtain the current session embedding $h_c$, which is described as follows.

$$h_c={\displaystyle {\sum }_{i=1}^n}{\alpha }_i{z}_i$$

(10)

$${\alpha }_i=q^T\sigma \left(W_4{z}_i+W_5{z}_n+W_6{z}_{avg}+b_7\right)$$

(11)

$$z_{avg}=\frac{1}{n}{\displaystyle {\sum }_{i=1}^n}{z}_i$$

(12)

where $\sigma (\cdot )$ is a sigmoid function, where $W_4,W_5,W_6\in R^{d\times d},q\in R^d$ is the weight matrices used to calculate the attention coefficient ${\alpha }_i$, and $b_7\in R^d$ is the partial set vector. $h_c$ is obtained by the weighted summation of the calculated attention coefficient ${\alpha }_i$ of the item in the session and the item embedding $z_i$. The target embedding $z_n$ is defined as the last item of the input session sequence, $z_{avg}$ is the average value represented by the items in the session, which represents a central feature of the session [39].

3.2.3. Session Interaction Layer

The target embedding and global embedding of the session actually only pay attention to the current session features but ignore the similar but different features between sessions, so this paper designs a session interaction layer. The GAT [40] aggregates neighbor node features through an attention mechanism, and the FGNN [14] uses GAT with weight information to learn node features. Inspired by the GAT and FGNN, this model calculates the co-occurrence degree of the same interactive items in existing sessions as the co-occurrence parameter ${\varphi }_{ij}$ and integrates the graph attention network for feature propagation so that sessions with high similarity can learn from each other features that have an important impact on recommendation results. The final embedding of a session depends on the combination of the session itself and its neighborhood sessions.

In the session interaction layer, first, a global session graph is constructed, in which each node represents a session, and the co-occurrence parameter ${\varphi }_{ij}$ of the items in the session is used to determine the connections of each node in the global session graph. Then, the current session embedding $h_c$ output by the session embedding layer is used as the initial input of the feature transfer function, and the similarity coefficient $e_{ij}$ between nodes and neighboring nodes is obtained by calculating the inner product of nodes in the global session graph and their co-occurrence parameters. Finally, the softmax function is used to normalize $e_{ij}$ to obtain the inter-session attention coefficient ${\beta }_{ij}$, which is described as follows.

$$e_{ij}=LeakyRelu(\left[h_i\odot h_j\Vert {\varphi }_{ij}]W_7\right)W_8$$

(13)

$${\beta }_{ij}=\frac{\exp \left(e_{ij}\right)}{{\displaystyle {\sum }_{h_k\in N_{h_i}}}\exp \left(e_{ik}\right)}$$

(14)

where, $LeakyRelu( )$ is a nonlinear function. Its negative slope is set to 0.2, $W_7\in R^{\left(d+1\right)\times d},W_8\in R^d$ is a learnable weight matrix, $\odot$ is element-wise multiplication, $N_{h_i}$ denotes the set of $h_i$ neighborhood sessions. $h_i, h_j, h_k$ are the outputs of the session embedding layer. The attention-aware neighborhood session embedding $h_i^s$ is given as follows.

$$h_i^s={\displaystyle \sum _{h_k\in N_{h_i}}}{\beta }_{ij}{h}_j$$

(15)

The final step of the session interaction layer will rely on the attention-aware neighborhood session embedding $h_i^s$ to be aggregated with the current session embedding $h_i$. The neighborhood aggregation formula adopts a gating mechanism to autonomously select more meaningful session features, which is described as follows.

$$\gamma =\sigma \left(W_9{h}_i+W_{10}{h}_i^s\right)$$

(16)

$$h_f=\gamma h_i+\left(1-\gamma \right)h_i^s$$

(17)

where $W_9,W_{10}\in R^{d\times d}$ is a learnable weight matrix. $\gamma$ is a gate obtained by a nonlinear activation function after calculating the linear transformation of $h_i$ and $h_i^s$, $h_f$ is the global embedding session.

3.2.4. Prediction Layer

In the prediction layer, the model evaluates the probability of the next clicked item in the session based on the session embedding $h_f$ obtained by the session interaction layer, calculates the inner product of $v_i$ and $h_f$ as the recommendation score of each candidate item, and uses the softmax function to obtain the final recommendation probability.

$$\widehat{y}=softmax\left({h_f}^T·v_i\right)$$

(18)

where $v_i$ is the initial vector of the item, $\widehat{y}\in R^m$ represents the probability that the candidate item is the next clicked item, and selecting the top $k$ candidate items in $\widehat{y}$ will be selected as the recommended item. To train the model, the loss function of the model is defined as the cross-entropy of the predicted value and the true value:

$$ℒ\left(\widehat{y}\right)=-{\displaystyle {\sum }_{i=1}^m}{y}_i\log \left({\widehat{y}}_i\right)+\left(1-y_i\right)\log \left(1-{\widehat{y}}_i\right)+\mathsf{\lambda }{\Vert \psi \Vert }^2$$

(19)

where $y_i$ represents the one-hot encoded vector of the true value of the next click, $\psi$ is the set of all learnable parameters, $\mathsf{\lambda }$ is the L₂ regularization parameter, and the model is trained using the back-propagation through time (BPTT) method.

4. Experiment and Analysis

In this section, the data set and experimental setup are first described, and then the designed experiments are presented to verify the performance of the proposed model SE-GNNRM, and the results under different experimental setups are analyzed in detail.

4.1. Data Sets

The real data sets used in the experiment are Yoochoose, Diginetica, and Tmall. The Yoochoose data sets are from RecSys Challenge 2015, which contains user click data of e-commerce websites within 6 months. This paper uses 1/64 of the Yoochoose data sets as the data sets to prevent the memory overflow caused by the Yoochoose data sets being too large. The Diginetica data sets are from CIKM Cup 2016, and this article only uses its transaction data. The Tmall data sets are from the IJCAI-15 competition data sets, which contain anonymous user shopping data on the Tmall shopping platform. In data preprocessing, all sessions of length 1 and items that appear less than five times in the data sets are filtered out, and the data sets are reordered in chronological order. For the Yoochoosse data sets, the data from the last day are used as the test set and the remaining data as the training set, while for the Diginetica data sets, the data from the last seven days are used as the test set and the remaining data as the training set. For the Tmall data sets, the last 100 s are used as the test set, and the rest of the data are used as the training set. Then, the sequences and corresponding labels by splitting the input sequence are generated [41]. For session $S=\left[v_{s,1},v_{s,2},v_{s,3},\dots v_{s,k}\right]$, the sequences and corresponding labels are generated by sequence preprocessing, i.e., $\left(\left[v_{s,1}\right],v_{s,2}\right),\left(\left[v_{s,1},v_{s,2}\right],v_{s,3}\right),\dots ,\left(\left[v_{s,1},v_{s,2},\dots ,v_{s,k-1}\right],v_{s,k}\right)$ is training and test sets for all three data sets. The statistics of the preprocessed data set are shown in

Table 1. Statistics data for the data sets.

Data Set	Yoochoose1/64	Diginetica	Tmall
Clicks	557,248	982,961	818,479
Train	369,859	719,470	351,268
Test	55,898	60,858	25,898
Items	16,766	43,097	40,728
Average length	6.16	5.12	6.69

.

4.2. Baseline Algorithm

To verify the effectiveness of the proposed model, it is compared with the following methods. The baseline method is described as follows.

• Item-KNN [5] uses cosine similarity to define the degree of similarity between sessions and recommends items that are similar to previous clicks.
• FPMC [7] is a sequence recommendation method based on Markov chain and matrices factorization.
• GRU4REC [9] uses RNN to model user sequences for session-based proposals.
• NARM [10] uses an RNN with an attention mechanism to capture the main purpose and sequential behavior of users, adding an attention mechanism to capture the user’s general interest and current interest in the current session.
• CSRM [11] consists of an internal and an external memory encoder to model the preferences of the current session and neighborhood sessions in order to better predict the intent of the current session.
• SR-GNN [4] uses GNN to capture complex item information transformation and uses an attention mechanism to capture long-term and short-term interests.
• FGNN [14] learns item vectors through a weighted attention layer and learns session features through a session graph feature extraction layer.
• STAMP [17] uses an attention network to capture the user’s general interest and current interest in the current session.
• CoSAN [42] uses a self-attention network to learn the current session embedding and collaborative feature information of items from neighborhood sessions.

4.3. Parameter Settings

The experimental parameters in this paper are set as follows, the dimension of item embedding and the training batch size are set to 100. Co-occurrence parameter ${\varphi }_{ij}$ of session interaction layer selection range is [0.0–0.9]. Use random initialization for each vector, with all parameters having a mean of 0 and a standard deviation of 0.1. All parameters are optimized using the cross-entropy loss function and the mini-batch Adam algorithm, where the initial learning rate is set to 0.001, the number of epochs is set to 30, and the decay will be 0.1 after every 3 epochs, using L2 regularization to prevent the algorithm from overfitting, L2 regularization coefficient selection range is [1 × 10⁻³, 5 × 10⁻⁴, 1 × 10⁻⁴, 5 × 10⁻⁵, 1 × 10⁻⁵]. For a fair comparison, the dimension of the item embedding of the baseline algorithm is set to 100, and other basic hyperparameters follow what is shown in the baseline algorithm literature. The number of hidden units in both GRU4REC and NARM is set to 100, the memory of CSRM is set to 100, the number of attention layers is 3 in FGNN, and the multi-head mechanism used is set to 8. The number of CoSAN neighbor sessions is set to 5.

4.4. Evaluation Metrics

Two metrics were selected for the experiments to evaluate the compared methods. P@20 (precision) is widely used as a measure of prediction accuracy, which represents the proportion of correctly recommended items among the top 20 items. MRR@20 (mean reciprocal rank) is the average of the reciprocal rank of correctly recommended items, where a larger MRR value represents a correct recommendation at the top of the ranking list.

In order to verify the performance of the model, two groups of experiments are designed in this paper, and the experimental results are analyzed. The first set of comparative experiments compares the performance of SE-GNNRM with nine baseline models. The results of the comparative experiments are shown in

Table 2. Comparative experimental results.

Models (%)	Yoochoose1/64		Diginetica		Tmall
	P@20	MRR@20	P@20	MRR@20	P@20	MRR@20
FPMC	45.62	15.01	26.53	6.95	16.06	7.32
Item-KNN	51.60	21.81	35.75	11.57	9.15	3.31
GRU4REC	60.64	22.89	30.79	15.46	10.83	5.89
NARM	68.32	28.63	48.43	16.11	23.30	10.70
CSRM	71.45	30.36	50.55	16.38	29.46	13.96
SR-GNN	70.57	30.94	50.73	17.59	27.57	13.72
FGNN	71.12	31.68	51.36	18.47	25.24	10.39
STAMP	68.74	28.67	46.62	15.13	26.47	13.36
CoSAN	69.71	29.92	48.34	15.22	27.21	13.12
SE-GNNRM	72.76	32.18	52.84	18.49	32.70	15.61

. The second set is hyper-parameter experiments to investigate the effect of the co-occurrence parameter ${\varphi }_{ij}$ on model performance.

4.5. Results and Analysis

4.5.1. Comparative Experiment

The experiment uses TensorFlow and PyTorch frameworks and uses Python language to simulate the existing nine baseline models and SE-GNNRM and compares and analyzes their performance. The comparison results are shown in Table 2.

From Table 2, it can be found that the SE-GNNRM proposed in this paper achieves the best performance on the two evaluation metrics P@20 and MRR@20. On the Yoochoose1/64 data sets, it is 1.42% and 1.58% higher than the suboptimal algorithm, on the Diginetica data set, it is 1.53% and 0.1% higher than the second-best algorithm, and on the Tmall data set, it is 10.9% and 11.8% higher than the second-best algorithm. The SE-GNNRM effectively demonstrates its ability to recommend all three data sets.

In the early recommendation models, the FPMC model limits dependencies in the session sequence, so the recommendation effect is poor. Item-KNN outperforms Markov chain-based algorithms, but Item-KNN only recommends highly similar items and does not fully consider the problem of order transitions in sessions.

Compared with traditional algorithms, deep learning-based models explicitly model the user’s global behavioral preferences and take into account the transition between the user’s previous behavior and the next click, effectively proving their ability to capture session sequences. There is a stronger ability to sequentially transform relationships. GRU4REC, a classic work on modeling session sequences using RNNs, proves that encoding the current session with GRU units is more suitable for session recommendation problems than traditional RNN and LSTM on session recommendation problems and takes the last layer as a general preference for sessions. However, the interest preferences represented by session embeddings are variable, and GRU4REC does not distinguish current preferences from general preferences. On the basis of GRU4REC, NARM solves this problem well. After using the GRU unit to encode the current session to obtain the item embedding, it dynamically selects and linearly combines different parts of the input session sequence through the soft attention mechanism. The experimental results also demonstrate the effectiveness of this mechanism for learning session preferences. CSRM also uses a GRU-based RNN to model the session sequence while also considering the current and general preferences of the session. Furthermore, CSRM designs an external memory encoding layer for learning the collaborative information of adjacent sessions, which makes it have better recommendation performance than many models modeling a single session.

In the comparative experimental results, the GNN-based algorithm is more competitive than the RNN-based algorithm. SR-GNN further considers the dependencies of item transitions in a session by modeling each session as a directed graph. The items in the session are the nodes in the graph, which can capture more complex and implicit dependencies within the session. Experimental results show that even though SR-GNN is based on single session modeling, its performance is almost on par with algorithm CSRM, which considers multiple sessions, proving that it is meaningful to use GNN to model session sequences. FGNN designs a weight map attention layer for learning the item embedding in the session graph and also uses a GRU-based readout function to abstract the session embedding. FGNN uses a variety of deep learning techniques and achieves pretty suitable performance in Yoocoose1/64 and Diginetica.

STAMP is an algorithm that uses an attention mechanism to explicitly represent the last item of the session and the average features of the session and items embedding in the session to capture the preference of the session. The recent session recommendation algorithm based on self-attention has shined in many fields. CoSAN uses the strong encoding ability of self-attention for sequence data to learn session sequences from the perspective of similarity and location information between items, incorporates the collaborative information based on the adjacent session identification, and also gets quite suitable results. The experimental results show that the algorithm of the attention mechanism outperforms some existing deep learning algorithms and is a powerful choice for dealing with the session recommendation problem.

In contrast, the SE-GNNRM algorithm proposed in this paper uses both a GNN and a self-attention mechanism to encode session sequences and designs a fusion function to fuse the two session vectors to complement each other. Then, in order to reasonably learn the long-term and short-term preferences of the session, the soft attention mechanism fused with reverse position information is used to obtain the session embedding, which not only takes into account the transformation of item information in the session but also strengthens the influence of the sequential position of the items. Finally, a session interaction layer is designed based on the idea that session preferences with high similarity may also be similar, which introduces the influence of multi-session collaborative information into the algorithm. The preference information of neighbor sessions is further aggregated through the graph attention network on the constructed global session graph, which makes it more fully and efficiently process session sequence information. In summary, the experimental results and analysis demonstrate the rationality and effectiveness of the SE-GNNRM proposed in this paper.

4.5.2. Co-Occurrence Parameter Analysis

This subsection shows the effect of choosing different co-occurrence parameters ${\varphi }_{ij}$ between 0.0 and 0.9 for the three data sets on the recommendation performance. In the session interaction layer, a dynamic global session graph needs to be constructed, and the connections between nodes in the global session graph are determined according to the co-occurrence parameters. There are edges only between nodes that meet the co-occurrence parameter threshold so that the neighbor sessions of the current session can be confirmed according to the global session graph. Therefore, for the session interaction layer, how to choose the appropriate co-occurrence parameters needs to be verified and discussed through experiments. The following figures show the performance of SEGR when selecting different co-occurrence parameters for three data sets. It can be observed from Figure 2 that the trend is flat between 0.3 and 0.8 on the Tmall data set, and SEGR finally chooses 0.6 as the co-occurrence parameter for the Tmall data sets. On the Yoochoose1/64 and Diginetica data sets, the best results are achieved at 0.5 and 0.4, respectively. It can be concluded that the optimal values on the three data sets tend to be in the middle range of the interval. This is explainable. If the selection of neighbor sessions is too low and noise may be introduced to affect the performance of the recommendation algorithm, and if the selection is too high, similar neighbor features cannot be fully learned, or even there are no neighbors.

5. Conclusions

For many existing session recommendation models, more consideration is given to the order transition within the session, ignoring the impact of similar sessions on the recommendation performance. Some intra-session transition relationships captured by the GNN are not conducive to recommendation. In this paper, an improved session recommendation model based on GNN and attention networks, namely a session-enhanced graph neural network (SE-GNNRM), is proposed. Firstly, the model is modeled based on the session sequence, and the initial session sequence is encoded by GNN and SAN, respectively, and the obtained item vector takes into account the complex item transition and the importance of the item at the same time. Secondly, in order to enable the model to learn similar session information, a session interaction layer is designed. A global session graph is constructed, and the features between similar sessions are learned through an attention mechanism that fuses the similarity information between sessions. Extensive experiments on three real data sets demonstrate the effectiveness of the model. In the future, it is hoped that the recommendation performance of the model can be improved through more efficient graph representation learning and the introduction of more meaningful auxiliary information.