HyperCLR: A Personalized Sequential Recommendation Algorithm Based on Hypergraph and Contrastive Learning

Abstract

Sequential recommendations aim to predict users’ next interactions by modeling their interaction sequences. Most existing work concentrates on user preferences within these sequences, overlooking the complex item relationships across sequences. Additionally, these studies often fail to address the diversity of user interests, thus not capturing their varied latent preferences effectively. To tackle these problems, this paper develops a novel recommendation algorithm based on hypergraphs and contrastive learning named HyperCLR. It dynamically incorporates the time and location embeddings of items to model high-order relationships in user preferences. Moreover, we developed a graph construction approach named IFDG, which utilizes global item visit frequencies and spatial distances to discern item relevancy. By sampling subgraphs from IFDG, HyperCLR can align the representations of identical interaction sequences closely while distinguishing them from the broader global context on IFDG. This approach enhances the accuracy of sequential recommendations. Furthermore, a gating mechanism is designed to tailor the global context information to individual user preferences. Extensive experiments on Taobao, Books and Games datasets have shown that HyperCLR consistently surpasses baselines, demonstrating the effectiveness of the method. In particular, in comparison to the best baseline methods, HyperCLR demonstrated a 29.1% improvement in performance on the Taobao dataset.

1. Introduction

Information overload is an important issue with the rapid development of the Internet, which means that network users are often faced with massive information and struggle to identify valuable content. To address this, recommendation systems have been developed [1]. These systems generate personalized item suggestions by analyzing users’ historical interaction data, browsing habits, and purchase records. A key aspect of these systems is sequential recommendation, which aims to capture the evolving preferences of users and predict their next item of interest [2]. The historical interaction data of users and items can be constructed as a graph. Recently, Graph Neural Networks (GNNs) have been widely used in sequential recommendation due to their superiority in graph-structured data processing. However, a major challenge is the diverse evolution of user interests, which complicates the interpretation of global item transition patterns. Moreover, while GNN-based methods [3] typically model data as directed graphs, they often fail to capture the complex, high-order item relationships in user sequences. Item transitions often result from interactions with multiple previous items in the real-world scenarios, creating intricate multi-to-multi high-order relationships that simple graphs cannot accurately represent [4]. Additionally, unpredictable and diverse user behaviors can introduce noise into the data.

Hypergraphs extend traditional graph structures by allowing each hyperedge to connect multiple nodes, which are suitable for capturing these potential relationships across multiple entities [4]. Recent research has explored the ability of hypergraphs in modeling interactions across users and items to provide more personalized recommendations. For example, Guo et al. [5] and Jiang et al. [6] attempt to recognize intricate relationships between users and items and capture the evolution of user preferences based on hypergraphs. In addition, contrastive learning has also been applied in sequential recommendation. It is a self-supervised learning method [7,8] that typically uses pairs of positive and negative samples to train models. This approach effectively reduces noise in the data, enabling the model to distinguish between similar and diverse user sequences. Zhou et al. [9] improve item representation learning through masked sequence modeling and contrastive learning, thus gaining deeper insights into user behavior patterns. Xia et al. [10] develop a contrastive learning strategy that focuses on user–item interactions, significantly raising the discriminative capability of the model.

For the first challenge, this paper leverages hypergraphs to model latent high-order relationships among user preferences in sequential recommendations [4]. We denote each user’s interaction data as a hyperedge and each item as a node in these hyperedges, thus forming a comprehensive hypergraph that contains item-level high-order relations. Through hypergraphs, this paper can analyze complex item transition patterns from users’ local interaction sequences to explore personalized preferences deeply. To address the second challenge, this paper proposes a HyperCLR algorithm that merges the global hypergraph with every user interaction sequence to build a Global Item Frequency Distance Graph (IFDG). This graph describes item transition patterns within user sequences, offering a global context for each user–item interaction by constructing edges based on interaction frequency and spatial distance as edge attributes. Inspired by GraphSAGE [11], this paper utilizes the technique of neighborhood sampling and obtains an enhanced representation of this network. Concerning the third challenge, this paper adopts contrastive learning [12] to refine the consistency between identical sequence representations within the enhanced graphs, mitigating the influence of noisy interactions in the data.

In this paper, we present HyperCLR, which is a personalized sequential recommendation algorithm based on both hypergraphs and contrastive learning. The whole framework of our method is shown in Figure 1. The contributions of this paper are concluded as follows:

(1)

Hypergraphs are utilized to extract and analyze high-order item transition patterns from local user interaction sequences, which enhances the understanding of personalized preferences. Given the impact of positional attributes of interaction sequences, this paper can dynamically integrate time and location data by fine tuning the hypergraph to current user preferences.

(2)

This paper constructs the Global Item Frequency Distance Graph (IFDG) to investigate the high-order interaction relationships among items, enabling the mining of item transformation relationships within sequences. In addition, this paper designs a gating mechanism to capture dynamic interest.

(3)

This paper adopts contrastive learning to alleviate data sparsity and the behavioral unpredictability of users, enhancing the coherence of sequence representations in these enhanced graph structures and reducing the impact of anomalous interactions.

(4)

Extensive experiments conducted on three real-world datasets demonstrate that HyperCLR consistently surpasses baselines, validating this approach’s effectiveness.

2. Related Work

2.1. Sequential Recommendation

Sequential recommendation systems aim to make predictions about users’ future interactions by analyzing their historical behaviors, which is structured in the time-order of item occurrences [2]. In recent years, deep neural networks have gained increasing attention in sequential recommendation systems. Notably, Convolutional Neural Networks (CNNs) and attention schemes have proven effective for this task. For example, Caser [13] utilizes convolutional filters to capture the sequential order of user interactions, while SASRec [14] employs an attention mechanism to understand the relationships between items, enhancing the detection of user intentions. Additionally, sequence modeling techniques are employed to encode user interaction sequences or to convert these sequences into graph structures, which facilitates a deeper exploration of item relevance [3,15,16]. The inherent complexity of user behavior, item characteristics, and interaction contexts in real-world scenarios poses a significant challenge in accurately modeling users’ dynamic preferences and consumption intentions in sequential recommendations.

2.2. Graph Neural Networks

In recent years, Graph Neural Networks (GNNs) have become prominent in sequential recommendation tasks due to their distinct benefits on graph-structured data [3,17]. In general, users and items can be considered as nodes in a graph in order to depict the sequence of interactions of each user in the graph. Then, GNNs are used to learn user and item representations. Nevertheless, most works primarily focus on the quantity of interactions between nodes, often neglecting to thoroughly analyze the strength of correlations between items through their interactions or to effectively capture high-order item features for more precise embeddings. This limitation can hinder the model’s ability to detail the nuanced transitions between graph nodes [4]. At this point, hypergraphs provide a solution by naturally capturing complex high-order relationships [4]. Pioneering efforts such as HGNN [18] and HyperGCN [19], have applied graph convolution techniques to hypergraphs, using advanced GNNs to delve into the intricate item transition relationships within sequences and to model the contextual relationships between items and sequences comprehensively. These methods enhance interpretability and learn dynamic preference transitions by modeling complex user–item and item–item relationships within user interaction sequences.

2.3. Contrastive Learning for Recommendation

Contrastive learning focuses on obtaining robust data representations by exploiting the similarities and differences between data samples [7,8]. The basic idea of contrastive learning is that similar samples are clustered closer together and dissimilar samples are clustered farther apart. This method effectively improves the model’s capability to identify and interpret data patterns without the need for explicit labels. In the context of recommendation systems, CL4SRec [20] incorporates contrastive learning by augmenting sequences with techniques such as cropping, masking, or reordering. This approach not only refines the recommendation process but also introduces a new dimension to sequence manipulation for model training. Moreover, contrastive learning is adaptable to graph-based representations, enhancing the detection of global item-related signals. For instance, GCL4SR [16] applies graph contrastive learning to recommendation tasks by randomly sampling neighbors within item transition graphs to obtain different item representations from different subgraphs. APGL4SR [21] utilizes graph contrastive learning to ascertain personalized global information, thus refining the intricate interactions within user sequences.

3. Problem Definition

In this section, a detailed introduction to the target problem and the related notations involved in the approach of this paper are provided.

The objective of sequential recommendation is to effectively model the interaction sequence for the given user u at a given time t and to accurately predict the next item likely to be interacted with at time $t+1$. The set of users is denoted as $U=\{u_1,u_2,\cdots ,u_{\left|U\right|}\}$ and the set of items is denoted as $V=\{v_1,v_2,\cdots ,v_{\left|V\right|}\}$. Each user $u\in U$ has an associated interaction sequence $S_u=\{{v_1}^u,{v_2}^u,\cdots ,{v_n}^u\}$, which lists the items interacted with, ordered by interaction timestamps. Here, n is the total number of items in the interaction history of u. Additionally, $p_u\in {\mathbb{R}}^{1\times d}$ represents the embedding vector for user u, and $e_i\in {\mathbb{R}}^{1\times d}$ represents the embedding vecto for item i, where d is the dimension of embeddings. The initial embedding of the sequence $S_u$ is represented by $E_S^0\in {\mathbb{R}}^{n\times d}$, where the t-th row corresponds to the embedding of the t-th item in $S_u$. The matrix $E\in {\mathbb{R}}^{\left|V\right|\times d}$ contains the embeddings of all items. The objective of sequential recommendation is to effectively model the interaction sequence for the given user u at a given time t and to accurately predict the next item likely to be interacted with at time $t+1$.

In this paper, $G_H=(V,E_H)$ represents a hypergraph, where each user is denoted as a hyperedge. Formally, each hyperedge associated with user u is represented as $[e_{u,1},e_{u,2},e_{u,3},\dots ,e_{u,m}]$ within the set E with each item $e_{u,m}$ belonging to the vertex set V. Here, V comprises item nodes accessed by user u, and E includes m hyperedges representing the user’s interactions. Each hyperedge $ϵ$ connects two or more vertices and is assigned a positive weight $W_{ϵϵ}$ with all these weights forming a diagonal matrix $W\in {\mathbb{R}}^{m\times m}$. The hypergraph structure is further described by an incidence matrix $H\in {\mathbb{R}}^{n\times m}$ [22]. In this matrix, if hyperedge $ϵ$ includes vertex $v_i$, then $H_{iϵ}=1$; otherwise, $H_{iϵ}=0$. The degree of each vertex $v_i$ and each hyperedge $ϵ$ are denoted by the diagonal matrices D and B, respectively. Specifically, $D_{ii}$ is calculated as $D_{ii}={\sum }_{ϵ=1}^m{W}_{ϵϵ}{H}_{iϵ}$ and $B_{ϵϵ}$ is $B_{ϵϵ}={\sum }_{i=1}^n{H}_{iϵ}$. In this setup, each hyperedge is assigned a uniform weight of 1 for simplicity.

4. Methodology

The framework of the proposed HyperCLR is shown in Figure 1. First, a global hypergraph is constructed based on the user interaction sequences. Next, from the constructed hypergraph and the global user interaction sequences, a Global Item Frequency Distance Graph (IFDG) will be built to learn higher-order interactions between items and explore their correlations. After that, this paper uses neighbor sampling to construct random subgraphs and use GATs (Graph Attention Networks) to learn item embeddings. Subsequently, to further improve the representation embeddings, this paper adopts contrastive learning on the basis of the learned embeddings from random subgraphs, mitigating data sparsity and user behavior uncertainty. Then, this paper extracts higher-order item transition patterns of users’ local interaction sequences to analyze users’ personalized preferences. Meanwhile, this paper dynamically fuses time and location information into users’ preferences and designs an interest-gating mechanism to accurately capture users’ dynamic preferences. Finally, this paper can use Transformer to understand the user’s preferences and predict the next item using a multi-layer perception (MLP). A detailed description of the HyperCLR follows.

4.1. Construction of Global Item Frequency Distance Transformation Graph (IFDG)

Hyperedges in the hypergragh focus on similarity and coherence among items while discarding the sequential order of interactions. Note that $e_{S_u,m-1}$ and $e_{S_u,m}$ appear in the same hyperedge only if there is a previous interaction between them [23], which ensures that all items within the same sequence are interconnected. This paper can transform the hypergraph into an undirected graph to model the information transmission process because the influence between any pair of items is mutual and independent of their order in the interaction [24]. For instance, the impact of item $v_i$ on $v_j$ is considered equivalent to that of $v_j$ on $v_i$ [25]. The construction process of the hypergraph is illustrated in Figure 2. $S_1$, $S_2$, and $S_3$ represent the interaction sequences of users $u_1$, $u_2$, and $u_3$, respectively.

Moreover, this paper can utilize the total interaction counts, $\mathrm{Count}(i,j)$, between all users and global items to assign weights to the edges connecting items within a sequence. $\mathrm{Count}(i,j)$ represents the total number of visits to items i and j across all user sequences, reflecting their co-occurrence in the same hyperedge. Previous work often relies solely on the interaction count between two item nodes to set edge weights, focusing primarily on the local context or user preferences within each sequence. This focus generally overlooks the relevance among users who exhibit similar behavioral patterns, such as sharing identical item subsequences [26]. As a result, they fail to capture the fine-grained transition relationships between items, neglecting the high-order connectivity across different user sequences. Unlike methods that solely model sequential transition patterns within individual sequences, HyperCLR learns from both user–item interactions and the co-occurrence patterns between items [27].

Items that frequently appear together are considered more similar. To further enhance the global item transition patterns and discover latent high-order relationships between items, this paper can also incorporate the positional distance $\Delta D$ between items in all original user interaction sequences. Initially, each distance is set to zero as outlined in Equation (1).

$$D=\left[\begin{array}{cccc}\Delta d_{11}& \Delta d_{12}& \dots & \Delta d_{1\left|V\right|}\\ \Delta d_{21}& \Delta d_{22}& \dots & \Delta d_{2\left|V\right|}\\ \Delta d_{i1}& \Delta d_{i2}& \Delta d_{ij}& \Delta d_{i\left|V\right|}\\ ⋮& ⋮& \ddots & ⋮\\ \Delta d_{\left|V\right|1}& \Delta d_{\left|V\right|2}& \dots & \Delta d_{\left|V\right|\left|V\right|}\end{array}\right],$$

(1)

where $\Delta d_{ij}$ quantifies the distance between two interacting items $v_i$ and $v_j$ within a user’s sequence. It accumulates based on the relative positions of these items whenever they appear together in the same sequence. The distance updating formula provided ensures that $\Delta D_{ij}$ grows as $v_i$ and $v_j$ appear more frequently in closer proximity within sequences, as shown in Equation (2):

$$\Delta D_{ij}=\left\{\begin{array}{cc}|j-i|+\Delta D_{ij}^{\prime }\hfill & \mathrm{if}{H}_{iϵ}=H_{jϵ}=1,ϵ\in \{1,2,3,\dots ,M\},\hfill \\ +\infty \hfill & \mathrm{if}{H}_{iϵ}\ne H_{jϵ}\mathrm{or}{H}_{iϵ}=H_{jϵ}=0,ϵ\in \{1,2,3,\dots ,M\},\hfill \end{array}\right.$$

(2)

where i and j denote the actual positions of $v_i$ and $v_j$ in the user interaction sequence. $\Delta D_{ij}^{\prime }$ represents the original value and $\Delta D_{ij}$ represents the updated value. Here, $H_{iϵ}=H_{jϵ}=1$ indicates that both $v_i$ and $v_j$ are part of the same hyperedge $ϵ$, reflecting direct interaction within the sequence. If $v_i$ and $v_j$ are not part of the same hyperedge, or if they do not appear in any hyperedge together, the distance is set to infinity, symbolizing no direct relationship within the given contexts.

After this process is applied across all user sequences, a tailored activation function $S(·)$ is introduced to normalize the edge weights between nodes $v_i$ and $v_j$ in the global item graph based on the calculated interaction distances. The function $S(\Delta D_{ij})$ is defined as shown in Equation (3):

$$S(\Delta D_{ij})=\left\{\begin{array}{cc}{\displaystyle \frac{1}{\Delta D_{ij}}}\hfill & \mathrm{if}\Delta D_{ij}\ne +\infty ,\hfill \\ 0\hfill & \mathrm{if}\Delta D_{ij}=+\infty .\hfill \end{array}\right.$$

(3)

A larger distance between two interacting items indicates a weaker correlation, while a smaller distance indicates a stronger correlation and hence a higher edge weight. By integrating the global item distance with the interaction frequency data, Count(i,j), and normalizing this combination, the study develops a Global Item Frequency Distance Graph (IFDG). The construction process of IFDG is shown in Figure 3. In this graph, the edge weights between items are defined using the Formula (4):

$$w_{ij}=w_{ji}=\mathrm{softmax}\left({\displaystyle \frac{2\times \mathrm{Count}(i,j)\times S\left({\Delta }_{ij}\right)}{D_{ij}+D_{ij}}}\right),$$

(4)

where $D_{ii}$ and $D_{jj}$ denote the degrees of nodes i and j within the hypergraph, indicating the number of interactions involving nodes i and j within a sequence. The IFDG is represented as $G_I(V,E_I)$.

4.2. Graph Convolution and Contrastive Learning

This paper constructs a stochastic subgraph construction from the IFDG according to the neighbor sampling in GraphSAGE [11], generating enhanced views of IFDG. Then, this paper can adopt the convolution operation in GAT [28] to enhance flexibility and adaptability in graph fusion. Specifically, for a random batch of nodes S from $G_I$, each node $v\in S$ is treated as a central node, and its neighbors are sampled for interaction. During the sampling process, nodes are uniformly sampled without consideration of edge weights, and the edges between sampled nodes are preserved along with their weights in $G_I$. Hence, this paper can obtain two augmented views ${\mathcal{G}}_S^{\prime }=(V_S^{\prime },E_S^{\prime })$ and ${\mathcal{G}}_S^{″}=(V_S^{″},E_S^{″})$, where $V_S^{\prime }$ and $E_S^{\prime }$ are the sets of nodes and edges, respectively. The node information fusion in this study is influenced by the information propagation and aggregation mechanism of GAT [28], where i is a central node and j is a neighboring node. Initially, $h_i^0=e_i$ and $h_j^0=e_j$. The graph convolution process is shown in Formulas (5)–(7):

Aggregation:

$$\begin{array}{cc}\hfill n_i^{\left(l\right)}& =\sum _{j\in N_i}{\alpha }_{ij}^{\left(l\right)}{h}_j^{\left(l\right)},\hfill \end{array}$$

(5)

$$\begin{array}{cc}\hfill {\alpha }_{ij}^{\left(l\right)}& ={\displaystyle \frac{exp\left(\sigma \left({\left(W^{\left(l\right)}{h}_i^{\left(l\right)}\right)}^T\right)\sigma \left({\left(W^{\left(l\right)}{h}_j^{\left(l\right)}\right)}^T\right)\right)}{{\sum }_{k\in N_i}exp\left(\sigma \left({\left(W^{\left(l\right)}{h}_i^{\left(l\right)}\right)}^T\right)\sigma \left({\left(W^{\left(l\right)}{h}_k^{\left(l\right)}\right)}^T\right)\right)}},\hfill \end{array}$$

(6)

Update:

$$h_i^{(l+1)}=\delta \left(W^{\left(l\right)}{n}_i^{\left(l\right)}\right),$$

(7)

where ${\mathcal{N}}_i$ denotes the neighbors of node i, and ${\mathbf{W}}^{\left(l\right)}$ is tasked with transforming node representations during the l-th propagation. $W_i$, $W_k$, and $W_j$ are the transformation matrices, while $\sigma (·)$ represents a nonlinear activation function. A higher ${\alpha }_{ij}^l$ indicates a stronger similarity and more significant information transfer from $v_j$ to $v_i$.

After multiple rounds of graph convolution, the sampled subgraphs enable the model to capture and learn crucial information about the interactions, distances, and inferred relevancies between each item and others within the same user space in the global item graph. More interactions between two items by different users, coupled with shorter distances in each interaction, indicate a higher degree of similarity and correlation. By effectively modeling the similarity information between item pairs, the model achieves more accurate embeddings of these items. Following multiple layers of information propagation and aggregation in the enhanced graphs, the final node embeddings in the GNN are represented as ${\mathbf{Z}}_S^{\prime }\in {\mathbb{R}}^{n\times d}$ based on the augmented view ${\mathcal{G}}_S^{\prime }$. A parallel enhanced representation, ${\mathbf{Z}}_S^{″}\in {\mathbb{R}}^{1\times d}$, is derived from ${\mathcal{G}}_S^{″}$.

HyperCLR relies on user interactions with items that may contain noisy data, such as a user mistakenly clicking on an item that is not of interest to them [29]. To address this challenge, this paper can employ graph contrastive learning to ensure that representations derived from the same sequence’s enhanced graph views remain similar, while those from different sequences exhibit variability. A supplementary learning objective is developed to discern whether two views originate from the same user interaction sequence. Specifically, views from the same sequence are considered positive pairs as $\left\{\left({\mathcal{G}}_S^{\prime },{\mathcal{G}}_S^{″}\right)\mid S\in G_I\right\}$, and views from different sequences are treated as negative pairs as $\left\{\left({\mathcal{G}}_S^{\prime },{\mathcal{G}}_K^{″}\right)\mid S,K\in G_I,S\ne K\right\}$.

As illustrated in Figure 4, the Item Frequency Distance Transformation Graph (IFDG) is constructed based on the hypergraph from Figure 2 and the original user sequences. Then, two augmented subgraphs from the IFDG can be constructed using the neighbor sampling in GraphSAGE [11]. Specifically, in Figure 4, the node $v_6$ is selected as the central node (other nodes are also selected as central nodes, although they are not shown in this figure), and randomly samples their neighbors without considering the original edge weights in the IFDG. After sampling, the edges between the nodes and the weights from $G_I$ are retained, ultimately generating two augmented views of the IFDG. The similar subsequences centered on $v_6$ in the two views serve as positive sample pairs $\left({\mathcal{G}}_S^{\prime },{\mathcal{G}}_S^{″}\right)$ for contrastive learning, while subsequences centered on other nodes that significantly differ from the $v_6$ subsequences serve as negative sample pairs $\left({\mathcal{G}}_S^{\prime },{\mathcal{G}}_K^{″}\right)$ during the contrastive learning process. Contrastive learning leverages the comparison between negative and positive samples to enhance the model’s ability to distinguish between different user behaviors. When handling diverse user behaviors, the model learns feature representations of different behavior patterns from a large set of samples, using contrastive learning to distinguish between genuine behavioral patterns and incidental anomalies. In essence, the model maximizes the mutual information between positive sample pairs and minimizes the mutual information between positive and negative samples, identifying and excluding noise that is unrelated to the target task during training. The contrastive objective used to differentiate between representations from the same interaction sequence and others is detailed as shown in Formula (8):

$${\mathcal{L}}_{GCL}=\sum _{S\in G_I}-log{\displaystyle \frac{exp\left(cos\left({\mathbf{z}}_S^{\prime },{\mathbf{z}}_S^{″}\right)/\tau \right)}{{\sum }_{K\in G_I}exp\left(cos\left({\mathbf{z}}_S^{\prime },{\mathbf{z}}_K^{″}\right)/\tau \right)}}.$$

(8)

Here, the node embeddings from the final layer, ${\mathbf{Z}}_{\mathbf{S}}^{\prime }$ and ${\mathbf{Z}}_{\mathbf{S}}^{″}$, are averaged to produce vectors ${\mathbf{z}}_{\mathbf{S}}^{\prime }$ and ${\mathbf{z}}_{\mathbf{S}}^{″}$, respectively, each within ${\mathbb{R}}^{1\times d}$. The cosine similarity function $cos(·)$ measures the similarity between these vectors. The hyperparameter $\tau$ is empirically set to $0.5$ in the experiments, which can facilitate the appropriate scaling of similarity scores during the learning process.

4.3. Hypergraph Convolution and User-Personalized Interest Gating

This paper uses the hypergraph convolutional network to analyze complex inter-item relationships. This paper addresses the primary challenge of defining convolution operations on hypergraphs by effectively propagating item embeddings. Building upon the methodology presented in [19], the definition of hypergraph convolution is expressed mathematically as shown in Formula (9):

$${\mathbf{x}}_i^{(l+1)}=\sum _{j=1}^N\sum _{ϵ=1}^M{H}_{iϵ}{H}_{jϵ}{W}_{ϵϵ}{\mathbf{x}}_j^{\left(l\right)}.$$

(9)

As suggested in [18], this paper does not use nonlinear activation functions and convolutional filter parameter matrices. Here, the weight $W_{ϵϵ}$ assigned to each hyperedge is 1. The normalization of the embedding operates as shown in Formula (10):

$${\mathbf{X}}_h^{(l+1)}={\mathbf{D}}^{-1}\mathbf{HWB}{\mathbf{B}}^{-1}{\mathbf{H}}^T{\mathbf{X}}_h^{\left(l\right)}.$$

(10)

Hypergraph convolution is described as a two-stage information aggregation process to learn feature transformation in a “node–hyperedge–node” manner. Initially, the operation ${\mathbf{H}}^{\top }{\mathbf{X}}_h^{\left(l\right)}$ aggregates information from nodes to hyperedges, followed by $\mathbf{H}$, which redistributes the accumulated information back to the nodes. After L layers of hypergraph convolution, the item embeddings from each layer are averaged to produce the final item embedding ${\mathbf{X}}_h={\displaystyle \frac{1}{L+1}}{\sum }_{l=0}^L{\mathbf{X}}_h^{\left(l\right)}$.

To capture the distinctions of user local interaction sequences that encompass temporal intervals, an item temporal interval encoder [30] is introduced. Considering that the time interval length between consecutive interactions might influence the impact of the current interaction on the subsequent one, the representation of each item is combined with its internal time interval. Given a sequence $E_S^0$ and its associated timestamp sequence $T_u$, the timestamps $\{t_1,\dots ,t_{|S_u|}\}$ are transformed into a sequence of time intervals as shown in Formula (11):

$$\left\{{\lambda }_1,{\lambda }_2,\dots ,{\lambda }_{\left|S_u\right|}\right\}=\left\{0,t_2-t_1,\dots ,t_{\left|S_u\right|}-t_{\left|S_u\right|-1}\right\}.$$

(11)

As depicted in Figure 5, the relevance of each item to a user decreases over time. An item interacted with earlier is less connected to the last recommended item [17]. Consequently, a decay activation function $F\left({\lambda }_i\right)$ is designed as shown in Formula (12):

$$F\left({\lambda }_i\right)=tanh\left({\displaystyle \frac{1}{({\lambda }_i+0.0001)}}\right).$$

(12)

The intuition is that the effect of each item on the final recommended item varies with the time interval between these two interactions. This function effectively reduces the weight of older interactions to emphasize more recent ones, which can adapt to the dynamic preferences of users over time. To prevent division by zero, a small constant $0.0001$ is added to $\Delta t$. The function $tanh(·)$ maps the value of $F\left({\lambda }_i\right)$ to the $(0,1)$ range, preventing outcomes that are excessively large or small.

After capturing the time information for each item in the user sequence, this paper computes the position embedding for the time interval ${\lambda }_i$ by Equation (13) [31]:

$${pos}_i=⌊\alpha log({\displaystyle \frac{{\lambda }_i}{c}}+1)⌋,$$

(13)

where $\alpha$ and c are scaling factors, and ${\delta }_i$ is the position embedding for the i-th item in $E_S^0$, ${\delta }_i\in {\mathbb{R}}^{1\times n}$. The embedding for the t-th item in sequence $E_S^0$ is Equation (14):

$${\mathbf{x}}_t^{\ast }=tanh\left({\mathbf{W}}_t\left[{\mathbf{x}}_t,{pos}_i,F\left({\delta }_i\right)\right]+\mathbf{b}\right),$$

(14)

where ${\mathbf{W}}_t\in {\mathbb{R}}^{d\times 2d}$ and $\mathbf{b}\in {\mathbb{R}}^d$ are learnable parameters. Sequence embeddings are represented by aggregating the representations of items in the sequence. Following the strategy used in SR-GNN, the embedding of sequence $E_S^0$ is refined as shown in Formulas (15) and (16):

$${\alpha }_t={\mathbf{f}}^{\top }\sigma \left({\mathbf{W}}_{as}{\mathbf{x}}_s^{\ast }+{\mathbf{W}}_{at}{\mathbf{x}}_t^{\ast }+\mathbf{c}\right),$$

(15)

$$p_u=\sum _{t=1}^m{\alpha }_t{\mathbf{x}}_t^{\ast },$$

(16)

where ${\mathbf{x}}_s^{\ast }$ is the high-order item embedding from $E_S^0$ after hypergraph convolution, which was obtained by averaging the embeddings of items it contains, ${\mathbf{x}}_s^{\ast }={\displaystyle \frac{1}{m}}{\sum }_{t=1}^m{\mathbf{x}}_m$. The user’s general interest embedding $p_u$ in the whole sequence is represented by a soft attention mechanism that aggregates item embeddings, and different items have different priorities. $\mathbf{f}\in {\mathbb{R}}^d$, ${\mathbf{W}}_{\alpha s}\in {\mathbb{R}}^{d\times d}$, and ${\mathbf{W}}_{\alpha t}\in {\mathbb{R}}^{d\times d}$ are attention parameters used to learn the weights ${\alpha }_t$.

The authors of [32] design a user-specific gating mechanism to capture global context information tailored to a user’s personalized preferences as shown in Formula (17):

$${\mathbf{Q}}_S^{\prime }={\mathbf{Z}}_S^{\prime }\otimes \sigma \left({\mathbf{Z}}_S^{\prime }{\mathbf{W}}_{g1}+{\mathbf{W}}_{g2}{p}_u^T\right),$$

(17)

where ${\mathbf{W}}_{g1}\in {\mathbb{R}}^{d\times 1}$ and ${\mathbf{W}}_{g2}\in {\mathbb{R}}^{L\times d}$, $\sigma (·)$ represent the sigmoid function, and ⊗ denotes element-wise multiplication. The user embedding $p_u^T$ describes the user’s general preferences. Similarly, this paper can obtain ${\mathbf{Q}}_S^{″}$ for the enhanced view ${\mathcal{G}}_S^{″}$.

4.4. Attention Layer

In this work, in addition to the advanced graph-enhanced sequence representations, we use traditional sequence modeling techniques to encode users’ interaction sequences. Specifically, a set of stacked Transformer encoders [31] is designed to effectively model these sequences. The transformation process for each layer, denoted as ℓ, begins with the node representations from the previous layer ${\mathbf{H}}^{\ell -1}$. The output for the ℓ-th layer of the Transformer encoder is computed as shown in Formulas (18) and (19):

$$H^{\ell }=\mathrm{FFN}\left(\mathrm{Concate}(hea{d}_1,\dots ,hea{d}_n)W^h\right),$$

(18)

$$hea{d}_i=\mathrm{Attention}(H^{\ell -1}{W}_i^Q,H^{\ell -1}{W}_i^K,H^{\ell -1}{W}_i^V).$$

(19)

Here, $FFN(·)$ denotes a feedforward network; h represents the number of heads; ${\mathbf{W}}_i^Q$, ${\mathbf{W}}_i^K$, and ${\mathbf{W}}_i^V$ are projection matrices in ${\mathbb{R}}^{d\times d/h}$ and ${\mathbf{W}}^h\in {\mathbb{R}}^{d\times d}$. The attention mechanism is defined as shown in Formula (20):

$$Attention(\mathbf{Q},\mathbf{K},\mathbf{V})=softmax\left({\displaystyle \frac{{\mathbf{QK}}^{\top }}{\sqrt{d}}}\right)\mathbf{V},$$

(20)

where $\mathbf{Q}$, $\mathbf{K}$, and $\mathbf{V}$ correspond to the query, key, and value matrices, respectively, with $\sqrt{d}$ serving as a normalization factor. Moreover, the representations derived from the graph views, ${\mathbf{Q}}_S^{\prime }$ and ${\mathbf{Q}}_S^{″}$, will be integrated with the initial embeddings of the user interaction sequence items ${\mathbf{E}}_S^{\left(0\right)}$ as shown in Formula (21):

$$\mathbf{M}=AttNet\left(Concat\left({\mathbf{Q}}_S^{\prime },{\mathbf{Q}}_S^{″},{\mathbf{E}}_S^{\left(0\right)}\right)\right),$$

(21)

where $\mathrm{AttNet}(·)$ denotes the final vector representation produced by the attention network after layer stacking and $\mathbf{M}\in {\mathbb{R}}^{1\times d}$.

4.5. Prediction Layer

The probability of interaction between a user and items at step $n+1$, following a user interaction sequence $S_u$ of length n, can be calculated as shown in Formula (22), where $\widehat{y}\in {\mathbb{R}}^{1\times \left|V\right|}$:

$$\widehat{y}=\mathrm{softmax}\left(\mathbf{M}{\mathbf{E}}^{\top }\right),$$

(22)

where $\mathbf{M}$ represents the final embedding matrix derived from the attention network and $\mathbf{E}$ stands for the item embedding matrix. The softmax function transforms the scores into probabilities to predict of the user’s likelihood of interacting with each item at the next time. The model is optimized during the training phase using a binary cross-entropy loss function as shown in Equation (23):

$${\mathcal{L}}_{\mathrm{main}}=-\sum _{i=1}^m{y}_{i}log\left({\widehat{y}}_i\right)+(1-y_i)log(1-{\widehat{y}}_i).$$

(23)

To further enhance the learning process, we adopt contrastive learning in HyperCLR, which leverages sequence comparison to obtain more robust representations. The whole loss function combining both learning tasks is expressed as shown in Equation (24):

$${\mathcal{L}}_{total}={\mathcal{L}}_{\mathrm{main}}+{\lambda }_1{\mathcal{L}}_{GCL}.$$

(24)

Here, ${\mathcal{L}}_{\mathrm{main}}$ and ${\mathcal{L}}_{GCL}$ deal with the sequential recommendation task and the graph contrastive learning task, respectively, and ${\lambda }_1$ is a hyperparameter that balances the two tasks.

5. Experiment

5.1. Experimental Settings

5.1.1. Datasets

The performance of HyperCLR can be assessed on three real-world datasets:

• Taobao [33]: It is a set of data collected from user browsing and purchasing activities across Tmall, Taobao, and Alipay from 1 July 2015, to 30 November 2015. For analysis, a random one-eighth sample of all purchase records was selected.
• Books and Games [34]: These are two datasets from the Amazon website that contain a large amount of interaction data between users and different categories of items, primarily focusing on user purchasing behavior.

For each dataset, redundant interactions per user were eliminated, and items were organized chronologically. To ensure data quality, user sequences with fewer than five interactions and items appearing fewer than five times were excluded [35]. In these experiments, the last item interacted with by each user was designated for testing, the penultimate item for validation, and all preceding items for training [36]. The statistical results are summarized in

Table 1. Dataset statistics.

Dataset	Users	Projects	Actions	Sparsity
Taobao	75,282	189,671	1,168,594	99.99%
Books	52,406	41,264	1,856,747	99.91%
Games	31,013	23,715	287,107	99.96%

.

5.1.2. Evaluation Metrics

To evaluate the effectiveness of the HyperCLR method, we chose two widely used metrics in sequential recommendations: Hit Rate (HR) and Normalized Discounted Cumulative Gain (NDCG).

(1)

HR@N: It measures whether the items in users’ Top-N predicted recommendations occur in users’ interactions, which reflects the accuracy of the model.

(2)

NDCG@N: NDCG measures whether the items of actual interest to the user are ranked higher, emphasizing the “order” of the predictions and the importance of ranking accuracy.

This paper focuses on HR@1,10 and NDCG@10. NDCG@1 is not considered because it has the same value as Hit@1. To manage the large item set efficiently, we constructed a testing pool of 100 items by pairing one genuine item with 99 randomly selected negative items for evaluation, enhancing computational efficiency while maintaining assessment precision.

5.1.3. Compared Baselines

This paper compared HyperCLR with the following sequential recommendation methods:

• GRU4Rec [37]: GRU4Rec employs Gated Recurrent Units (GRUs), a variant of Recurrent Neural Networks (RNNs), to efficiently process sessions in small batches and execute output sampling.
• Caser [13]: Caser utilizes Convolutional Neural Networks (CNNs) to analyze users’ historical behavior sequences. It achieves a comprehensive understanding of user behavior patterns through the implementation of horizontal and vertical convolutional layers.
• SASRec [14]: SASRec adopts a self-attention mechanism to assign weights unidirectionally to all preceding user interactions, dynamically capturing the relevance among items within the sequence.
• HGN [38]: The Hierarchical Gated Network (HGN) utilizes a multi-layer graph structure equipped with gated networks to enhance sequential recommendations, effectively managing item information across various scales.
• LightGCN [39]: LightGCN simplifies Graph Convolutional Networks (GCNs) by removing nonlinear activations and feature transformations, reducing model complexity and improving recommendation performance.
• CL4SRec [20]: CL4SRec integrates contrastive learning into sequential recommendations by creating positive and negative sample pairs, which helps with accurately learning and distinguishing user behavior patterns.
• HyperRec [40]: HyperRec constructs hypergraphs using timestamps and extracts short-term item correlations. It fuses dynamic item embeddings with short-term user intentions to model user preferences more accurately.
• APGL4SR [21]: APGL4SR incorporates personalized global information into the recommendation model, significantly improving its capability to customize recommendations according to individual user preferences.

5.1.4. Setup and Implementation Details

These experiments are implemented with an Intel CORE i7 10875H processor and an Nvidia RTX 2060 graphics card. The code of HyperCLR is based on Pycharm 2022 and PyTorch 1.7. For fair comparison, we used the original code of the benchmarks and adopt the optimal parameter settings suggested in the literature. Unless otherwise stated, the default settings are an embedding dimension of 128, a sampling depth of 4, five graph convolution layers, a dropout rate of $0.2$, and a learning rate of $0.1$.

5.2. Overall Performance Comparison

Table 2. Experimental results on datasets for different models.

Dataset	Metric	GRU4Rec	Caser	SASRec	HGN	LightGCN	CL4SRec	HyperRec	APGL4SR	HyperCLR
Taobao	HR@1	0.0246	0.0281	0.0352	0.0436	0.0437	0.0521	0.0591	0.0568	0.0763
	HR@10	0.0582	0.0673	0.0927	0.1278	0.1506	0.1830	0.2057	0.2234	0.2364
	NDCG@10	0.0437	0.0481	0.0586	0.0863	0.1164	0.1540	0.1839	0.2064	0.2321
Books	HR@1	0.0148	0.0265	0.0389	0.0319	0.0468	0.0397	0.0621	0.0869	0.0921
	HR@10	0.0634	0.0748	0.0783	0.1082	0.1490	0.1623	0.1843	0.1907	0.2078
	NDCG@10	0.0785	0.1092	0.1453	0.1857	0.2309	0.2530	0.2743	0.2987	0.3154
Games	HR@1	0.0249	0.0329	0.0384	0.0472	0.0518	0.0549	0.0597	0.0698	0.0759
	HR@10	0.0797	0.0919	0.1395	0.1792	0.2069	0.2398	0.2673	0.2930	0.3101
	NDCG@10	0.0619	0.0895	0.0951	0.1432	0.1873	0.2109	0.2319	0.2638	0.2759

illustrates the performance of all methods. The best results are marked in bold, and the second best results are underlined. Models based on Recurrent Neural Networks (RNNs) and Convolutional Neural Networks (CNNs), such as GRU4Rec and Caser, outperform models employing self-attention mechanisms (e.g., SASRec). The latter’s superior performance is mainly due to the self-attention mechanism’s ability to effectively capture dependencies between items, including those that are temporally far apart. GNN-based models (e.g., HGN and LightGCN) outperform SASRec’s results, demonstrating the benefits of Graph Neural Networks in sequential recommendation tasks. CL4SRec improves these models by augmenting sequence representations by exploiting the local context in individual sequences, suggesting that a self-supervised learning objective can help to improve the performance of sequential recommendations. However, CL4SRec does not perform as well as APGL4SR. This discrepancy arises because CL4SRec focuses primarily on local sequence contexts without integrating broader item transition patterns.

In contrast, HyperCLR outperforms all of the baseline methods on various datasets. It employs a novel approach by using subgraphs from the transition graph constructed from all users’ sequences, which helps with learning item relevancy and enhancing sequence representations. This approach not only provides a rich mix of global and local context information for model learning but also dynamically incorporates the time and location embeddings of items. Furthermore, HyperCLR utilizes hypergraph modeling to capture high-order personalized preference relationships between users. This comprehensive model framework allows HyperCLR to effectively harness specific global context information based on user personalized preferences, achieving superior performance across the board.

5.3. Ablation Experiments

In this subsection, we investigate the impact of three pivotal components in the HyperCLR model: the IFDG graph construction, graph convolution combined with contrastive learning, and the personalized interest gating mechanism. With HR@10 and NDCG@10 as metrics, the detailed results on the Taobao dataset are presented in

Table 3. Performance of HyperCLR compared with its variations.

Model	HR@10	NDCG@10
HyperCLR-1	0.2017	0.2168
HyperCLR-2	0.2214	0.2197
HyperCLR-3	0.2159	0.2239
HyperCLR	0.2364	0.2321

.

The three variants of HyperCLR designed are as follows:

(1)

HyperCLR-1: This variant deletes the edge weights in the IFDG graph, facilitating the evaluation of their role in capturing the subtleties of item interactions within the graph.

(2)

HyperCLR-2: In this configuration, the graph contrastive learning loss is removed by setting ${\lambda }_1$ to zero, and the GAT (Graph Attention Network) convolution module is removed. This approach examines the significance of both contrastive loss and attention-based graph convolutions in boosting recommendation performance.

(3)

HyperCLR-3: This version excludes both the hypergraph convolution and the personalized interest gating mechanism, allowing for an assessment of their contributions to the model’s overall effectiveness.

From Table 3, the best results are marked in bold. It can be discovered that HyperCLR-1, which lacks edge weighting in the IFDG graph, shows a slight decrease in both Hit Rate (HR) and Normalized Discounted Cumulative Gain (NDCG). This suggests that item transition frequencies across sequences can help differentiate the importance of neighboring items, thereby improving the performance of recommendations. It also can be found that HyperCLR outperforms HyperCLR-2, indicating the substantial roles of graph convolution and contrastive learning loss in refining item and sequence representations for recommendations. Moreover, HyperCLR exceeds HyperCLR-3 across all metrics, affirming that leveraging contextual information about users’ personalized preferences markedly boosts sequential recommendation performance. This highlights the essential role of personalized interest gating mechanisms in understanding user preferences and customizing recommendations, which enhances both accuracy and relevance for users.

5.4. Hyperparameter Analysis

When evaluating the effect of a specific parameter, this paper sets all other parameters to the optimal hyperparameter. In this section, we evaluate the effect of the item embedding dimension d, sampling depth M, and the number of GAT convolution layers on the performance of the HyperCLR model.

5.4.1. Analysis of Embedding Dimension d

This paper explores the influence of the embedding dimension d on the Taobao dataset. The results, illustrated in Figure 6, demonstrate that the model’s performance increases with the embedding dimension up to $d=128$ beyond which it either stabilizes or slightly declines. This suggests that a larger embedding dimension does not invariably enhance performance. It means that there exists an optimal dimension of $d=128$, beyond which the risk of overfitting might diminish returns.

5.4.2. Analysis of Sampling Depth

This paper analyzes the effects of various sampling depths on the performance of the HyperCLR model. Sampling depths of $M=\{1,2,3,4,5\}$ are tested across different datasets to assess their NDCG@10 performance, as shown in Figure 7.

The results in Figure 7 indicate that a minimal sampling depth is insufficient for capturing complex high-order item interactions. Optimal performance is noted at a sampling depth of $M=4$, which effectively encapsulates comprehensive user–item relationship dynamics, thus improving the model’s scalability. Nonetheless, an excessive increase in sampling depth can introduce noise from loosely related nodes, potentially degrading performance.

5.4.3. GAT Convolution Layer Analysis

This paper delves into the effects of varying the number of GAT convolution layers on the HyperCLR model. The performance across three datasets is evaluated with layer counts from 1 to 6, using NDCG@10 as the metric. The outcomes, depicted in Figure 8, reveal that increasing the number of layers initially boosts the model’s performance, which peaks with 5 layers. This setting appears to optimally balance multi-order neighbor information for effective graph node embedding learning, whereas excessive layers may induce training instability and overfitting risks.

5.5. Discussion

Although the personalized sequential recommendation algorithm based on hypergraph and contrastive learning proposed in this paper outperforms baseline models in terms of performance, it still has several limitations. First, the construction and optimization of the hypergraph structure are relatively complex. This complexity, particularly when dealing with large-scale datasets, results in high computational costs, potentially limiting the algorithm’s efficiency and scalability in real-world applications. Second, while contrastive learning is effective at capturing useful features in the data, its performance is highly dependent on the selection and sampling strategy of negative samples. An improper selection of negative samples could lead to overfitting or a decline in model performance. Moreover, HyperCLR is primarily optimized for sequential recommendation tasks. Its effectiveness for other types of recommendation tasks, such as session-based or social recommendation, may require further investigation and adjustments. Therefore, future research could focus on simplifying the hypergraph construction process, optimizing negative sample sampling strategies, and exploring the adaptability of this algorithm across different recommendation scenarios.

6. Conclusions

This paper presents HyperCLR, which dynamically incorporates the time and location embeddings of items and utilizes hypergraph modeling to capture high-order relationships of personalized user preferences. HyperCLR designs the IFDG framework to create the Global Item Frequency Distance Graph (IFDG), making the model comprehensively assess item relevancy. Building on this, HyperCLR utilizes contrastive learning to ensure maximal consistency within identical interaction sequences while minimizing discrepancies between global context representations on IFDG and local sequence representations. The effectiveness of sequential recommendation is significantly increased this way. A specialized gating mechanism is used to collect unique global contextual information from each user based on the user’s preferences. Rigorous testing across diverse datasets has consistently revealed that HyperCLR outperforms baselines on multiple performance indicators, affirming the efficacy of this approach. In the future, it is possible to integrate broader user behavior data into HyperCLR to advance sequential recommendations, providing more precise and tailored services to users.